Update prom/prometheus Docker tag to v3.11.0 #12

Open
renovate-bot wants to merge 1 commit from renovate/prom-prometheus-3.x into main
Collaborator

This PR contains the following updates:

Package Update Change
prom/prometheus minor v3.5.0v3.11.0

Release Notes

prometheus/prometheus (prom/prometheus)

v3.11.0: 3.11.0 / 2026-04-02

Compare Source

  • [CHANGE] Hetzner SD: The __meta_hetzner_datacenter label is deprecated for the role robot but kept for backward compatibility, use the __meta_hetzner_robot_datacenter label instead. For the role hcloud, the label is deprecated and will stop working after the 1 July 2026. #​17850
  • [CHANGE] Hetzner SD: The __meta_hetzner_hcloud_datacenter_location and __meta_hetzner_hcloud_datacenter_location_network_zone labels are deprecated, use the __meta_hetzner_hcloud_location and __meta_hetzner_hcloud_location_network_zone labels instead. #​17850
  • [CHANGE] Promtool: Redirect debug output to stderr to avoid interfering with stdout-based tool output. #​18346
  • [FEATURE] AWS SD: Add Elasticache Role. #​18099
  • [FEATURE] AWS SD: Add RDS Role. #​18206
  • [FEATURE] Azure SD: Add support for Azure Workload Identity authentication method. #​17207
  • [FEATURE] Discovery: Introduce prometheus_sd_last_update_timestamp_seconds metric to track the last time a service discovery update was sent to consumers. #​18194
  • [FEATURE] Kubernetes SD: Add support for node role selectors for pod roles. #​18006
  • [FEATURE] Kubernetes SD: Introduce pod-based labels for deployment, cronjob, and job controller names: __meta_kubernetes_pod_deployment_name, __meta_kubernetes_pod_cronjob_name and __meta_kubernetes_pod_job_name, respectively. #​17774
  • [FEATURE] PromQL: Add </ and >/ operators for trimming observations from native histograms. #​17904
  • [FEATURE] PromQL: Add experimental histogram_quantiles variadic function for computing multiple quantiles at once. #​17285
  • [FEATURE] TSDB: Add storage.tsdb.retention.percentage configuration to configure the maximum percent of disk usable for TSDB storage. #​18080
  • [FEATURE] TSDB: Add an experimental fast-startup feature flag that writes a series_state.json file to the WAL directory to track active series state across restarts. #​18303
  • [FEATURE] TSDB: Add an experimental st-storage feature flag. When enabled, Prometheus stores ingested start timestamps (ST, previously called Created Timestamp) from scrape or OTLP in the TSDB and Agent WAL, and exposes them via Remote Write 2. #​18062
  • [FEATURE] TSDB: Add an experimental xor2-encoding feature flag for the new TSDB block float sample chunk encoding that is optimized for scraped data and allows encoding start timestamps. #​18062
  • [ENHANCEMENT] HTTP client: Add AWS external_id support for sigv4. #​17916
  • [ENHANCEMENT] Kubernetes SD: Deduplicate deprecation warning logs from the Kubernetes API to reduce noise. #​17829
  • [ENHANCEMENT] TSDB: Remove old temporary checkpoints when creating a Checkpoint. #​17598
  • [ENHANCEMENT] UI: Add autocomplete support for experimental first_over_time and ts_of_first_over_time PromQL functions. #​18318
  • [ENHANCEMENT] Vultr SD: Upgrade govultr library from v2 to v3 for continued security patches and maintenance. #​18347
  • [PERF] PromQL: Improve performance and reduce heap allocations in joins (VectorBinop)/And/Or/Unless. #​17159
  • [PERF] PromQL: Partially address performance regression in native histogram aggregations due to using KahanAdd. #​18252
  • [PERF] Remote write: Optimize WAL watching used for RW sending to reuse internal buffers. #​18250
  • [PERF] TSDB: Optimize LabelValues intersection performance for matchers. #​18069
  • [PERF] UI: Skip restacking on hover in stacked series charts. #​18230
  • [BUGFIX] AWS SD: Fix EC2 SD ignoring the configured endpoint option, a regression from the AWS SDK v2 migration. #​18133
  • [BUGFIX] AWS SD: Fix panic in EC2 SD when DescribeAvailabilityZones returns nil ZoneName or ZoneId. #​18133
  • [BUGFIX] Agent: Fix memory leak caused by duplicate SeriesRefs being loaded as active series. #​17538
  • [BUGFIX] Alerting: Fix alert state incorrectly resetting to pending when the FOR period is increased in the config file. #​18244
  • [BUGFIX] Azure SD: Fix system-assigned managed identity not working when client_id is empty. #​18323
  • [BUGFIX] Consul SD: Fix filter parameter not being applied to health service endpoint, causing Node and Node.Meta filters to be ignored. #​17349
  • [BUGFIX] Kubernetes SD: Fix duplicate targets generated by *DualStack EndpointSlices policies. #​18192
  • [BUGFIX] OTLP: Fix ErrTooOldSample being returned as HTTP 500 instead of 400 in PRW v2 histogram write paths, preventing infinite client retry loops. #​18084
  • [BUGFIX] OTLP: Fix exemplars getting mixed between incorrect parts of a histogram. #​18056
  • [BUGFIX] PromQL: Do not skip histogram buckets in queries where histogram trimming is used. #​18263
  • [BUGFIX] Remote write: Fix prometheus_remote_storage_sent_batch_duration_seconds measuring before the request was sent. #​18214
  • [BUGFIX] Rules: Fix alert state restoration when rule labels contain Go template expressions. #​18375
  • [BUGFIX] Scrape: Fix panic when parsing bare label names without an equal sign in brace-only metric notation. #​18229
  • [BUGFIX] TSDB: Fail early when use-uncached-io feature flag is set on unsupported environments. #​18219
  • [BUGFIX] TSDB: Fall back to CLI flag values when retention is removed from config file. #​18200
  • [BUGFIX] TSDB: Fix memory leaks in buffer pools by clearing reference fields before returning buffers to pools. #​17895
  • [BUGFIX] TSDB: Fix missing mmap of histogram chunks during WAL replay. #​18306
  • [BUGFIX] TSDB: Fix storage.tsdb.retention.time unit mismatch in file causing retention to be 1e6 times longer than configured. #​18200
  • [BUGFIX] Tracing: Fix missing traceID in query log when tracing is enabled, previously only spanID was emitted. #​18189
  • [BUGFIX] UI: Fix tooltip Y-offset drift when using multiple graph panels. #​18228
  • [BUGFIX] UI: Update retention display in runtime info when config is reloaded. #​18200

v3.10.0: 3.10.0 / 2026-02-24

Compare Source

Prometheus now offers a distroless Docker image variant alongside the default
busybox image. The distroless variant provides enhanced security with a minimal
base image, uses UID/GID 65532 (nonroot) instead of nobody, and removes the
VOLUME declaration. Both variants are available with -busybox and -distroless
tag suffixes (e.g., prom/prometheus:latest-busybox, prom/prometheus:latest-distroless).
The busybox image remains the default with no suffix for backwards compatibility
(e.g., prom/prometheus:latest points to the busybox variant).

For users migrating existing named volumes from the busybox image to the distroless variant, the ownership can be adjusted with:

docker run --rm -v prometheus-data:/prometheus alpine chown -R 65532:65532 /prometheus

Then, the container can be started with the old volume with:

docker run -v prometheus-data:/prometheus prom/prometheus:latest-distroless

User migrating from bind mounts might need to ajust permissions too, depending on their setup.

  • [CHANGE] Alerting: Add alertmanager dimension to following metrics: prometheus_notifications_dropped_total, prometheus_notifications_queue_capacity, prometheus_notifications_queue_length. #​16355
  • [CHANGE] UI: Hide expanded alert annotations by default, enabling more information density on the /alerts page. #​17611
  • [FEATURE] AWS SD: Add MSK Role. #​17600
  • [FEATURE] PromQL: Add fill() / fill_left() / fill_right() binop modifiers for specifying default values for missing series. #​17644
  • [FEATURE] Web: Add OpenAPI 3.2 specification for the HTTP API at /api/v1/openapi.yaml. #​17825
  • [FEATURE] Dockerfile: Add distroless image variant using UID/GID 65532 and no VOLUME declaration. Busybox image remains default. #​17876
  • [FEATURE] Web: Add on-demand wall time profiling under <URL>/debug/pprof/fgprof. #​18027
  • [ENHANCEMENT] PromQL: Add more detail to histogram quantile monotonicity info annotations. #​15578
  • [ENHANCEMENT] Alerting: Independent alertmanager sendloops. #​16355
  • [ENHANCEMENT] TSDB: Experimental support for early compaction of stale series in the memory with configurable threshold stale_series_compaction_threshold in the config file. #​16929
  • [ENHANCEMENT] Service Discovery: Service discoveries are now removable from the Prometheus binary through the Go build tag remove_all_sd and individual service discoveries can be re-added with the build tags enable_<sd name>_sd. Users can build a custom Prometheus with only the necessary SDs for a smaller binary size. #​17736
  • [ENHANCEMENT] Promtool: Support promql syntax features promql-duration-expr and promql-extended-range-selectors. #​17926
  • [PERF] PromQL: Avoid unnecessary label extraction in PromQL functions. #​17676
  • [PERF] PromQL: Improve performance of regex matchers like .*-.*-.*. #​17707
  • [PERF] OTLP: Add label caching for OTLP-to-Prometheus conversion to reduce allocations and improve latency. #​17860
  • [PERF] API: Compute /api/v1/targets/relabel_steps in a single pass instead of re-running relabeling for each prefix. #​17969
  • [PERF] tsdb: Optimize LabelValues intersection performance for matchers. #​18069
  • [BUGFIX] PromQL: Prevent query strings containing only UTF-8 continuation bytes from crashing Prometheus. #​17735
  • [BUGFIX] Web: Fix missing X-Prometheus-Stopping header for /-/ready endpoint in NotReady state. #​17795
  • [BUGFIX] PromQL: Fix PromQL info() function returning empty results when filtering by a label that exists on both the input metric and target_info. #​17817
  • [BUGFIX] TSDB: Fix a bug during exemplar buffer grow/shrink that could cause exemplars to be incorrectly discarded. #​17863
  • [BUGFIX] UI: Fix broken graph display after page reload, due to broken Y axis min encoding/decoding. #​17869
  • [BUGFIX] TSDB: Fix memory leaks in buffer pools by clearing reference fields (Labels, Histogram pointers, metadata strings) before returning buffers to pools. #​17879
  • [BUGFIX] PromQL: info function: fix series without identifying labels not being returned. #​17898
  • [BUGFIX] OTLP: Filter __name__ from OTLP attributes to prevent duplicate labels. #​17917
  • [BUGFIX] TSDB: Fix division by zero when computing stale series ratio with empty head. #​17952
  • [BUGFIX] OTLP: Fix potential silent data loss for sum metrics. #​17954
  • [BUGFIX] PromQL: Fix smoothed interpolation across counter resets. #​17988
  • [BUGFIX] PromQL: Fix panic with @ modifier on empty ranges. #​18020
  • [BUGFIX] PromQL: Fix avg_over_time for a single native histogram. #​18058

v3.9.1: 3.9.1 / 2026-01-07

Compare Source

  • [BUGFIX] Agent: fix crash shortly after startup from invalid type of object. #​17802
  • [BUGFIX] Scraping: fix relabel keep/drop not working. #​17807

v3.9.0: 3.9.0 / 2026-01-06

Compare Source

Note for users of Native Histograms

In version 3.9, Native Histograms is no longer experimental, and the feature flag native-histogram has no effect. You must now turn on
the config setting scrape_native_histograms to collect Native Histogram samples from exporters.

Changelog

  • [CHANGE] Native Histograms are no longer experimental! Make the native-histogram feature flag a no-op. Use scrape_native_histograms config option instead. #​17528
  • [CHANGE] API: Add maximum limit of 10,000 sets of statistics to TSDB status endpoint. #​17647
  • [FEATURE] API: Add /api/v1/features for clients to understand which features are supported. #​17427
  • [FEATURE] Promtool: Add start_timestamp field for unit tests. #​17636
  • [FEATURE] Promtool: Add --format seriesjson option to tsdb dump to output just series labels in JSON format. #​13409
  • [FEATURE] Add --storage.tsdb.delay-compact-file.path flag for better interoperability with Thanos. #​17435
  • [FEATURE] UI: Add an option on the query drop-down menu to duplicate that query panel. #​17714
  • [ENHANCEMENT]: TSDB: add flag --storage.tsdb.block-reload-interval to configure TSDB Block Reload Interval. #​16728
  • [ENHANCEMENT] UI: Add graph option to start the chart's Y axis at zero. #​17565
  • [ENHANCEMENT] Scraping: Classic protobuf format no longer requires the unit in the metric name. #​16834
  • [ENHANCEMENT] PromQL, Rules, SD, Scraping: Add native histograms to complement existing summaries. #​17374
  • [ENHANCEMENT] Notifications: Add a histogram prometheus_notifications_latency_histogram_seconds to complement the existing summary. #​16637
  • [ENHANCEMENT] Remote-write: Add custom scope support for AzureAD authentication. #​17483
  • [ENHANCEMENT] SD: add a config label with job name for most prometheus_sd_refresh metrics. #​17138
  • [ENHANCEMENT] TSDB: New histogram prometheus_tsdb_sample_ooo_delta, the distribution of out-of-order samples in seconds. Collected for all samples, accepted or not. #​17477
  • [ENHANCEMENT] Remote-read: Validate histograms received via remote-read. #​17561
  • [PERF] TSDB: Small optimizations to postings index. #​17439
  • [PERF] Scraping: Speed up relabelling of series. #​17530
  • [PERF] PromQL: Small optimisations in binary operators. #​17524, #​17519.
  • [BUGFIX] UI: PromQL autocomplete now shows the correct type and HELP text for OpenMetrics counters whose samples end in _total. #​17682
  • [BUGFIX] UI: Fixed codemirror-promql incorrectly showing label completion suggestions after the closing curly brace of a vector selector. #​17602
  • [BUGFIX] UI: Query editor no longer suggests a duration unit if one is already present after a number. #​17605
  • [BUGFIX] PromQL: Fix some "vector cannot contain metrics with the same labelset" errors when experimental delayed name removal is enabled. #​17678
  • [BUGFIX] PromQL: Fix possible corruption of PromQL text if the query had an empty ignoring() and non-empty grouping. #​17643
  • [BUGFIX] PromQL: Fix resets/changes to return empty results for anchored selectors when all samples are outside the range. #​17479
  • [BUGFIX] PromQL: Check more consistently for many-to-one matching in filter binary operators. #​17668
  • [BUGFIX] PromQL: Fix collision in unary negation with non-overlapping series. #​17708
  • [BUGFIX] PromQL: Fix collision in label_join and label_replace with non-overlapping series. #​17703
  • [BUGFIX] PromQL: Fix bug with inconsistent results for queries with OR expression when experimental delayed name removal is enabled. #​17161
  • [BUGFIX] PromQL: Ensure that rate/increase/delta of histograms results in a gauge histogram. #​17608
  • [BUGFIX] PromQL: Do not panic while iterating over invalid histograms. #​17559
  • [BUGFIX] TSDB: Reject chunk files whose encoded chunk length overflows int. #​17533
  • [BUGFIX] TSDB: Do not panic during resolution reduction of invalid histograms. #​17561
  • [BUGFIX] Remote-write Receive: Avoid duplicate labels when experimental type-and-unit-label feature is enabled. #​17546
  • [BUGFIX] OTLP Receiver: Only write metadata to disk when experimental metadata-wal-records feature is enabled. #​17472

v3.8.1: 3.8.1 / 2025-12-16

Compare Source

  • [BUGFIX] remote: Fix Remote Write receiver, so it does not send wrong response headers for v1 flow and cause Prometheus senders to emit false partial error log and metrics. #​17683

v3.8.0: 3.8.0 / 2025-11-28

Compare Source

Note for users of Native Histograms

This is the first release with Native Histograms as a stable feature. However, scraping Native Histograms has to be activated explicitly via the scrape_native_histograms config setting (newly introduced in this release). To ease the transition, the --enable-feature=native-histograms flag is not a complete no-op in this release, but changes the default value of scrape_native_histograms to true. In the next release (v3.9), the feature flag will be a complete no-op, and the default value of scrape_native_histograms will always be false. If you have been using the feature flag so far, the recommended course of action is the following:

  1. Upgrade to v3.8 and keep the feature flag. Everything should work as before.
  2. At your own pace, set scrape_native_histograms to true in all relevant scrape configs. (There is a global and a per-scrape-config version of scrape_native_histograms, allowing granular control if needed. It is a good idea to also set scrape_native_histograms explicitly to false where you do not want to scrape Native Histograms. In this way, you do not depend on the default value of the setting anymore.)
  3. Remove the feature flag and make sure that everything still works as intended.
  4. Now you are ready for an upgrade to the next release (v3.9).

Changelog

  • [CHANGE] Remote-write 2 (receiving): Update to 2.0-rc.4 spec. "created timestamp" (CT) is now called "start timestamp" (ST). #​17411
  • [CHANGE] TSDB: Native Histogram Custom Bounds with a NaN threshold are now rejected. #​17287
  • [FEATURE] OAuth2: support jwt-bearer grant-type (RFC7523 3.1). #​17592
  • [FEATURE] Dockerfile: Add OpenContainers spec labels to Dockerfile. #​16483
  • [FEATURE] SD: Add unified AWS service discovery for ec2, lightsail and ecs services. #​17046
  • [FEATURE] Native histograms are now a stable, but optional feature, use the scrape_native_histograms config setting. #​17232 #​17315
  • [FEATURE] UI: Support anchored and smoothed keyword in promql editor. #​17239
  • [FEATURE] UI: Show detailed relabeling steps for each discovered target. #​17337
  • [FEATURE] Alerting: Add urlQueryEscape to template functions. #​17403
  • [FEATURE] Promtool: Add Remote-Write 2.0 support to promtool push metrics via the --protobuf_message flag. #​17417
  • [ENHANCEMENT] Clarify the docs about handling negative native histograms. #​17249
  • [ENHANCEMENT] Mixin: Add static UID to the remote-write dashboard. #​17256
  • [ENHANCEMENT] PromQL: Reconcile mismatched NHCB bounds in Add and Sub. #​17278
  • [ENHANCEMENT] Alerting: Add "unknown" state for alerting rules that haven't been evaluated yet. #​17282
  • [ENHANCEMENT] Scrape: Allow simultaneous use of classic histogram → NHCB conversion and zero-timestamp ingestion. #​17305
  • [ENHANCEMENT] UI: Add smoothed/anchored in explain. #​17334
  • [ENHANCEMENT] OTLP: De-duplicate any target_info samples with the same timestamp for the same series. #​17400
  • [ENHANCEMENT] Document use_fips_sts_endpoint in sigv4 config sections. #​17304
  • [ENHANCEMENT] Document Prometheus Agent. #​14519
  • [PERF] PromQL: Speed up parsing of variadic functions. #​17316
  • [PERF] UI: Speed up alerts/rules/... pages by not rendering collapsed content. #​17485
  • [PERF] UI: Performance improvement when getting label name and values in promql editor. #​17194
  • [PERF] UI: Speed up /alerts for many firing alerts via virtual scrolling. #​17254
  • [BUGFIX] PromQL: Fix slice indexing bug in info function on churning series. #​17199
  • [BUGFIX] API: Reduce lock contention on /api/v1/targets. #​17306
  • [BUGFIX] PromQL: Consistent handling of gauge vs. counter histograms in aggregations. #​17312
  • [BUGFIX] TSDB: Allow NHCB with -Inf as the first custom value. #​17320
  • [BUGFIX] UI: Fix duplicate loading of data from the API speed up rendering of some pages. #​17357
  • [BUGFIX] Old UI: Fix createExpressionLink to correctly build /graph URLs so links from Alerts/Rules work again. #​17365
  • [BUGFIX] PromQL: Avoid panic when parsing malformed info call. #​17379
  • [BUGFIX] PromQL: Include histograms when enforcing sample_limit. #​17390
  • [BUGFIX] Config: Fix panic if TLS CA file is absent. #​17418
  • [BUGFIX] PromQL: Fix histogram_fraction for classic histograms and NHCB if lower bound is in the first bucket. #​17424

v3.7.3: 3.7.3 / 2025-10-29

Compare Source

  • [BUGFIX] UI: Revert changed (and breaking) redirect behavior for -web.external-url if -web.route-prefix is configured, which was introduced in #​17240. #​17389
  • [BUGFIX] Fix federation of some native histograms. #​17299 #​17409
  • [BUGFIX] promtool: check config would fail when --lint=none flag was set. #​17399 #​17414
  • [BUGFIX] Remote-write: fix a deadlock in the queue resharding logic that can lead to suboptimal queue behavior. #​17412

v3.7.2: 3.7.2 / 2025-10-22

Compare Source

  • [BUGFIX] AWS SD: Fix AWS SDK v2 credentials handling for EC2 and Lightsail discovery. #​17355
  • [BUGFIX] AWS SD: Load AWS region from IMDS when not set. #​17376
  • [BUGFIX] Relabeling: Fix labelmap action validation with the legacy metric name validation scheme. #​17372
  • [BUGFIX] PromQL: Fix parsing failure when anchored and smoothed are used as metric names and label names. #​17353
  • [BUGFIX] PromQL: Fix formatting of range vector selectors with smoothed/anchored modifier. #​17354

v3.7.1: 3.7.1 / 2025-10-16

Compare Source

  • [BUGFIX] OTLP: Prefix key_ to label name when translating an OTel attribute name starting with a single underscore, and keep multiple consecutive underscores in label name when translating an OTel attribute name. This reverts the breaking changes introduced in 3.7.0. #​17344

v3.7.0: 3.7.0 / 2025-10-15

Compare Source

Warning

We discovered a breaking change in the OTLP endpoint: OpenTelemetry attribute names starting with underscore, for example _attrib, are no longer automatically translated to key_attrib. Please use 3.7.1 instead for OTLP ingestion.

Changelog

  • [CHANGE] Remote-write: the following metrics are deprecated:
    • prometheus_remote_storage_samples_in_total, use prometheus_wal_watcher_records_read_total{type="samples"} and prometheus_remote_storage_samples_dropped_total instead,
    • prometheus_remote_storage_histograms_in_total, use prometheus_wal_watcher_records_read_total{type=~".*histogram_samples"} and prometheus_remote_storage_histograms_dropped_total instead,
    • prometheus_remote_storage_exemplars_in_total, use prometheus_wal_watcher_records_read_total{type="exemplars"} and prometheus_remote_storage_exemplars_dropped_total instead,
    • prometheus_remote_storage_highest_timestamp_in_seconds, use the more accurate prometheus_remote_storage_queue_highest_timestamp_seconds instead in dashboards and alerts to properly account for relabeling and for more accuracy. #​17065
  • [FEATURE] PromQL: Add support for experimental anchored and smoothed rate behind feature flag promql-extended-range-selectors. #​16457
  • [FEATURE] Federation: Add support for native histograms with custom buckets (NHCB). #​17215
  • [FEATURE] PromQL: Add first_over_time(...) and ts_of_first_over_time(...) behind feature flag experimental-promql-functions. #​16963 #​17021
  • [FEATURE] Remote-write: Add support for Azure Workload Identity as an authentication method for the receiver. #​16788
  • [FEATURE] Remote-write: Add type and unit labels to outgoing time series in remote-write 2.0 when the type-and-unit-labels feature flag is enabled. #​17033
  • [FEATURE] OTLP: Write start time of metrics as created time zero samples into TSDB when created-timestamp-zero-ingestion feature flag is enabled. #​16951
  • [ENHANCEMENT] PromQL: Add warn-level annotations for counter reset conflicts in certain histogram operations. #​17051 #​17094
  • [ENHANCEMENT] UI: Add scrape interval and scrape timeout to targets page. #​17158
  • [ENHANCEMENT] TSDB: Reduce the resolution of native histograms read from chunks or remote read if the schema is exponential. #​17213
  • [ENHANCEMENT] Remote write: Add logging for unexpected metadata in sample batches, when metadata entries are found in samples-only batches. #​17034 #​17082
  • [ENHANCEMENT] Rules: Support concurrent evaluation for rules querying ALERTS and ALERTS_FOR_STATE. #​17064
  • [ENHANCEMENT] TSDB: Add logs to improve visibility into internal operations. #​17074
  • [PERF] OTLP: Write directly to TSDB instead of passing through a Remote-Write adapter when receiving OTLP metrics. #​16951
  • [PERF] OTLP: Reduce number of logs emitted from OTLP endpoint. No need to log duplicate sample errors. #​17201
  • [PERF] PromQL: Move more work to preprocessing step. #​16896
  • [PERF] PromQL: Reduce allocations when walking the syntax tree. #​16593
  • [PERF] TSDB: Optimize appender creation, slightly speeding up startup. #​16922
  • [PERF] TSDB: Improve speed of querying a series with multiple matchers. #​13971
  • [BUGFIX] Alerting: Mutating alerts relabeling (using replace actions, etc.) within a alertmanager_config.alert_relabel_configs block is now scoped correctly and no longer yields altered alerts to subsequent blocks. #​17063
  • [BUGFIX] Config: Infer valid escaping scheme when scrape config validation scheme is set. #​16923
  • [BUGFIX] TSDB: Correctly handle appending mixed-typed samples to the same series. #​17071 #​17241 #​17290 #​17295 #​17296
  • [BUGFIX] Remote-write: Prevent sending unsupported native histograms with custom buckets (NHCB) over Remote-write 1.0, log warning. #​17146
  • [BUGFIX] TSDB: Fix metadata entries handling on metadata-wal-records experimental feature for native histograms with custom buckets (NHCB) in protobuf scraping. #​17156
  • [BUGFIX] TSDB: Ignore Native Histograms with invalid schemas during WAL/WBL replay. #​17214
  • [BUGFIX] PromQL: Avoid empty metric names in annotations for histogram_quantile(). #​16794
  • [BUGFIX] PromQL: Correct inaccurate character positions in errors for some aggregate expressions. #​16996 #​17031
  • [BUGFIX] PromQL: Fix info() function on churning series. #​17135
  • [BUGFIX] PromQL: Set native histogram to gauge type when subtracting or multiplying/dividing with negative factors. #​17004
  • [BUGFIX] TSDB: Reject unsupported native histogram schemas when attempting to append to TSDB. For scrape and remote-write implement reducing the resolution to fit the maximum if the schema is within the -9 to 52. #​17189
  • [BUGFIX] Remote-write: Fix HTTP handler to return after writing error response for invalid compression. #​17050
  • [BUGFIX] Remote-write: Return HTTP error 400 instead of 5xx for wrongly formatted Native Histograms. #​17210
  • [BUGFIX] Scrape: Prevent staleness markers from generating unnecessary series. #​16429
  • [BUGFIX] TSDB: Avoid misleading Failed to calculate size of \"wal\" dir error logs during WAL clean-up. #​17006
  • [BUGFIX] TSDB: Prevent erroneously dropping series records during WAL checkpoints. #​17029
  • [BUGFIX] UI: Fix redirect to path of -web.external-url if -web.route-prefix is configured. #​17240
  • [BUGIFX] Remote-write: Do not panic on invalid symbol table in remote-write 2.0. #​17160

v3.6.0: 3.6.0 / 2025-09-17

Compare Source

  • [FEATURE] PromQL: Add step(), and min() and max() on durations, behind feature flag promql-duration-expr. #​16777
  • [FEATURE] API: Add a /v1/status/tsdb/blocks endpoint exposing metadata about loaded blocks. #​16695
  • [FEATURE] Templates: Add toDuration() and now() functions. #​16619
  • [ENHANCEMENT] Discovery: Add support for attaching namespace metadata to targets. #​16831
  • [ENHANCEMENT] OTLP: Support new UnderscoreEscapingWithoutSuffixes strategy via otlp.translation_strategy. #​16849
  • [ENHANCEMENT] OTLP: Support including scope metadata as metric labels via otlp.promote_scope_metadata. #​16878
  • [ENHANCEMENT] OTLP: Add __type__ and __unit__ labels when feature flag type-and-unit-labels is enabled. #​16630
  • [ENHANCEMENT] Tracing: Send the traceparent HTTP header during scrapes. #​16425
  • [ENHANCEMENT] UI: Add option to disable info and warning query messages under Query page settings. #​16901
  • [ENHANCEMENT] UI: Improve metadata handling for _count/_sum/_bucket suffixes. #​16910
  • [ENHANCEMENT] TSDB: Track stale series in the Head block via the prometheus_tsdb_head_stale_series metric. #​16925
  • [PERF] PromQL: Improve performance due to internal optimizations. #​16797
  • [BUGFIX] Config: Fix "unknown global name escaping method" error messages produced during config validation. #​16801
  • [BUGFIX] Discovery: Fix race condition during shutdown. #​16820
  • [BUGFIX] OTLP: Generate target_info samples between the earliest and latest samples per resource. #​16737
  • [BUGFIX] PromQL: Fail when NaN is passed as parameter to topk(), bottomk(), limitk() and limit_ratio(). #​16725
  • [BUGFIX] PromQL: Fix extrapolation for native counter histograms. #​16828
  • [BUGFIX] PromQL: Reduce numerical errors by disabling some optimizations. #​16895
  • [BUGFIX] PromQL: Fix inconsistencies when using native histograms in subqueries. #​16879
  • [BUGFIX] PromQL: Fix inconsistent annotations for rate() and increase() on histograms when feature flag type-and-unit-labels is enabled. #​16915
  • [BUGFIX] Scraping: Fix memory corruption in slicelabels builds. #​16946
  • [BUGFIX] TSDB: Fix panic on append when feature flag created-timestamp-zero-ingestion is enabled. #​16332
  • [BUGFIX] TSDB: Fix panic on append for native histograms with empty buckets. #​16893

v3.5.1: 3.5.1 / 2026-01-07

Compare Source

This is the current "Long Term Support" release.
No code changes since 3.5.0, just some dependency updates:

  • Docker library updated from 28.2.2 to 28.5.2. #​17821
  • Built with Go 1.24.11.

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.


  • If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

This PR contains the following updates: | Package | Update | Change | |---|---|---| | [prom/prometheus](https://github.com/prometheus/prometheus) | minor | `v3.5.0` → `v3.11.0` | --- ### Release Notes <details> <summary>prometheus/prometheus (prom/prometheus)</summary> ### [`v3.11.0`](https://github.com/prometheus/prometheus/releases/tag/v3.11.0): 3.11.0 / 2026-04-02 [Compare Source](https://github.com/prometheus/prometheus/compare/v3.10.0...v3.11.0) - \[CHANGE] Hetzner SD: The `__meta_hetzner_datacenter` label is deprecated for the role `robot` but kept for backward compatibility, use the `__meta_hetzner_robot_datacenter` label instead. For the role `hcloud`, the label is deprecated and will stop working after the 1 July 2026. [#&#8203;17850](https://github.com/prometheus/prometheus/issues/17850) - \[CHANGE] Hetzner SD: The `__meta_hetzner_hcloud_datacenter_location` and `__meta_hetzner_hcloud_datacenter_location_network_zone` labels are deprecated, use the `__meta_hetzner_hcloud_location` and `__meta_hetzner_hcloud_location_network_zone` labels instead. [#&#8203;17850](https://github.com/prometheus/prometheus/issues/17850) - \[CHANGE] Promtool: Redirect debug output to stderr to avoid interfering with stdout-based tool output. [#&#8203;18346](https://github.com/prometheus/prometheus/issues/18346) - \[FEATURE] AWS SD: Add Elasticache Role. [#&#8203;18099](https://github.com/prometheus/prometheus/issues/18099) - \[FEATURE] AWS SD: Add RDS Role. [#&#8203;18206](https://github.com/prometheus/prometheus/issues/18206) - \[FEATURE] Azure SD: Add support for Azure Workload Identity authentication method. [#&#8203;17207](https://github.com/prometheus/prometheus/issues/17207) - \[FEATURE] Discovery: Introduce `prometheus_sd_last_update_timestamp_seconds` metric to track the last time a service discovery update was sent to consumers. [#&#8203;18194](https://github.com/prometheus/prometheus/issues/18194) - \[FEATURE] Kubernetes SD: Add support for node role selectors for pod roles. [#&#8203;18006](https://github.com/prometheus/prometheus/issues/18006) - \[FEATURE] Kubernetes SD: Introduce pod-based labels for deployment, cronjob, and job controller names: `__meta_kubernetes_pod_deployment_name`, `__meta_kubernetes_pod_cronjob_name` and `__meta_kubernetes_pod_job_name`, respectively. [#&#8203;17774](https://github.com/prometheus/prometheus/issues/17774) - \[FEATURE] PromQL: Add `</` and `>/` operators for trimming observations from native histograms. [#&#8203;17904](https://github.com/prometheus/prometheus/issues/17904) - \[FEATURE] PromQL: Add experimental `histogram_quantiles` variadic function for computing multiple quantiles at once. [#&#8203;17285](https://github.com/prometheus/prometheus/issues/17285) - \[FEATURE] TSDB: Add `storage.tsdb.retention.percentage` configuration to configure the maximum percent of disk usable for TSDB storage. [#&#8203;18080](https://github.com/prometheus/prometheus/issues/18080) - \[FEATURE] TSDB: Add an experimental `fast-startup` feature flag that writes a `series_state.json` file to the WAL directory to track active series state across restarts. [#&#8203;18303](https://github.com/prometheus/prometheus/issues/18303) - \[FEATURE] TSDB: Add an experimental `st-storage` feature flag. When enabled, Prometheus stores ingested start timestamps (ST, previously called Created Timestamp) from scrape or OTLP in the TSDB and Agent WAL, and exposes them via Remote Write 2. [#&#8203;18062](https://github.com/prometheus/prometheus/issues/18062) - \[FEATURE] TSDB: Add an experimental `xor2-encoding` feature flag for the new TSDB block float sample chunk encoding that is optimized for scraped data and allows encoding start timestamps. [#&#8203;18062](https://github.com/prometheus/prometheus/issues/18062) - \[ENHANCEMENT] HTTP client: Add AWS `external_id` support for sigv4. [#&#8203;17916](https://github.com/prometheus/prometheus/issues/17916) - \[ENHANCEMENT] Kubernetes SD: Deduplicate deprecation warning logs from the Kubernetes API to reduce noise. [#&#8203;17829](https://github.com/prometheus/prometheus/issues/17829) - \[ENHANCEMENT] TSDB: Remove old temporary checkpoints when creating a Checkpoint. [#&#8203;17598](https://github.com/prometheus/prometheus/issues/17598) - \[ENHANCEMENT] UI: Add autocomplete support for experimental `first_over_time` and `ts_of_first_over_time` PromQL functions. [#&#8203;18318](https://github.com/prometheus/prometheus/issues/18318) - \[ENHANCEMENT] Vultr SD: Upgrade govultr library from v2 to v3 for continued security patches and maintenance. [#&#8203;18347](https://github.com/prometheus/prometheus/issues/18347) - \[PERF] PromQL: Improve performance and reduce heap allocations in joins (VectorBinop)/And/Or/Unless. [#&#8203;17159](https://github.com/prometheus/prometheus/issues/17159) - \[PERF] PromQL: Partially address performance regression in native histogram aggregations due to using `KahanAdd`. [#&#8203;18252](https://github.com/prometheus/prometheus/issues/18252) - \[PERF] Remote write: Optimize WAL watching used for RW sending to reuse internal buffers. [#&#8203;18250](https://github.com/prometheus/prometheus/issues/18250) - \[PERF] TSDB: Optimize LabelValues intersection performance for matchers. [#&#8203;18069](https://github.com/prometheus/prometheus/issues/18069) - \[PERF] UI: Skip restacking on hover in stacked series charts. [#&#8203;18230](https://github.com/prometheus/prometheus/issues/18230) - \[BUGFIX] AWS SD: Fix EC2 SD ignoring the configured `endpoint` option, a regression from the AWS SDK v2 migration. [#&#8203;18133](https://github.com/prometheus/prometheus/issues/18133) - \[BUGFIX] AWS SD: Fix panic in EC2 SD when DescribeAvailabilityZones returns nil ZoneName or ZoneId. [#&#8203;18133](https://github.com/prometheus/prometheus/issues/18133) - \[BUGFIX] Agent: Fix memory leak caused by duplicate SeriesRefs being loaded as active series. [#&#8203;17538](https://github.com/prometheus/prometheus/issues/17538) - \[BUGFIX] Alerting: Fix alert state incorrectly resetting to pending when the FOR period is increased in the config file. [#&#8203;18244](https://github.com/prometheus/prometheus/issues/18244) - \[BUGFIX] Azure SD: Fix system-assigned managed identity not working when `client_id` is empty. [#&#8203;18323](https://github.com/prometheus/prometheus/issues/18323) - \[BUGFIX] Consul SD: Fix filter parameter not being applied to health service endpoint, causing Node and Node.Meta filters to be ignored. [#&#8203;17349](https://github.com/prometheus/prometheus/issues/17349) - \[BUGFIX] Kubernetes SD: Fix duplicate targets generated by `*DualStack` EndpointSlices policies. [#&#8203;18192](https://github.com/prometheus/prometheus/issues/18192) - \[BUGFIX] OTLP: Fix ErrTooOldSample being returned as HTTP 500 instead of 400 in PRW v2 histogram write paths, preventing infinite client retry loops. [#&#8203;18084](https://github.com/prometheus/prometheus/issues/18084) - \[BUGFIX] OTLP: Fix exemplars getting mixed between incorrect parts of a histogram. [#&#8203;18056](https://github.com/prometheus/prometheus/issues/18056) - \[BUGFIX] PromQL: Do not skip histogram buckets in queries where histogram trimming is used. [#&#8203;18263](https://github.com/prometheus/prometheus/issues/18263) - \[BUGFIX] Remote write: Fix `prometheus_remote_storage_sent_batch_duration_seconds` measuring before the request was sent. [#&#8203;18214](https://github.com/prometheus/prometheus/issues/18214) - \[BUGFIX] Rules: Fix alert state restoration when rule labels contain Go template expressions. [#&#8203;18375](https://github.com/prometheus/prometheus/issues/18375) - \[BUGFIX] Scrape: Fix panic when parsing bare label names without an equal sign in brace-only metric notation. [#&#8203;18229](https://github.com/prometheus/prometheus/issues/18229) - \[BUGFIX] TSDB: Fail early when `use-uncached-io` feature flag is set on unsupported environments. [#&#8203;18219](https://github.com/prometheus/prometheus/issues/18219) - \[BUGFIX] TSDB: Fall back to CLI flag values when retention is removed from config file. [#&#8203;18200](https://github.com/prometheus/prometheus/issues/18200) - \[BUGFIX] TSDB: Fix memory leaks in buffer pools by clearing reference fields before returning buffers to pools. [#&#8203;17895](https://github.com/prometheus/prometheus/issues/17895) - \[BUGFIX] TSDB: Fix missing mmap of histogram chunks during WAL replay. [#&#8203;18306](https://github.com/prometheus/prometheus/issues/18306) - \[BUGFIX] TSDB: Fix storage.tsdb.retention.time unit mismatch in file causing retention to be 1e6 times longer than configured. [#&#8203;18200](https://github.com/prometheus/prometheus/issues/18200) - \[BUGFIX] Tracing: Fix missing traceID in query log when tracing is enabled, previously only spanID was emitted. [#&#8203;18189](https://github.com/prometheus/prometheus/issues/18189) - \[BUGFIX] UI: Fix tooltip Y-offset drift when using multiple graph panels. [#&#8203;18228](https://github.com/prometheus/prometheus/issues/18228) - \[BUGFIX] UI: Update retention display in runtime info when config is reloaded. [#&#8203;18200](https://github.com/prometheus/prometheus/issues/18200) ### [`v3.10.0`](https://github.com/prometheus/prometheus/releases/tag/v3.10.0): 3.10.0 / 2026-02-24 [Compare Source](https://github.com/prometheus/prometheus/compare/v3.9.1...v3.10.0) Prometheus now offers a distroless Docker image variant alongside the default busybox image. The distroless variant provides enhanced security with a minimal base image, uses UID/GID 65532 (nonroot) instead of nobody, and removes the VOLUME declaration. Both variants are available with `-busybox` and `-distroless` tag suffixes (e.g., `prom/prometheus:latest-busybox`, `prom/prometheus:latest-distroless`). The busybox image remains the default with no suffix for backwards compatibility (e.g., `prom/prometheus:latest` points to the busybox variant). For users migrating existing **named** volumes from the busybox image to the distroless variant, the ownership can be adjusted with: ``` docker run --rm -v prometheus-data:/prometheus alpine chown -R 65532:65532 /prometheus ``` Then, the container can be started with the old volume with: ``` docker run -v prometheus-data:/prometheus prom/prometheus:latest-distroless ``` User migrating from bind mounts might need to ajust permissions too, depending on their setup. - \[CHANGE] Alerting: Add `alertmanager` dimension to following metrics: `prometheus_notifications_dropped_total`, `prometheus_notifications_queue_capacity`, `prometheus_notifications_queue_length`. [#&#8203;16355](https://github.com/prometheus/prometheus/issues/16355) - \[CHANGE] UI: Hide expanded alert annotations by default, enabling more information density on the `/alerts` page. [#&#8203;17611](https://github.com/prometheus/prometheus/issues/17611) - \[FEATURE] AWS SD: Add MSK Role. [#&#8203;17600](https://github.com/prometheus/prometheus/issues/17600) - \[FEATURE] PromQL: Add `fill()` / `fill_left()` / `fill_right()` binop modifiers for specifying default values for missing series. [#&#8203;17644](https://github.com/prometheus/prometheus/issues/17644) - \[FEATURE] Web: Add OpenAPI 3.2 specification for the HTTP API at `/api/v1/openapi.yaml`. [#&#8203;17825](https://github.com/prometheus/prometheus/issues/17825) - \[FEATURE] Dockerfile: Add distroless image variant using UID/GID 65532 and no VOLUME declaration. Busybox image remains default. [#&#8203;17876](https://github.com/prometheus/prometheus/issues/17876) - \[FEATURE] Web: Add on-demand wall time profiling under `<URL>/debug/pprof/fgprof`. [#&#8203;18027](https://github.com/prometheus/prometheus/issues/18027) - \[ENHANCEMENT] PromQL: Add more detail to histogram quantile monotonicity info annotations. [#&#8203;15578](https://github.com/prometheus/prometheus/issues/15578) - \[ENHANCEMENT] Alerting: Independent alertmanager sendloops. [#&#8203;16355](https://github.com/prometheus/prometheus/issues/16355) - \[ENHANCEMENT] TSDB: Experimental support for early compaction of stale series in the memory with configurable threshold `stale_series_compaction_threshold` in the config file. [#&#8203;16929](https://github.com/prometheus/prometheus/issues/16929) - \[ENHANCEMENT] Service Discovery: Service discoveries are now removable from the Prometheus binary through the Go build tag `remove_all_sd` and individual service discoveries can be re-added with the build tags `enable_<sd name>_sd`. Users can build a custom Prometheus with only the necessary SDs for a smaller binary size. [#&#8203;17736](https://github.com/prometheus/prometheus/issues/17736) - \[ENHANCEMENT] Promtool: Support promql syntax features `promql-duration-expr` and `promql-extended-range-selectors`. [#&#8203;17926](https://github.com/prometheus/prometheus/issues/17926) - \[PERF] PromQL: Avoid unnecessary label extraction in PromQL functions. [#&#8203;17676](https://github.com/prometheus/prometheus/issues/17676) - \[PERF] PromQL: Improve performance of regex matchers like `.*-.*-.*`. [#&#8203;17707](https://github.com/prometheus/prometheus/issues/17707) - \[PERF] OTLP: Add label caching for OTLP-to-Prometheus conversion to reduce allocations and improve latency. [#&#8203;17860](https://github.com/prometheus/prometheus/issues/17860) - \[PERF] API: Compute `/api/v1/targets/relabel_steps` in a single pass instead of re-running relabeling for each prefix. [#&#8203;17969](https://github.com/prometheus/prometheus/issues/17969) - \[PERF] tsdb: Optimize LabelValues intersection performance for matchers. [#&#8203;18069](https://github.com/prometheus/prometheus/issues/18069) - \[BUGFIX] PromQL: Prevent query strings containing only UTF-8 continuation bytes from crashing Prometheus. [#&#8203;17735](https://github.com/prometheus/prometheus/issues/17735) - \[BUGFIX] Web: Fix missing `X-Prometheus-Stopping` header for `/-/ready` endpoint in `NotReady` state. [#&#8203;17795](https://github.com/prometheus/prometheus/issues/17795) - \[BUGFIX] PromQL: Fix PromQL `info()` function returning empty results when filtering by a label that exists on both the input metric and `target_info`. [#&#8203;17817](https://github.com/prometheus/prometheus/issues/17817) - \[BUGFIX] TSDB: Fix a bug during exemplar buffer grow/shrink that could cause exemplars to be incorrectly discarded. [#&#8203;17863](https://github.com/prometheus/prometheus/issues/17863) - \[BUGFIX] UI: Fix broken graph display after page reload, due to broken Y axis min encoding/decoding. [#&#8203;17869](https://github.com/prometheus/prometheus/issues/17869) - \[BUGFIX] TSDB: Fix memory leaks in buffer pools by clearing reference fields (Labels, Histogram pointers, metadata strings) before returning buffers to pools. [#&#8203;17879](https://github.com/prometheus/prometheus/issues/17879) - \[BUGFIX] PromQL: info function: fix series without identifying labels not being returned. [#&#8203;17898](https://github.com/prometheus/prometheus/issues/17898) - \[BUGFIX] OTLP: Filter `__name__` from OTLP attributes to prevent duplicate labels. [#&#8203;17917](https://github.com/prometheus/prometheus/issues/17917) - \[BUGFIX] TSDB: Fix division by zero when computing stale series ratio with empty head. [#&#8203;17952](https://github.com/prometheus/prometheus/issues/17952) - \[BUGFIX] OTLP: Fix potential silent data loss for sum metrics. [#&#8203;17954](https://github.com/prometheus/prometheus/issues/17954) - \[BUGFIX] PromQL: Fix smoothed interpolation across counter resets. [#&#8203;17988](https://github.com/prometheus/prometheus/issues/17988) - \[BUGFIX] PromQL: Fix panic with `@` modifier on empty ranges. [#&#8203;18020](https://github.com/prometheus/prometheus/issues/18020) - \[BUGFIX] PromQL: Fix `avg_over_time` for a single native histogram. [#&#8203;18058](https://github.com/prometheus/prometheus/issues/18058) ### [`v3.9.1`](https://github.com/prometheus/prometheus/releases/tag/v3.9.1): 3.9.1 / 2026-01-07 [Compare Source](https://github.com/prometheus/prometheus/compare/v3.9.0...v3.9.1) - \[BUGFIX] Agent: fix crash shortly after startup from invalid type of object. [#&#8203;17802](https://github.com/prometheus/prometheus/issues/17802) - \[BUGFIX] Scraping: fix relabel keep/drop not working. [#&#8203;17807](https://github.com/prometheus/prometheus/issues/17807) ### [`v3.9.0`](https://github.com/prometheus/prometheus/releases/tag/v3.9.0): 3.9.0 / 2026-01-06 [Compare Source](https://github.com/prometheus/prometheus/compare/v3.8.1...v3.9.0) #### Note for users of Native Histograms In version 3.9, Native Histograms is no longer experimental, and the feature flag `native-histogram` has no effect. You must now turn on the config setting `scrape_native_histograms` to collect Native Histogram samples from exporters. #### Changelog - \[CHANGE] Native Histograms are no longer experimental! Make the `native-histogram` feature flag a no-op. Use `scrape_native_histograms` config option instead. [#&#8203;17528](https://github.com/prometheus/prometheus/issues/17528) - \[CHANGE] API: Add maximum limit of 10,000 sets of statistics to TSDB status endpoint. [#&#8203;17647](https://github.com/prometheus/prometheus/issues/17647) - \[FEATURE] API: Add /api/v1/features for clients to understand which features are supported. [#&#8203;17427](https://github.com/prometheus/prometheus/issues/17427) - \[FEATURE] Promtool: Add `start_timestamp` field for unit tests. [#&#8203;17636](https://github.com/prometheus/prometheus/issues/17636) - \[FEATURE] Promtool: Add `--format seriesjson` option to `tsdb dump` to output just series labels in JSON format. [#&#8203;13409](https://github.com/prometheus/prometheus/issues/13409) - \[FEATURE] Add `--storage.tsdb.delay-compact-file.path` flag for better interoperability with Thanos. [#&#8203;17435](https://github.com/prometheus/prometheus/issues/17435) - \[FEATURE] UI: Add an option on the query drop-down menu to duplicate that query panel. [#&#8203;17714](https://github.com/prometheus/prometheus/issues/17714) - \[ENHANCEMENT]: TSDB: add flag `--storage.tsdb.block-reload-interval` to configure TSDB Block Reload Interval. [#&#8203;16728](https://github.com/prometheus/prometheus/issues/16728) - \[ENHANCEMENT] UI: Add graph option to start the chart's Y axis at zero. [#&#8203;17565](https://github.com/prometheus/prometheus/issues/17565) - \[ENHANCEMENT] Scraping: Classic protobuf format no longer requires the unit in the metric name. [#&#8203;16834](https://github.com/prometheus/prometheus/issues/16834) - \[ENHANCEMENT] PromQL, Rules, SD, Scraping: Add native histograms to complement existing summaries. [#&#8203;17374](https://github.com/prometheus/prometheus/issues/17374) - \[ENHANCEMENT] Notifications: Add a histogram `prometheus_notifications_latency_histogram_seconds` to complement the existing summary. [#&#8203;16637](https://github.com/prometheus/prometheus/issues/16637) - \[ENHANCEMENT] Remote-write: Add custom scope support for AzureAD authentication. [#&#8203;17483](https://github.com/prometheus/prometheus/issues/17483) - \[ENHANCEMENT] SD: add a `config` label with job name for most `prometheus_sd_refresh` metrics. [#&#8203;17138](https://github.com/prometheus/prometheus/issues/17138) - \[ENHANCEMENT] TSDB: New histogram `prometheus_tsdb_sample_ooo_delta`, the distribution of out-of-order samples in seconds. Collected for all samples, accepted or not. [#&#8203;17477](https://github.com/prometheus/prometheus/issues/17477) - \[ENHANCEMENT] Remote-read: Validate histograms received via remote-read. [#&#8203;17561](https://github.com/prometheus/prometheus/issues/17561) - \[PERF] TSDB: Small optimizations to postings index. [#&#8203;17439](https://github.com/prometheus/prometheus/issues/17439) - \[PERF] Scraping: Speed up relabelling of series. [#&#8203;17530](https://github.com/prometheus/prometheus/issues/17530) - \[PERF] PromQL: Small optimisations in binary operators. [#&#8203;17524](https://github.com/prometheus/prometheus/issues/17524), [#&#8203;17519](https://github.com/prometheus/prometheus/issues/17519). - \[BUGFIX] UI: PromQL autocomplete now shows the correct type and HELP text for OpenMetrics counters whose samples end in `_total`. [#&#8203;17682](https://github.com/prometheus/prometheus/issues/17682) - \[BUGFIX] UI: Fixed codemirror-promql incorrectly showing label completion suggestions after the closing curly brace of a vector selector. [#&#8203;17602](https://github.com/prometheus/prometheus/issues/17602) - \[BUGFIX] UI: Query editor no longer suggests a duration unit if one is already present after a number. [#&#8203;17605](https://github.com/prometheus/prometheus/issues/17605) - \[BUGFIX] PromQL: Fix some "vector cannot contain metrics with the same labelset" errors when experimental delayed name removal is enabled. [#&#8203;17678](https://github.com/prometheus/prometheus/issues/17678) - \[BUGFIX] PromQL: Fix possible corruption of PromQL text if the query had an empty `ignoring()` and non-empty grouping. [#&#8203;17643](https://github.com/prometheus/prometheus/issues/17643) - \[BUGFIX] PromQL: Fix resets/changes to return empty results for anchored selectors when all samples are outside the range. [#&#8203;17479](https://github.com/prometheus/prometheus/issues/17479) - \[BUGFIX] PromQL: Check more consistently for many-to-one matching in filter binary operators. [#&#8203;17668](https://github.com/prometheus/prometheus/issues/17668) - \[BUGFIX] PromQL: Fix collision in unary negation with non-overlapping series. [#&#8203;17708](https://github.com/prometheus/prometheus/issues/17708) - \[BUGFIX] PromQL: Fix collision in label\_join and label\_replace with non-overlapping series. [#&#8203;17703](https://github.com/prometheus/prometheus/issues/17703) - \[BUGFIX] PromQL: Fix bug with inconsistent results for queries with OR expression when experimental delayed name removal is enabled. [#&#8203;17161](https://github.com/prometheus/prometheus/issues/17161) - \[BUGFIX] PromQL: Ensure that `rate`/`increase`/`delta` of histograms results in a gauge histogram. [#&#8203;17608](https://github.com/prometheus/prometheus/issues/17608) - \[BUGFIX] PromQL: Do not panic while iterating over invalid histograms. [#&#8203;17559](https://github.com/prometheus/prometheus/issues/17559) - \[BUGFIX] TSDB: Reject chunk files whose encoded chunk length overflows int. [#&#8203;17533](https://github.com/prometheus/prometheus/issues/17533) - \[BUGFIX] TSDB: Do not panic during resolution reduction of invalid histograms. [#&#8203;17561](https://github.com/prometheus/prometheus/issues/17561) - \[BUGFIX] Remote-write Receive: Avoid duplicate labels when experimental type-and-unit-label feature is enabled. [#&#8203;17546](https://github.com/prometheus/prometheus/issues/17546) - \[BUGFIX] OTLP Receiver: Only write metadata to disk when experimental metadata-wal-records feature is enabled. [#&#8203;17472](https://github.com/prometheus/prometheus/issues/17472) ### [`v3.8.1`](https://github.com/prometheus/prometheus/releases/tag/v3.8.1): 3.8.1 / 2025-12-16 [Compare Source](https://github.com/prometheus/prometheus/compare/v3.8.0...v3.8.1) - \[BUGFIX] remote: Fix Remote Write receiver, so it does not send wrong response headers for v1 flow and cause Prometheus senders to emit false partial error log and metrics. [#&#8203;17683](https://github.com/prometheus/prometheus/issues/17683) ### [`v3.8.0`](https://github.com/prometheus/prometheus/releases/tag/v3.8.0): 3.8.0 / 2025-11-28 [Compare Source](https://github.com/prometheus/prometheus/compare/v3.7.3...v3.8.0) #### Note for users of Native Histograms This is the first release with Native Histograms as a stable feature. However, scraping Native Histograms has to be activated explicitly via the `scrape_native_histograms` config setting (newly introduced in this release). To ease the transition, the `--enable-feature=native-histograms` flag is not a complete no-op in this release, but changes the default value of `scrape_native_histograms` to `true`. In the next release (v3.9), the feature flag *will* be a complete no-op, and the default value of `scrape_native_histograms` will always be `false`. If you have been using the feature flag so far, the recommended course of action is the following: 1. Upgrade to v3.8 and keep the feature flag. Everything should work as before. 2. At your own pace, set `scrape_native_histograms` to `true` in all relevant scrape configs. (There is a global and a per-scrape-config version of `scrape_native_histograms`, allowing granular control if needed. It is a good idea to also set `scrape_native_histograms` explicitly to `false` where you do not want to scrape Native Histograms. In this way, you do not depend on the default value of the setting anymore.) 3. Remove the feature flag and make sure that everything still works as intended. 4. Now you are ready for an upgrade to the next release (v3.9). #### Changelog - \[CHANGE] Remote-write 2 (receiving): Update to [2.0-rc.4 spec](https://github.com/prometheus/docs/blob/60c24e450010df38cfcb4f65df874f6f9b26dbcb/docs/specs/prw/remote_write_spec_2_0.md). "created timestamp" (CT) is now called "start timestamp" (ST). [#&#8203;17411](https://github.com/prometheus/prometheus/issues/17411) - \[CHANGE] TSDB: Native Histogram Custom Bounds with a NaN threshold are now rejected. [#&#8203;17287](https://github.com/prometheus/prometheus/issues/17287) - \[FEATURE] OAuth2: support jwt-bearer grant-type (RFC7523 3.1). [#&#8203;17592](https://github.com/prometheus/prometheus/issues/17592) - \[FEATURE] Dockerfile: Add OpenContainers spec labels to Dockerfile. [#&#8203;16483](https://github.com/prometheus/prometheus/issues/16483) - \[FEATURE] SD: Add unified AWS service discovery for ec2, lightsail and ecs services. [#&#8203;17046](https://github.com/prometheus/prometheus/issues/17046) - \[FEATURE] Native histograms are now a stable, but optional feature, use the `scrape_native_histograms` config setting. [#&#8203;17232](https://github.com/prometheus/prometheus/issues/17232) [#&#8203;17315](https://github.com/prometheus/prometheus/issues/17315) - \[FEATURE] UI: Support anchored and smoothed keyword in promql editor. [#&#8203;17239](https://github.com/prometheus/prometheus/issues/17239) - \[FEATURE] UI: Show detailed relabeling steps for each discovered target. [#&#8203;17337](https://github.com/prometheus/prometheus/issues/17337) - \[FEATURE] Alerting: Add urlQueryEscape to template functions. [#&#8203;17403](https://github.com/prometheus/prometheus/issues/17403) - \[FEATURE] Promtool: Add Remote-Write 2.0 support to `promtool push metrics` via the `--protobuf_message` flag. [#&#8203;17417](https://github.com/prometheus/prometheus/issues/17417) - \[ENHANCEMENT] Clarify the docs about handling negative native histograms. [#&#8203;17249](https://github.com/prometheus/prometheus/issues/17249) - \[ENHANCEMENT] Mixin: Add static UID to the remote-write dashboard. [#&#8203;17256](https://github.com/prometheus/prometheus/issues/17256) - \[ENHANCEMENT] PromQL: Reconcile mismatched NHCB bounds in `Add` and `Sub`. [#&#8203;17278](https://github.com/prometheus/prometheus/issues/17278) - \[ENHANCEMENT] Alerting: Add "unknown" state for alerting rules that haven't been evaluated yet. [#&#8203;17282](https://github.com/prometheus/prometheus/issues/17282) - \[ENHANCEMENT] Scrape: Allow simultaneous use of classic histogram → NHCB conversion and zero-timestamp ingestion. [#&#8203;17305](https://github.com/prometheus/prometheus/issues/17305) - \[ENHANCEMENT] UI: Add smoothed/anchored in explain. [#&#8203;17334](https://github.com/prometheus/prometheus/issues/17334) - \[ENHANCEMENT] OTLP: De-duplicate any `target_info` samples with the same timestamp for the same series. [#&#8203;17400](https://github.com/prometheus/prometheus/issues/17400) - \[ENHANCEMENT] Document `use_fips_sts_endpoint` in `sigv4` config sections. [#&#8203;17304](https://github.com/prometheus/prometheus/issues/17304) - \[ENHANCEMENT] Document Prometheus Agent. [#&#8203;14519](https://github.com/prometheus/prometheus/issues/14519) - \[PERF] PromQL: Speed up parsing of variadic functions. [#&#8203;17316](https://github.com/prometheus/prometheus/issues/17316) - \[PERF] UI: Speed up alerts/rules/... pages by not rendering collapsed content. [#&#8203;17485](https://github.com/prometheus/prometheus/issues/17485) - \[PERF] UI: Performance improvement when getting label name and values in promql editor. [#&#8203;17194](https://github.com/prometheus/prometheus/issues/17194) - \[PERF] UI: Speed up /alerts for many firing alerts via virtual scrolling. [#&#8203;17254](https://github.com/prometheus/prometheus/issues/17254) - \[BUGFIX] PromQL: Fix slice indexing bug in info function on churning series. [#&#8203;17199](https://github.com/prometheus/prometheus/issues/17199) - \[BUGFIX] API: Reduce lock contention on `/api/v1/targets`. [#&#8203;17306](https://github.com/prometheus/prometheus/issues/17306) - \[BUGFIX] PromQL: Consistent handling of gauge vs. counter histograms in aggregations. [#&#8203;17312](https://github.com/prometheus/prometheus/issues/17312) - \[BUGFIX] TSDB: Allow NHCB with -Inf as the first custom value. [#&#8203;17320](https://github.com/prometheus/prometheus/issues/17320) - \[BUGFIX] UI: Fix duplicate loading of data from the API speed up rendering of some pages. [#&#8203;17357](https://github.com/prometheus/prometheus/issues/17357) - \[BUGFIX] Old UI: Fix createExpressionLink to correctly build /graph URLs so links from Alerts/Rules work again. [#&#8203;17365](https://github.com/prometheus/prometheus/issues/17365) - \[BUGFIX] PromQL: Avoid panic when parsing malformed `info` call. [#&#8203;17379](https://github.com/prometheus/prometheus/issues/17379) - \[BUGFIX] PromQL: Include histograms when enforcing sample\_limit. [#&#8203;17390](https://github.com/prometheus/prometheus/issues/17390) - \[BUGFIX] Config: Fix panic if TLS CA file is absent. [#&#8203;17418](https://github.com/prometheus/prometheus/issues/17418) - \[BUGFIX] PromQL: Fix `histogram_fraction` for classic histograms and NHCB if lower bound is in the first bucket. [#&#8203;17424](https://github.com/prometheus/prometheus/issues/17424) ### [`v3.7.3`](https://github.com/prometheus/prometheus/releases/tag/v3.7.3): 3.7.3 / 2025-10-29 [Compare Source](https://github.com/prometheus/prometheus/compare/v3.7.2...v3.7.3) - \[BUGFIX] UI: Revert changed (and breaking) redirect behavior for `-web.external-url` if `-web.route-prefix` is configured, which was introduced in [#&#8203;17240](https://github.com/prometheus/prometheus/issues/17240). [#&#8203;17389](https://github.com/prometheus/prometheus/issues/17389) - \[BUGFIX] Fix federation of some native histograms. [#&#8203;17299](https://github.com/prometheus/prometheus/issues/17299) [#&#8203;17409](https://github.com/prometheus/prometheus/issues/17409) - \[BUGFIX] promtool: `check config` would fail when `--lint=none` flag was set. [#&#8203;17399](https://github.com/prometheus/prometheus/issues/17399) [#&#8203;17414](https://github.com/prometheus/prometheus/issues/17414) - \[BUGFIX] Remote-write: fix a deadlock in the queue resharding logic that can lead to suboptimal queue behavior. [#&#8203;17412](https://github.com/prometheus/prometheus/issues/17412) ### [`v3.7.2`](https://github.com/prometheus/prometheus/releases/tag/v3.7.2): 3.7.2 / 2025-10-22 [Compare Source](https://github.com/prometheus/prometheus/compare/v3.7.1...v3.7.2) - \[BUGFIX] AWS SD: Fix AWS SDK v2 credentials handling for EC2 and Lightsail discovery. [#&#8203;17355](https://github.com/prometheus/prometheus/issues/17355) - \[BUGFIX] AWS SD: Load AWS region from IMDS when not set. [#&#8203;17376](https://github.com/prometheus/prometheus/issues/17376) - \[BUGFIX] Relabeling: Fix `labelmap` action validation with the legacy metric name validation scheme. [#&#8203;17372](https://github.com/prometheus/prometheus/issues/17372) - \[BUGFIX] PromQL: Fix parsing failure when `anchored` and `smoothed` are used as metric names and label names. [#&#8203;17353](https://github.com/prometheus/prometheus/issues/17353) - \[BUGFIX] PromQL: Fix formatting of range vector selectors with `smoothed`/`anchored` modifier. [#&#8203;17354](https://github.com/prometheus/prometheus/issues/17354) ### [`v3.7.1`](https://github.com/prometheus/prometheus/releases/tag/v3.7.1): 3.7.1 / 2025-10-16 [Compare Source](https://github.com/prometheus/prometheus/compare/v3.7.0...v3.7.1) - \[BUGFIX] OTLP: Prefix `key_` to label name when translating an OTel attribute name starting with a single underscore, and keep multiple consecutive underscores in label name when translating an OTel attribute name. This reverts the breaking changes introduced in 3.7.0. [#&#8203;17344](https://github.com/prometheus/prometheus/issues/17344) ### [`v3.7.0`](https://github.com/prometheus/prometheus/releases/tag/v3.7.0): 3.7.0 / 2025-10-15 [Compare Source](https://github.com/prometheus/prometheus/compare/v3.6.0...v3.7.0) #### Warning We discovered a breaking change in the OTLP endpoint: OpenTelemetry attribute names starting with underscore, for example `_attrib`, are no longer automatically translated to `key_attrib`. Please use 3.7.1 instead for OTLP ingestion. #### Changelog - \[CHANGE] Remote-write: the following metrics are deprecated: - `prometheus_remote_storage_samples_in_total`, use `prometheus_wal_watcher_records_read_total{type="samples"}` and `prometheus_remote_storage_samples_dropped_total` instead, - `prometheus_remote_storage_histograms_in_total`, use `prometheus_wal_watcher_records_read_total{type=~".*histogram_samples"}` and `prometheus_remote_storage_histograms_dropped_total` instead, - `prometheus_remote_storage_exemplars_in_total`, use `prometheus_wal_watcher_records_read_total{type="exemplars"}` and `prometheus_remote_storage_exemplars_dropped_total` instead, - `prometheus_remote_storage_highest_timestamp_in_seconds`, use the more accurate `prometheus_remote_storage_queue_highest_timestamp_seconds` instead in dashboards and alerts to properly account for relabeling and for more accuracy. [#&#8203;17065](https://github.com/prometheus/prometheus/issues/17065) - \[FEATURE] PromQL: Add support for experimental anchored and smoothed rate behind feature flag `promql-extended-range-selectors`. [#&#8203;16457](https://github.com/prometheus/prometheus/issues/16457) - \[FEATURE] Federation: Add support for native histograms with custom buckets (NHCB). [#&#8203;17215](https://github.com/prometheus/prometheus/issues/17215) - \[FEATURE] PromQL: Add `first_over_time(...)` and `ts_of_first_over_time(...)` behind feature flag `experimental-promql-functions`. [#&#8203;16963](https://github.com/prometheus/prometheus/issues/16963) [#&#8203;17021](https://github.com/prometheus/prometheus/issues/17021) - \[FEATURE] Remote-write: Add support for Azure Workload Identity as an authentication method for the receiver. [#&#8203;16788](https://github.com/prometheus/prometheus/issues/16788) - \[FEATURE] Remote-write: Add type and unit labels to outgoing time series in remote-write 2.0 when the `type-and-unit-labels` feature flag is enabled. [#&#8203;17033](https://github.com/prometheus/prometheus/issues/17033) - \[FEATURE] OTLP: Write start time of metrics as created time zero samples into TSDB when `created-timestamp-zero-ingestion` feature flag is enabled. [#&#8203;16951](https://github.com/prometheus/prometheus/issues/16951) - \[ENHANCEMENT] PromQL: Add warn-level annotations for counter reset conflicts in certain histogram operations. [#&#8203;17051](https://github.com/prometheus/prometheus/issues/17051) [#&#8203;17094](https://github.com/prometheus/prometheus/issues/17094) - \[ENHANCEMENT] UI: Add scrape interval and scrape timeout to targets page. [#&#8203;17158](https://github.com/prometheus/prometheus/issues/17158) - \[ENHANCEMENT] TSDB: Reduce the resolution of native histograms read from chunks or remote read if the schema is exponential. [#&#8203;17213](https://github.com/prometheus/prometheus/issues/17213) - \[ENHANCEMENT] Remote write: Add logging for unexpected metadata in sample batches, when metadata entries are found in samples-only batches. [#&#8203;17034](https://github.com/prometheus/prometheus/issues/17034) [#&#8203;17082](https://github.com/prometheus/prometheus/issues/17082) - \[ENHANCEMENT] Rules: Support concurrent evaluation for rules querying `ALERTS` and `ALERTS_FOR_STATE`. [#&#8203;17064](https://github.com/prometheus/prometheus/issues/17064) - \[ENHANCEMENT] TSDB: Add logs to improve visibility into internal operations. [#&#8203;17074](https://github.com/prometheus/prometheus/issues/17074) - \[PERF] OTLP: Write directly to TSDB instead of passing through a Remote-Write adapter when receiving OTLP metrics. [#&#8203;16951](https://github.com/prometheus/prometheus/issues/16951) - \[PERF] OTLP: Reduce number of logs emitted from OTLP endpoint. No need to log duplicate sample errors. [#&#8203;17201](https://github.com/prometheus/prometheus/issues/17201) - \[PERF] PromQL: Move more work to preprocessing step. [#&#8203;16896](https://github.com/prometheus/prometheus/issues/16896) - \[PERF] PromQL: Reduce allocations when walking the syntax tree. [#&#8203;16593](https://github.com/prometheus/prometheus/issues/16593) - \[PERF] TSDB: Optimize appender creation, slightly speeding up startup. [#&#8203;16922](https://github.com/prometheus/prometheus/issues/16922) - \[PERF] TSDB: Improve speed of querying a series with multiple matchers. [#&#8203;13971](https://github.com/prometheus/prometheus/issues/13971) - \[BUGFIX] Alerting: Mutating alerts relabeling (using `replace` actions, etc.) within a `alertmanager_config.alert_relabel_configs` block is now scoped correctly and no longer yields altered alerts to subsequent blocks. [#&#8203;17063](https://github.com/prometheus/prometheus/issues/17063) - \[BUGFIX] Config: Infer valid escaping scheme when scrape config validation scheme is set. [#&#8203;16923](https://github.com/prometheus/prometheus/issues/16923) - \[BUGFIX] TSDB: Correctly handle appending mixed-typed samples to the same series. [#&#8203;17071](https://github.com/prometheus/prometheus/issues/17071) [#&#8203;17241](https://github.com/prometheus/prometheus/issues/17241) [#&#8203;17290](https://github.com/prometheus/prometheus/issues/17290) [#&#8203;17295](https://github.com/prometheus/prometheus/issues/17295) [#&#8203;17296](https://github.com/prometheus/prometheus/issues/17296) - \[BUGFIX] Remote-write: Prevent sending unsupported native histograms with custom buckets (NHCB) over Remote-write 1.0, log warning. [#&#8203;17146](https://github.com/prometheus/prometheus/issues/17146) - \[BUGFIX] TSDB: Fix metadata entries handling on `metadata-wal-records` experimental feature for native histograms with custom buckets (NHCB) in protobuf scraping. [#&#8203;17156](https://github.com/prometheus/prometheus/issues/17156) - \[BUGFIX] TSDB: Ignore Native Histograms with invalid schemas during WAL/WBL replay. [#&#8203;17214](https://github.com/prometheus/prometheus/issues/17214) - \[BUGFIX] PromQL: Avoid empty metric names in annotations for `histogram_quantile()`. [#&#8203;16794](https://github.com/prometheus/prometheus/issues/16794) - \[BUGFIX] PromQL: Correct inaccurate character positions in errors for some aggregate expressions. [#&#8203;16996](https://github.com/prometheus/prometheus/issues/16996) [#&#8203;17031](https://github.com/prometheus/prometheus/issues/17031) - \[BUGFIX] PromQL: Fix `info()` function on churning series. [#&#8203;17135](https://github.com/prometheus/prometheus/issues/17135) - \[BUGFIX] PromQL: Set native histogram to gauge type when subtracting or multiplying/dividing with negative factors. [#&#8203;17004](https://github.com/prometheus/prometheus/issues/17004) - \[BUGFIX] TSDB: Reject unsupported native histogram schemas when attempting to append to TSDB. For scrape and remote-write implement reducing the resolution to fit the maximum if the schema is within the -9 to 52. [#&#8203;17189](https://github.com/prometheus/prometheus/issues/17189) - \[BUGFIX] Remote-write: Fix HTTP handler to return after writing error response for invalid compression. [#&#8203;17050](https://github.com/prometheus/prometheus/issues/17050) - \[BUGFIX] Remote-write: Return HTTP error `400` instead of `5xx` for wrongly formatted Native Histograms. [#&#8203;17210](https://github.com/prometheus/prometheus/issues/17210) - \[BUGFIX] Scrape: Prevent staleness markers from generating unnecessary series. [#&#8203;16429](https://github.com/prometheus/prometheus/issues/16429) - \[BUGFIX] TSDB: Avoid misleading `Failed to calculate size of \"wal\" dir` error logs during WAL clean-up. [#&#8203;17006](https://github.com/prometheus/prometheus/issues/17006) - \[BUGFIX] TSDB: Prevent erroneously dropping series records during WAL checkpoints. [#&#8203;17029](https://github.com/prometheus/prometheus/issues/17029) - \[BUGFIX] UI: Fix redirect to path of `-web.external-url` if `-web.route-prefix` is configured. [#&#8203;17240](https://github.com/prometheus/prometheus/issues/17240) - \[BUGIFX] Remote-write: Do not panic on invalid symbol table in remote-write 2.0. [#&#8203;17160](https://github.com/prometheus/prometheus/issues/17160) ### [`v3.6.0`](https://github.com/prometheus/prometheus/releases/tag/v3.6.0): 3.6.0 / 2025-09-17 [Compare Source](https://github.com/prometheus/prometheus/compare/v3.5.1...v3.6.0) - \[FEATURE] PromQL: Add `step()`, and `min()` and `max()` on durations, behind feature flag `promql-duration-expr`. [#&#8203;16777](https://github.com/prometheus/prometheus/issues/16777) - \[FEATURE] API: Add a `/v1/status/tsdb/blocks` endpoint exposing metadata about loaded blocks. [#&#8203;16695](https://github.com/prometheus/prometheus/issues/16695) - \[FEATURE] Templates: Add `toDuration()` and `now()` functions. [#&#8203;16619](https://github.com/prometheus/prometheus/issues/16619) - \[ENHANCEMENT] Discovery: Add support for attaching namespace metadata to targets. [#&#8203;16831](https://github.com/prometheus/prometheus/issues/16831) - \[ENHANCEMENT] OTLP: Support new `UnderscoreEscapingWithoutSuffixes` strategy via `otlp.translation_strategy`. [#&#8203;16849](https://github.com/prometheus/prometheus/issues/16849) - \[ENHANCEMENT] OTLP: Support including scope metadata as metric labels via `otlp.promote_scope_metadata`. [#&#8203;16878](https://github.com/prometheus/prometheus/issues/16878) - \[ENHANCEMENT] OTLP: Add `__type__` and `__unit__` labels when feature flag `type-and-unit-labels` is enabled. [#&#8203;16630](https://github.com/prometheus/prometheus/issues/16630) - \[ENHANCEMENT] Tracing: Send the traceparent HTTP header during scrapes. [#&#8203;16425](https://github.com/prometheus/prometheus/issues/16425) - \[ENHANCEMENT] UI: Add option to disable info and warning query messages under `Query page settings`. [#&#8203;16901](https://github.com/prometheus/prometheus/issues/16901) - \[ENHANCEMENT] UI: Improve metadata handling for `_count/_sum/_bucket` suffixes. [#&#8203;16910](https://github.com/prometheus/prometheus/issues/16910) - \[ENHANCEMENT] TSDB: Track stale series in the Head block via the `prometheus_tsdb_head_stale_series` metric. [#&#8203;16925](https://github.com/prometheus/prometheus/issues/16925) - \[PERF] PromQL: Improve performance due to internal optimizations. [#&#8203;16797](https://github.com/prometheus/prometheus/issues/16797) - \[BUGFIX] Config: Fix "unknown global name escaping method" error messages produced during config validation. [#&#8203;16801](https://github.com/prometheus/prometheus/issues/16801) - \[BUGFIX] Discovery: Fix race condition during shutdown. [#&#8203;16820](https://github.com/prometheus/prometheus/issues/16820) - \[BUGFIX] OTLP: Generate `target_info` samples between the earliest and latest samples per resource. [#&#8203;16737](https://github.com/prometheus/prometheus/issues/16737) - \[BUGFIX] PromQL: Fail when `NaN` is passed as parameter to `topk()`, `bottomk()`, `limitk()` and `limit_ratio()`. [#&#8203;16725](https://github.com/prometheus/prometheus/issues/16725) - \[BUGFIX] PromQL: Fix extrapolation for native counter histograms. [#&#8203;16828](https://github.com/prometheus/prometheus/issues/16828) - \[BUGFIX] PromQL: Reduce numerical errors by disabling some optimizations. [#&#8203;16895](https://github.com/prometheus/prometheus/issues/16895) - \[BUGFIX] PromQL: Fix inconsistencies when using native histograms in subqueries. [#&#8203;16879](https://github.com/prometheus/prometheus/issues/16879) - \[BUGFIX] PromQL: Fix inconsistent annotations for `rate()` and `increase()` on histograms when feature flag `type-and-unit-labels` is enabled. [#&#8203;16915](https://github.com/prometheus/prometheus/issues/16915) - \[BUGFIX] Scraping: Fix memory corruption in `slicelabels` builds. [#&#8203;16946](https://github.com/prometheus/prometheus/issues/16946) - \[BUGFIX] TSDB: Fix panic on append when feature flag `created-timestamp-zero-ingestion` is enabled. [#&#8203;16332](https://github.com/prometheus/prometheus/issues/16332) - \[BUGFIX] TSDB: Fix panic on append for native histograms with empty buckets. [#&#8203;16893](https://github.com/prometheus/prometheus/issues/16893) ### [`v3.5.1`](https://github.com/prometheus/prometheus/releases/tag/v3.5.1): 3.5.1 / 2026-01-07 [Compare Source](https://github.com/prometheus/prometheus/compare/v3.5.0...v3.5.1) This is the current "Long Term Support" release. No code changes since 3.5.0, just some dependency updates: - Docker library updated from 28.2.2 to 28.5.2. [#&#8203;17821](https://github.com/prometheus/prometheus/issues/17821) - Built with Go 1.24.11. </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xNC4xIiwidXBkYXRlZEluVmVyIjoiNDMuMTQuMSIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOltdfQ==-->
Update prom/prometheus Docker tag to v3.9.1
Some checks failed
trunk / Build Python Wheel 📦 (push) Has been skipped
trunk / Build and push container image to GHCR (push) Has been skipped
trunk / Publish 📦 to PyPI (push) Has been skipped
trunk / Create GitHub Release (push) Has been skipped
trunk / Run tests (push) Failing after 3s
e4f914681f
renovate-bot force-pushed renovate/prom-prometheus-3.x from e4f914681f
Some checks failed
trunk / Build Python Wheel 📦 (push) Has been skipped
trunk / Build and push container image to GHCR (push) Has been skipped
trunk / Publish 📦 to PyPI (push) Has been skipped
trunk / Create GitHub Release (push) Has been skipped
trunk / Run tests (push) Failing after 3s
to 41f81be76b
Some checks failed
trunk / Run tests (push) Failing after 5s
trunk / Build Python Wheel 📦 (push) Has been skipped
trunk / Build and push container image to GHCR (push) Has been skipped
trunk / Publish 📦 to PyPI (push) Has been skipped
trunk / Create GitHub Release (push) Has been skipped
2026-02-26 12:01:59 +00:00
Compare
renovate-bot changed title from Update prom/prometheus Docker tag to v3.9.1 to Update prom/prometheus Docker tag to v3.10.0 2026-02-26 12:02:14 +00:00
renovate-bot force-pushed renovate/prom-prometheus-3.x from 41f81be76b
Some checks failed
trunk / Run tests (push) Failing after 5s
trunk / Build Python Wheel 📦 (push) Has been skipped
trunk / Build and push container image to GHCR (push) Has been skipped
trunk / Publish 📦 to PyPI (push) Has been skipped
trunk / Create GitHub Release (push) Has been skipped
to 4fa86e2efc
Some checks failed
trunk / Run tests (push) Failing after 5s
trunk / Build Python Wheel 📦 (push) Has been skipped
trunk / Build and push container image to GHCR (push) Has been skipped
trunk / Publish 📦 to PyPI (push) Has been skipped
trunk / Create GitHub Release (push) Has been skipped
2026-04-02 12:01:48 +00:00
Compare
renovate-bot changed title from Update prom/prometheus Docker tag to v3.10.0 to Update prom/prometheus Docker tag to v3.11.0 2026-04-02 12:01:56 +00:00
Some checks failed
trunk / Run tests (push) Failing after 5s
trunk / Build Python Wheel 📦 (push) Has been skipped
trunk / Build and push container image to GHCR (push) Has been skipped
trunk / Publish 📦 to PyPI (push) Has been skipped
trunk / Create GitHub Release (push) Has been skipped
This pull request can be merged automatically.
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin renovate/prom-prometheus-3.x:renovate/prom-prometheus-3.x
git switch renovate/prom-prometheus-3.x

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main
git merge --no-ff renovate/prom-prometheus-3.x
git switch renovate/prom-prometheus-3.x
git rebase main
git switch main
git merge --ff-only renovate/prom-prometheus-3.x
git switch renovate/prom-prometheus-3.x
git rebase main
git switch main
git merge --no-ff renovate/prom-prometheus-3.x
git switch main
git merge --squash renovate/prom-prometheus-3.x
git switch main
git merge --ff-only renovate/prom-prometheus-3.x
git switch main
git merge renovate/prom-prometheus-3.x
git push origin main
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
StarkZarn/meshtastic-prometheus-exporter!12
No description provided.