Add support for http histogram buckets by MonkeyCanCode · Pull Request #4921 · apache/polaris

MonkeyCanCode · 2026-06-28T22:16:47Z

Feature for #4802. As the cardinality is very high with Micrometer (e.g. register 69 metrics per endpoint per unique label), we proceeded with SLOs approach.

Sample metrics:

# HELP http_server_requests_seconds HTTP server request processing time
# TYPE http_server_requests_seconds histogram
http_server_requests_seconds_bucket{application="Polaris",method="POST",outcome="SUCCESS",status="200",uri="/api/catalog/v1/oauth/tokens",le="0.01",} 0.0
http_server_requests_seconds_bucket{application="Polaris",method="POST",outcome="SUCCESS",status="200",uri="/api/catalog/v1/oauth/tokens",le="0.05",} 0.0
http_server_requests_seconds_bucket{application="Polaris",method="POST",outcome="SUCCESS",status="200",uri="/api/catalog/v1/oauth/tokens",le="0.1",} 0.0
http_server_requests_seconds_bucket{application="Polaris",method="POST",outcome="SUCCESS",status="200",uri="/api/catalog/v1/oauth/tokens",le="1.0",} 1.0
http_server_requests_seconds_bucket{application="Polaris",method="POST",outcome="SUCCESS",status="200",uri="/api/catalog/v1/oauth/tokens",le="5.0",} 1.0
http_server_requests_seconds_bucket{application="Polaris",method="POST",outcome="SUCCESS",status="200",uri="/api/catalog/v1/oauth/tokens",le="+Inf",} 1.0
http_server_requests_seconds_count{application="Polaris",method="POST",outcome="SUCCESS",status="200",uri="/api/catalog/v1/oauth/tokens",} 1.0
http_server_requests_seconds_sum{application="Polaris",method="POST",outcome="SUCCESS",status="200",uri="/api/catalog/v1/oauth/tokens",} 0.424681917
http_server_requests_seconds_bucket{application="Polaris",method="GET",outcome="SUCCESS",status="200",uri="/api/management/v1/principals",le="0.01",} 0.0
http_server_requests_seconds_bucket{application="Polaris",method="GET",outcome="SUCCESS",status="200",uri="/api/management/v1/principals",le="0.05",} 0.0
http_server_requests_seconds_bucket{application="Polaris",method="GET",outcome="SUCCESS",status="200",uri="/api/management/v1/principals",le="0.1",} 1.0
http_server_requests_seconds_bucket{application="Polaris",method="GET",outcome="SUCCESS",status="200",uri="/api/management/v1/principals",le="1.0",} 1.0
http_server_requests_seconds_bucket{application="Polaris",method="GET",outcome="SUCCESS",status="200",uri="/api/management/v1/principals",le="5.0",} 1.0
http_server_requests_seconds_bucket{application="Polaris",method="GET",outcome="SUCCESS",status="200",uri="/api/management/v1/principals",le="+Inf",} 1.0
http_server_requests_seconds_count{application="Polaris",method="GET",outcome="SUCCESS",status="200",uri="/api/management/v1/principals",} 1.0
http_server_requests_seconds_sum{application="Polaris",method="GET",outcome="SUCCESS",status="200",uri="/api/management/v1/principals",} 0.073774667

Checklist

🛡️ Don't disclose security issues! (contact security@apache.org)
🔗 Clearly explained why the changes are needed, or linked related issues: Fixes #
🧪 Added/updated tests with good coverage, or manually tested (and explained how)
💡 Added comments for complex logic
🧾 Updated CHANGELOG.md (if needed)
📚 Updated documentation in site/content/in-dev/unreleased (if needed)

snazy

I think the MeterFilter approach is reasonable, because I don’t see a Quarkus config property that enables percentile histograms just for http.server.requests.

One thing to tighten: the property/docs describe this as Prometheus histogram buckets, but the filter applies to all active registries. Could we either constrain it with @MeterFilterConstraint(applyTo = PrometheusMeterRegistry.class) or make the config/docs generic?

flyrain

LGTM overall. Thanks @MonkeyCanCode ! Left some minor comments.

snazy · 2026-06-30T09:22:56Z

+     * <p>When set, one histogram bucket is published per boundary so backends can compute
+     * percentiles (e.g. p95, p99) at the configured durations.
+     */
+    Optional<List<Duration>> histogramSlos();


Why Optional<List<Duration>> here?
Optional appears superfluous as Lists can be empty.

So when tried to use @WithDefault(""), this doesn't appear to be working as Smallrye appears to threat "" as null for List<Duration>.

Smallrye Config requires Optional<List<>> (and Optional<Set<>> ) if the list/set is optional. It does not conflate missing list vs empty list. (oddly enough, you don't need this for maps).

snazy · 2026-06-30T09:53:14Z

 | `polaris.metrics.realm-id-tag.enable-in-http-metrics` | `false` | `boolean` | Whether to include the Realm ID tag in the HTTP server request metrics. <br><br>Beware that if the cardinality of this tag is too high, it can cause performance issues or  even crash the server.  |
 | `polaris.metrics.realm-id-tag.http-metrics-max-cardinality` | `100` | `int` | The maximum number of Realm ID tag values allowed for the HTTP server request metrics. <br><br>This is used to prevent the number of tags from growing indefinitely and causing  performance issues or crashing the server.   <br><br>If the number of tags exceeds this value, a warning will be logged and no more HTTP server  request metrics will be recorded.  |
 | `polaris.metrics.user-principal-tag.enable-in-api-metrics` | `false` | `boolean` | Whether to include the User Principal tag in the API request metrics. <br><br>Beware that if the cardinality of this tag is too high, it can cause performance issues or  even crash the server.  |
+| `polaris.metrics.http-server-requests.histogram-slos` |  | `list of duration` | Service Level Objective (SLO) boundaries for the HTTP server request duration histogram. <br><br>When set, one histogram bucket is published per boundary so backends can compute  percentiles (e.g. p95, p99) at the configured durations.  |


histogram-slos is okay, but I think histogram-bucket-boundaries is clearer

I am fine with either way. Do you preferred this change? @flyrain @adutra as your guys are reviewing this PR as well.

I''m torn, because Micrometer uses the "SLO" terminology, e.g. DistributionStatisticConfig.builder().serviceLevelObjectives(), so this term may sound more familiar to people accustomed to Micrometer than "Bucket boundaries".

flyrain

Thanks @MonkeyCanCode !

Co-authored-by: Robert Stupp <snazy@snazy.de>

Add support for http histogram buckets

4dc105d

github-project-automation Bot added this to Basic Kanban Board Jun 28, 2026

github-project-automation Bot moved this to PRs In Progress in Basic Kanban Board Jun 28, 2026

MonkeyCanCode requested review from adutra, dimas-b and flyrain June 28, 2026 22:16

MonkeyCanCode mentioned this pull request Jun 28, 2026

Publish histogram buckets for HTTP server request duration metrics #4802

Open

Add support for http histogram buckets

3b10eae

snazy reviewed Jun 29, 2026

View reviewed changes

Comment thread runtime/service/src/test/java/org/apache/polaris/service/ratelimiter/RateLimiterFilterTest.java Outdated

Comment thread runtime/service/src/test/java/org/apache/polaris/service/Profiles.java Outdated

adutra reviewed Jun 29, 2026

View reviewed changes

Comment thread site/content/in-dev/unreleased/telemetry.md Outdated

Comment thread runtime/service/src/main/java/org/apache/polaris/service/metrics/MeterFilterProducer.java Outdated

flyrain reviewed Jun 29, 2026

View reviewed changes

Comment thread CHANGELOG.md Outdated

Comment thread site/content/in-dev/unreleased/telemetry.md Outdated

Add support for http histogram buckets via SLOs

846a581

MonkeyCanCode requested review from adutra, flyrain and snazy June 30, 2026 02:30

snazy reviewed Jun 30, 2026

View reviewed changes

adutra previously approved these changes Jun 30, 2026

View reviewed changes

github-project-automation Bot moved this from PRs In Progress to Ready to merge in Basic Kanban Board Jun 30, 2026

flyrain previously approved these changes Jun 30, 2026

View reviewed changes

Apply suggestions from code review

19e0eb6

Co-authored-by: Robert Stupp <snazy@snazy.de>

MonkeyCanCode dismissed stale reviews from flyrain and adutra via 19e0eb6 June 30, 2026 23:58

adutra approved these changes Jul 1, 2026

View reviewed changes

flyrain approved these changes Jul 1, 2026

View reviewed changes

MonkeyCanCode merged commit b3fcad6 into apache:main Jul 2, 2026
24 checks passed

github-project-automation Bot moved this from Ready to merge to Done in Basic Kanban Board Jul 2, 2026

Uh oh!

Conversation

MonkeyCanCode commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Uh oh!

snazy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

flyrain left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

snazy Jun 30, 2026

Choose a reason for hiding this comment

Uh oh!

MonkeyCanCode Jun 30, 2026

Choose a reason for hiding this comment

Uh oh!

adutra Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

snazy Jun 30, 2026

Choose a reason for hiding this comment

Uh oh!

MonkeyCanCode Jun 30, 2026

Choose a reason for hiding this comment

Uh oh!

adutra Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

flyrain left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

MonkeyCanCode commented Jun 28, 2026 •

edited

Loading