MP Config properties for Histogram and Timers to extend and customize output

api/src/main/java/org/eclipse/microprofile/metrics/Snapshot.java

@@ -1,6 +1,6 @@
 /*
  **********************************************************************
- * Copyright (c) 2017, 2022 Contributors to the Eclipse Foundation
+ * Copyright (c) 2017, 2023 Contributors to the Eclipse Foundation
  *               2010-2013 Coda Hale, Yammer.com
  *
  * See the NOTICES file(s) distributed with this work for additional
@@ -54,10 +54,18 @@ public abstract class Snapshot {
      * Returns an array of {@link PercentileValue} containing the percentiles and associated values of this
      * {@link Snapshot} at the moment invocation.
      *
-     * @return an array of {@link PercentileValue}
+     * @return an array of {@link PercentileValue} if it is available or an empty array if not available
      */
     public abstract PercentileValue[] percentileValues();
 
+    /**
+     * Returns an array of {@link HistogramBucket} containing the bucket and associated value of this {@link Snapshot}
+     * at the moment invocation.
+     *
+     * @return an array of {@link HistogramBucket} if it is available or an empty array if not available
+     */
+    public abstract HistogramBucket[] bucketValues();
+
     /**
      * Writes the values of the snapshot to the given stream.
      *
@@ -67,8 +75,8 @@ public abstract class Snapshot {
     public abstract void dump(OutputStream output);
 
     /**
-     * Represents a percentile and its value at the moment it was sampled from the Snapshot.
-     * Percentile values of a {@link Timer} are represented in units of nanoseconds.
+     * Represents a percentile and its value at the moment it was sampled from the Snapshot. Percentile values of a
+     * {@link Timer} are represented in units of nanoseconds.
      *
      * See {@link #percentileValue()}
      */
@@ -89,18 +97,18 @@ public PercentileValue(double percentile, double value) {
         }
 
         /**
-         * Returns percentile
+         * Returns the percentile
          *
-         * @return double percentile
+         * @return the percentile
          */
         public double getPercentile() {
             return this.percentile;
         }
 
         /**
-         * Returns value at percentile
+         * Returns the value at percentile
          *
-         * @return double value at percentile
+         * @return the value at the percentile
          */
         public double getValue() {
             return this.value;
@@ -111,4 +119,50 @@ public String toString() {
         }
 
     }
+
+    /**
+     * Represents a cumulative histogram bucket at the moment it was sampled from the Snapshot. The bucket of
+     * {@link Timer} will be represented in nanoseconds.
+     *
+     * See {@link #bucketValues()}
+     */
+    public static class HistogramBucket {
+        private final double bucket;
+        private final long count;
+
+        /**
+         *
+         * @param count
+         *            count at this bucket
+         * @param bucket
+         *            the upper limit value of this bucket
+         */
+        public HistogramBucket(double bucket, long count) {
+            this.bucket = bucket;
+            this.count = count;
+        }
+
+        /**
+         * Returns the count of the bucket
+         *
+         * @return the count of the bucket
+         */
+        public long getCount() {
+            return this.count;
+        }
+
+        /**
+         * Returns the upper limit value of this bucket
+         *
+         * @return the upper limit value of this bucket
+         */
+        public double getBucket() {
+            return this.bucket;
+        }
+
+        public String toString() {
+            return "[Bucket: " + (this.bucket) + " with count: " + this.count + "]";
+        }
+
+    }
 }

spec/src/main/asciidoc/changelog.adoc

@@ -25,19 +25,13 @@
 == Changes in 5.1
 A full list of changes may be found on the link:https://github.com/eclipse/microprofile-metrics/milestone/16[MicroProfile Metrics 5.1 Milestone]
 
-=== Other changes
-
-* Include new recommendation for multi-application deployments to use a  `mp.metrics.defaultAppName` property (https://github.com/eclipse/microprofile-metrics/issues/766[766])
-
-[[release_notes_5_1]]
-== Changes in 5.1
-A full list of changes may be found on the link:https://github.com/eclipse/microprofile-metrics/milestone/16[MicroProfile Metrics 5.1 Milestone]
-
 === Functional Changes
-
+* Introduced MP Config properties that customize how Histogram and Timer metrics track and output statistics for percentiles and histogram-buckets. (https://github.com/eclipse/microprofile-metrics/issues/691[691])(https://github.com/eclipse/microprofile-metrics/issues/676[676])(https://github.com/eclipse/microprofile-metrics/issues/675[675])(https://github.com/eclipse/microprofile-metrics/issues/674[674])(https://github.com/eclipse/microprofile-metrics/issues/587[587])
 * @RegistryScope is now a qualifier  (https://github.com/eclipse/microprofile-metrics/issues/749[749])
 
+=== Other changes
 
+* Include new recommendation for multi-application deployments to use a  `mp.metrics.defaultAppName` property (https://github.com/eclipse/microprofile-metrics/issues/766[766])
 
 [[release_notes_5_0]]
 == Changes in 5.0

spec/src/main/asciidoc/histogram-timer-config.adoc

@@ -0,0 +1,220 @@
+//
+// Copyright (c) 2023 Contributors to the Eclipse Foundation
+//
+// See the NOTICE file(s) distributed with this work for additional
+// information regarding copyright ownership.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+
+[[histogram-timer-config]]
+== Configuration for Histogram and Timers
+
+This section defines MicroProfile Config properties that extend the ability of the histogram and timer metrics to track and output additional statistics. This includes customizing the percentile output as well as enabling and customizing histogram-bucket values. These MicroProfile config properties can be defined at the server level or at the application level through the `META-INF/microprofile-config.properties` file within the application archive.
+
+[[percentile-configuration]]
+=== Percentile customization for histograms and timers
+
+The percentiles that are tracked and are output can be overridden and customized by using the MicroProfile Config property `mp.metrics.distribution.percentiles`. The property follows these rules:
+
+* The property accepts a semi-colon separated set of values that consist of a metric name followed by an equals sign (`=`) and comma separated percentile values _(See <<percentiles-examples,examples>> below)_.
+* The asterisk (`*`) can be used as a wildcard at the end of the metric name.
+* The percentile values are float values between 0.0 and 1.0 inclusively. Invalid values will be ignored.
+* Setting the property with a metric name but no percentile values disables percentile output for that metric.
+* Setting the property without any value disables percentiles for all histogram and timer metrics.
+* Values defined later take precedence over values before (i.e., right to left precedence).
+
+[[percentiles-examples]]
+.MicroProfile Config `mp.metrics.distribution.percentiles` property examples
+----
+//alpha.histogram will publish the 30th and 40th percentile and alpha.timer will publish the 50th and 80th percentile
+mp.metrics.distribution.percentiles=alpha.histogram=0.3,0.4;alpha.timer=0.5,0.8
+
+//any timer or histogram that match alpha.* will publish the 60th percentile
+mp.metrics.distribution.percentiles=alpha.*=0.6
+
+//any timer or histogram that match alpha.* will publish the 60th percentile and alpha.test.histogram which will only publish the 40th percentile due to precedence
+mp.metrics.distribution.percentiles=alpha.*=0.6;alpha.test.histogram=0.4
+
+//disables percentiles for all histogram and timer metrics
+mp.metrics.distribution.percentiles=*=
+
+//disables percentiles for all histogram and timer metrics
+mp.metrics.distribution.percentiles=
+----
+
+NOTE: Histograms and Timers that do not match the metric name definitions of the `mp.metrics.distribution.percentiles` property will output the default 50th, 75th, 95th, 99th and 99.9th percentiles.
+
+=== Defining histogram-buckets for histograms and timers
+
+The MicroProfile Metrics runtime offers MicroProfile Config properties to enable and explicitly set (cumulative) histogram-buckets for histograms and timers.
+The property `mp.metrics.distribution.timer.buckets` applies to timer metrics and the property `mp.metrics.distribution.histogram.buckets` applies to histograms. These two properties follow these rules unless explicitly describing a specific property:
+
+* The property accepts a semi-colon separated set of values that consist of a metric name followed by an equals sign (`=`) and comma separated bucket values:
+** `mp.metrics.distribution.histogram.buckets` accepts a comma separated list of float and integer values greater than 0 _(See <<histogram-bucket-sample,examples>> below)_.
+** `mp.metrics.distribution.timer.buckets` accepts a comma separated list of integers with an appended time unit (valid time units are `ms` for milliseconds, `s` for seconds, `m` for minutes and `h` for hours). Float values will not be accepted. Values with no time unit default to milliseconds. _(See <<timer-buckets-sample,examples>> below)_
+* The asterisk (`*`) can be used as a wildcard at the end of the metric name.
+* Defining a metric name with no bucket values or invalid values provides no effect.
+* Values defined later take precedence over values before (i.e., right to left precedence).
+
+NOTE: When customizing histogram-bucket values for the histogram and timer metrics, the metrics included in the Prometheus output will automatically switch from a `summary` type to the `histogram` type. Any percentile values that are also outputted will belong under the `histogram` type. The Prometheus output will also automatically include a `+Inf` bucket to capture all values. This will be demonstrated in the <<histogram-bucket-output, histogram sample output>> and <<timer-buckets-output, timer sample output>>. The metric output for the buckets will also be appended with `_bucket`, and a tag `le` is used to denote the bucket value.
+
+[[histogram-bucket-sample]]
+.MicroProfile Config `mp.metrics.distribution.histogram.buckets` property examples
+----
+//alpha.histogram will publish histogram-buckets with maximum values of 10.0, 50.0 and 100.0 while the beta.histogram will publish histogram buckets with maximum values of 30.0, 50 and 123.
+mp.metrics.distribution.histogram.buckets=alpha.histogram=10.0,50.0,100.0;beta.histogram=30.0,50.0,123
+
+//any histogram that matches with `beta.*` will publish histogram buckets with maximum values of 50.0 and 100.0 except the beta.test.histogram which will publish a bucket with a maximum value of 100.0 due to precedence.
+mp.metrics.distribution.histogram.buckets=beta.*=50.0,100.0;beta.test.histogram=100.0
+
+----
+
+NOTE: Histograms that do not match any of the metric names defined by the `mp.metrics.distribution.histogram.buckets` property will output no buckets.
+
+The following is a sample output of the `alpha.histogram` defined in the first `mp.metrics.distribution.histogram.buckets` definition above.
+
+[[histogram-bucket-output]]
+.Sample alpha.histogram Prometheus output
+----
+# HELP alpha_histogram_cookies  
+# TYPE alpha_histogram_cookies histogram <1>
+alpha_histogram_cookies{mp_scope="application",quantile="0.5",} 9.25 <2>
+alpha_histogram_cookies{mp_scope="application",quantile="0.75",} 51.75 <2>
+alpha_histogram_cookies{mp_scope="application",quantile="0.95",} 51.75 <2>
+alpha_histogram_cookies{mp_scope="application",quantile="0.98",} 51.75 <2>
+alpha_histogram_cookies{mp_scope="application",quantile="0.99",} 51.75 <2>
+alpha_histogram_cookies{mp_scope="application",quantile="0.999",} 51.75 <2>
+alpha_histogram_cookies_bucket{mp_scope="application",le="10.0",} 2.0 <3>
+alpha_histogram_cookies_bucket{mp_scope="application",le="50.0",} 3.0 <3>
+alpha_histogram_cookies_bucket{mp_scope="application",le="100.0",} 3.0 <3>
+alpha_histogram_cookies_bucket{mp_scope="application",le="+Inf",} 3.0 <3><4>
+alpha_histogram_cookies_count{mp_scope="application",} 3.0
+alpha_histogram_cookies_sum{mp_scope="application",} 64.0
+# HELP alpha_histogram_cookies_max  
+# TYPE alpha_histogram_cookies_max gauge
+alpha_histogram_cookies_max{mp_scope="application",} 50.0
+----
+<1> The Prometheus metric type is `histogram`
+
+<2> percentiles are part of  the `histogram` type instead of `summary`
+
+<3> histogram buckets
+
+<4> The `+Inf` bucket is included in the output
+
+
+
+[[timer-buckets-sample]]
+.MicroProfile Config `mp.metrics.distribution.timer.buckets` property examples
+----
+//alpha.timer will publish histogram-buckets with maximum values of 500 milliseconds, 2 seconds, and 3 minutes while the beta.timer will publish histogram buckets with maximum values of 10 seconds, 2 minutes and 5 hours. Timer metrics that do not match will not have any histogram buckets.
+mp.metrics.distribution.timer.buckets=alpha.timer=500ms,2s,3m;beta.timer=10s,2m,5h
+
+//any timer that matches with `alpha.*` will publish histogram buckets with maximum values of 50 seconds and 100 seconds while the alpha.test.timer which will publish a bucket with a maximum value 100 milliseconds due to precedence. Timer metrics that do not match will not have any histogram buckets.
+mp.metrics.distribution.timer.buckets=alpha.*=50s,100s;alpha.test.timer=100
+
+----
+
+NOTE: Timers that do not match any of the metric names defined by the `mp.metrics.distribution.timer.buckets` property will output no buckets.
+
+NOTE: The time units defined by the property will be converted to second based buckets for the Prometheus output as timers are based in seconds.
+
+The following is a sample output of the `alpha.timer` defined in the first `mp.metrics.distribution.timer.buckets` definition above.
+
+[[timer-buckets-output]]
+.Sample alpha.timer Prometheus output
+----
+# HELP alpha_timer_seconds_max  
+# TYPE alpha_timer_seconds_max gauge
+alpha_timer_seconds_max{mp_scope="application"} 5.633
+# HELP alpha_timer_seconds  
+# TYPE alpha_timer_seconds histogram <1>
+alpha_timer_seconds{mp_scope="application", quantile="0.5",} 0.67108864 <2>
+alpha_timer_seconds{quantile="0.75",} 5.603590144 <2>
+alpha_timer_seconds{mp_scope="application", quantile="0.95",} 5.603590144 <2>
+alpha_timer_seconds{mp_scope="application", quantile="0.98",} 5.603590144 <2>
+alpha_timer_seconds{mp_scope="application", quantile="0.99",} 5.603590144 <2>
+alpha_timer_seconds{mp_scope="application", quantile="0.999",} 5.603590144 <2>
+alpha_timer_seconds_bucket{mp_scope="application", le="0.5",} 0.0 <3><5>
+alpha_timer_seconds_bucket{mp_scope="application", le="2.0",} 1.0 <3><5>
+alpha_timer_seconds_bucket{mp_scope="application", le="180.0",} 2.0 <3><5>
+alpha_timer_seconds_bucket{mp_scope="application", le="+Inf",} 2.0 <3><4>
+alpha_timer_seconds_count{mp_scope="application"} 2.0
+alpha_timer_seconds_sum{mp_scope="application"} 6.333
+----
+
+<1> The Prometheus metric type is `histogram`
+
+<2> percentiles are part of  the `histogram` type instead of `summary`
+
+<3> histogram buckets
+
+<4> The `+Inf` bucket is included in the output
+
+<5> bucket values converted to seconds
+
+=== (Optional) Enabling a default set of histogram-buckets for histograms and timers
+
+Vendors may choose to optionally provide the `mp.metrics.distribution.percentiles-histogram.enabled` property. This will enable a matching histogram or timer metric to output a default set of bucket values defined by the vendor. The property follows these rules:
+
+* The property accepts a semi-colon separated set of values that consist of a metric name followed by an equals sign (`=`) and either `true` or `false` (see <<default-buckets-sample,example>> below).
+* The asterisk (`*`) can be used as a wildcard at the end of the metric name.
+* Defining a metric name with no values or invalid values has no effect.
+* Values defined later take precedence over values before (i.e., right to left precedence).
+
+[[default-buckets-sample]]
+.MicroProfile Config `mp.metrics.distribution.percentiles-histogram.enabled` property examples
+----
+//vendor will provide default buckets for the alpha.timer 
+mp.metrics.distribution.percentiles-histogram.enabled=alpha.timer=true;alpha.histogram=false
+----
+
+NOTE: The `mp.metrics.distribution.percentiles-histogram.enabled` property does not affect any buckets defined by the `mp.metrics.distribution.histogram.buckets` or `mp.metrics.distribution.timer.buckets` properties. A `false` value for `mp.metrics.distribution.percentiles-histogram.enabled` will not disable any custom buckets defined by `mp.metrics.distribution.histogram.buckets` or `mp.metrics.distribution.timer.buckets` for any matching histogram or timer.
+
+It is recommended that if the vendor provides support for the `mp.metrics.distribution.percentiles-histogram.enabled` property, then the following properties be supported as well:
+
+* `mp.metrics.distribution.histogram.min-value`
+* `mp.metrics.distribution.histogram.max-value`
+* `mp.metrics.distribution.timer.min-value` 
+* `mp.metrics.distribution.timer.max-value`
+
+The number of default buckets and maximum value of each of those default buckets may vary between vendor implementations. These properties are used to set a minimum and maximum limit for the default buckets provided when `mp.metrics.distribution.percentiles-histogram.enabled` has enabled default buckets for the matching metric. The properties share the following rules and explicit rules are outlined for specific properties:
+
+* The property accepts a semi-colon separated set of values that define a metric name followed by an equals sign (`=`) and a value:
+** `mp.metrics.distribution.histogram.*` properties accept float or integer values greater than 0 _(See <<min-max-sample,examples>> below)_.
+** `mp.metrics.distribution.timer.*` accepts an integer value with an appended time unit (valid time units are `ms` for milliseconds, `s` for seconds, `m` for minutes and `h` for hours). Float values will not be accepted. Values with no time unit will default to milliseconds. _(See <<min-max-sample,examples>> below)_
+* The asterisk (`*`) can be used as a wildcard at the end of the metric name.
+* Defining a metric name with no value or an invalid value provides no effect.
+* Values defined later take precedence over values before (i.e., right to left precedence).
+
+
+
+
+[[min-max-sample]]
+.MicroProfile Config histogram/timer min/max property examples
+----
+//any histogram matching alpha.* will not have any buckets in the output that are below 300 except alpha.histogram which has a lower bound of 400 due to precedence
+mp.metrics.distribution.histogram.min-value=alpha.*=300;alpha.histogram=400
+
+//any histogram matching alpha.* will not have any buckets in the output that are above 500 except alpha.histogram which has a upper bound of 400 due to precedence
+mp.metrics.distribution.histogram.max-value=alpha.*=500;alpha.histogram=400
+
+//any timer matching alpha.* will not have any buckets in the output that are above 1 second except alpha.timer which has a upper bound of 0.5 seconds due to precedence
+mp.metrics.distribution.timer.max-value=alpha.*=1s;alpha.timer=500ms
+
+//any timer matching alpha.* will not have any buckets in the output that are below 0.4 seconds except alpha.timer which has a lower bound of 0.5 seconds due to precedence
+mp.metrics.distribution.timer.min-value=alpha.*=400ms;alpha.timer=500ms
+----
+
+NOTE: The use of the min and max properties do not apply to the buckets defined through `mp.metrics.distribution.timer.buckets` and `mp.metrics.distribution.histogram.buckets` properties.

spec/src/main/asciidoc/microprofile-metrics-spec.asciidoc

@@ -57,6 +57,8 @@ include::base-metrics.adoc[]
 
 include::app-programming-model.adoc[]
 
+include::histogram-timer-config.adoc[]
+
 include::micrometer-backends.adoc[]
 
 include::appendix.adoc[]