Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify MeterProvider configurable cardinality limits #2960

Merged
merged 37 commits into from
May 8, 2023
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
61264c7
Specify MetricReader configurable limits
jmacd Nov 18, 2022
7ba02d1
Add PR number and spec-matrix entry
jmacd Nov 18, 2022
16f6170
edits
jmacd Nov 18, 2022
82fb36d
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
jmacd Nov 28, 2022
0f78e17
changelog order
jmacd Nov 28, 2022
26b2ce9
corrections, update spec matrix
jmacd Nov 28, 2022
57d35a5
Update specification/metrics/sdk.md
jmacd Nov 30, 2022
e755ee6
Update specification/metrics/sdk.md
jmacd Nov 30, 2022
00c44f6
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
jmacd Dec 12, 2022
8c87107
Merge branch 'jmacd/metrics_limits' of github.com:jmacd/opentelemetry…
jmacd Dec 12, 2022
20c7555
revision
jmacd Dec 13, 2022
dcbcaf3
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
jmacd Dec 16, 2022
7cdc7ab
toc
jmacd Dec 16, 2022
f12c8b5
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
jmacd Jan 4, 2023
df16924
Apply suggestions from code review
jmacd Jan 6, 2023
1d7a117
Merge branch 'main' into jmacd/metrics_limits
reyang Jan 17, 2023
15bce36
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
jmacd Feb 9, 2023
d116a28
Update with ideas from Jan Spec SIG
jmacd Feb 9, 2023
a92c4c7
Merge branch 'jmacd/metrics_limits' of github.com:jmacd/opentelemetry…
jmacd Feb 9, 2023
c72fae4
refine text
jmacd Feb 10, 2023
bfbf348
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
jmacd Mar 3, 2023
6cd74e0
Revise based on prototype in otel-launcher-go
jmacd Mar 3, 2023
0e59093
revise spelling
jmacd Mar 3, 2023
c4a5a61
revise spelling
jmacd Mar 3, 2023
7e2bf6d
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
jmacd Mar 27, 2023
81ae9e7
clarify wording; give MetricReader control over the default
jmacd Mar 27, 2023
88c1557
reword
jmacd Mar 27, 2023
fbf66c7
use aggregation_cardinality_limit
jmacd Mar 28, 2023
e53c22d
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
jmacd Apr 17, 2023
f1f47f4
clarify attribute SETs h/t @pirgeo
jmacd Apr 17, 2023
a0a6596
Update specification/metrics/sdk.md
jmacd Apr 17, 2023
7099f07
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
jmacd Apr 19, 2023
7505081
update changelog
jmacd Apr 19, 2023
9569ef5
lint
jmacd Apr 19, 2023
1a98c25
Update specification/metrics/sdk.md
jmacd May 4, 2023
4aaad07
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
jmacd May 5, 2023
cee0c74
Merge branch 'jmacd/metrics_limits' of github.com:jmacd/opentelemetry…
jmacd May 5, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ release.

- Define Experimental MetricProducer as a third-party provider of metric data to MetricReaders.
([#2951](https://github.com/open-telemetry/opentelemetry-specification/pull/2951))
- Recommended limits applied by default at the MetricProvider level to protect
metrics pipelines against excessive data production from a single
instrumentation library or instrument.
([#2960](https://github.com/open-telemetry/opentelemetry-specification/pull/2960))

### Logs

Expand Down
1 change: 1 addition & 0 deletions spec-compliance-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ formats is required. Implementing more than one format is optional.
| The API provides a way to set and get a global default `MeterProvider`. | X | + | + | + | + | | | | | + | - | |
| It is possible to create any number of `MeterProvider`s. | X | + | + | + | + | | | | | + | + | |
| `MeterProvider` provides a way to get a `Meter`. | | + | + | + | + | | | | | + | - | |
| `MeterProvider` provides a way to set cardinality limits. | X | | | | | | | | | | | |
| `get_meter` accepts name, `version` and `schema_url`. | | + | + | + | + | | | | | + | - | |
| `get_meter` accepts `attributes`. | | | | | | | | | | | | |
| When an invalid `name` is specified a working `Meter` implementation is returned as a fallback. | | + | + | + | + | | | | | + | - | |
Expand Down
58 changes: 58 additions & 0 deletions specification/metrics/sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ linkTitle: SDK
* [Observations inside asynchronous callbacks](#observations-inside-asynchronous-callbacks)
* [Resolving duplicate instrument registration conflicts](#resolving-duplicate-instrument-registration-conflicts)
- [Attribute limits](#attribute-limits)
- [Cardinality limits](#cardinality-limits)
- [Exemplar](#exemplar)
* [ExemplarFilter](#exemplarfilter)
* [ExemplarReservoir](#exemplarreservoir)
Expand Down Expand Up @@ -70,6 +71,12 @@ metrics produced by any `Meter` from the `MeterProvider`. The [tracing SDK
specification](../trace/sdk.md#additional-span-interfaces) has provided some
suggestions regarding how to implement this efficiently.

A `MeterProvider` SHOULD provide a way to specify the configurable
[cardinality limits](#cardinality-limits) to be applied to `Meter`
instances. SDKs that support configurable limits MUST provide an
option to configure the default limits to be applied to all `Meter`
instances by default.

### Meter Creation

New `Meter` instances are always created through a `MeterProvider`
Expand Down Expand Up @@ -586,6 +593,57 @@ Attributes which belong to Metrics are exempt from the
time. Attribute truncation or deletion could affect identity of metric time
series and the topic requires further analysis.

## Cardinality limits

**Status**: [Experimental](../document-status.md)

MeterProviders SHOULD support being configured with limits to be applied
to metrics produced from a single Instrumentation Library Scope. This
mechanism supports protecting metrics pipelines from excessive da
ta
jmacd marked this conversation as resolved.
Show resolved Hide resolved
production, which is usually caused by a problem known as ["explosition
jmacd marked this conversation as resolved.
Show resolved Hide resolved
of cardinality"](supplementary-guidelines.md#synchronous-example-cumulative-aggregation-temporality),
in which the number of timeseries produced by application code exceeds a
reasonable number. These limits help protect the overall system from
jmacd marked this conversation as resolved.
Show resolved Hide resolved
individual sources of excessive metrics instrumentation.

When the limits are reached by an the individual instrumentation
library scope, the entire batch of metrics for the scope MUST be
dropped and an error reported to the user indicating:
jmacd marked this conversation as resolved.
Show resolved Hide resolved

- Detail about the limit that was exceeded (e.g., the library name and
version, the configured limit, the instruments of that library
ranked by cardinality).
- Recommend the user to configure a [View](#view) that filters the
offending cardinality or drops selected instruments to correct the problem.

Note that limits expressed here refer to exported data for a single
collection cycle. The number of timeseries referred to in each of
these limits equals the number of distinct data points produced over
one interval.

Under some circumstances, depending on the configured aggregation
temporality, the MeterProvider may be able to recover after these
limits are reached. In particular, for Synchronous instruments
configured with Delta aggregation temporality, the MeterProvider
SHOULD remove timeseries from memory when they become stale, making it
jmacd marked this conversation as resolved.
Show resolved Hide resolved
possible to recover from an explosion of cardinality from one
collection cycle to the next provided the application stops using
excessive cardinality. However, for MeterProviders that have
Synchronous instruments configured with Cumulative aggregation
temporality, once a limit is reached the MeterProvider SHOULD disable
it for the remaining lifetime of the MeterProvider.

The configurable limits are:

1. The number of timeseries per instrument, with recommended default value 2000.
2. The number of timeseries per instrumentation library scope, with recommended default value 2000.

For both of these limits, the limit MAY be unset to indicate an
unlimited configuration and the zero value is considered invalid.
jmacd marked this conversation as resolved.
Show resolved Hide resolved
MeterProviders SHOULD NOT use an unlimited memory configuration by
default.

## Exemplar

**Status**: [Feature-freeze](../document-status.md)
Expand Down