Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[receiver/prometheusreceiver]: trim type and unit suffixes from metric name #16071

Merged
merged 4 commits into from
Nov 18, 2022
Merged

[receiver/prometheusreceiver]: trim type and unit suffixes from metric name #16071

merged 4 commits into from
Nov 18, 2022

Conversation

paivagustavo
Copy link
Member

@paivagustavo paivagustavo commented Nov 3, 2022 •
edited
Loading

Description: Trim type's and unit's suffixes from metric name as per otel specs.

There was a lot of changes required to pass down a featuregate.Registry in order to be testable. The important bit in the receiver code path is the metricFamily.appendMetric which trims the suffixes when creating an OTLP metric.

Units suffixes will only take effect when using the OpenMetrics format, since the prometheus parser doesn't support units.

This is being guarded by the gate: --feature-gates=pkg.translator.prometheus.NormalizeName

Link to tracking Issue: #8950

Testing: Units tests were added for the translator package, and prometheus receiver tests updated.

@dashpole
Copy link
Contributor

dashpole commented Nov 4, 2022

Can you put this behind the --feature-gates=pkg.translator.prometheus.NormalizeName feature gate?

@paivagustavo paivagustavo changed the title prometheusreceiver: trim type and unit suffixes from metric name [receiver/prometheusreceiver]: trim type and unit suffixes from metric name Nov 4, 2022
@paivagustavo paivagustavo force-pushed the gustavo/promreceiver/normalize_metric_suffixes branch from ea6b8fd to d8ca297 Compare November 4, 2022 20:16
@paivagustavo
Copy link
Member Author

@dashpole I've guarded this with the featuregate as asked, and added some specific tests for this on the receiver side as well.

@dashpole
Copy link
Contributor

dashpole commented Nov 7, 2022

One of the tests had an invalid case, exposing both foo and a foo_total metric

Note that this case is valid in prometheus format, just not in OpenMetrics format.

@paivagustavo paivagustavo force-pushed the gustavo/promreceiver/normalize_metric_suffixes branch 3 times, most recently from feb607f to 9d23af1 Compare November 7, 2022 20:30
@paivagustavo
Copy link
Member Author

Note that this case is valid in prometheus format, just not in OpenMetrics format.

That makes sense, reverted those tests changes :)

@paivagustavo paivagustavo force-pushed the gustavo/promreceiver/normalize_metric_suffixes branch from 9d23af1 to c592716 Compare November 9, 2022 14:50
@dashpole dashpole mentioned this pull request Nov 11, 2022
Closed
@paivagustavo paivagustavo force-pushed the gustavo/promreceiver/normalize_metric_suffixes branch from 354d892 to 79073b9 Compare November 14, 2022 14:18
@paivagustavo
Copy link
Member Author

Seems like TestEnsureRecordedMetrics fromgithub.com/open-telemetry/opentelemetry-collector-contrib/receiver/opencensusreceiver/internal/ocmetrics is flaky. Just failed at another commit on main: https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/3464963384/jobs/5787150358.

@dashpole do you mind retrying please?

@paivagustavo paivagustavo force-pushed the gustavo/promreceiver/normalize_metric_suffixes branch from fdd1e62 to 5550604 Compare November 15, 2022 15:47
This was referenced Nov 15, 2022
Merged
Open
@codeboten codeboten merged commit 27c9f42 into open-telemetry:main Nov 18, 2022
@codeboten codeboten deleted the gustavo/promreceiver/normalize_metric_suffixes branch November 18, 2022 18:17
JaredTan95 pushed a commit to openinsight-proj/opentelemetry-collector-contrib that referenced this pull request Nov 21, 2022
@paivagustavo @JaredTan95
[receiver/prometheusreceiver]: trim type and unit suffixes from metri…
af1b7bc 
…c name (open-telemetry#16071)

 Trim type's and unit's suffixes from metric name as per otel specs.

There was a lot of changes required to pass down a featuregate.Registry in order to be testable. The important bit in the receiver code path is the metricFamily.appendMetric which trims the suffixes when creating an OTLP metric.

Units suffixes will only take effect when using the OpenMetrics format, since the prometheus parser doesn't support units.

This is being guarded by the gate: --feature-gates=pkg.translator.prometheus.NormalizeName
JaredTan95 added a commit to openinsight-proj/opentelemetry-collector-contrib that referenced this pull request Nov 24, 2022
@JaredTan95 @bogdandrutu
@aboguszewski-sumo @TylerHelmuth @djaglowski @schmikei @MovieStoreGuy @github-actions @codeboten @atoulme @songy23 @mdelapenya @nityanandagohain @singku @dmitryax @evan-bradley @atingchen @wzy531a @sethAmazon @dloucasfx @opentelemetrybot @pcwiese @paivagustavo
[processor/servicegraph]speculate virtual node before edge get expire…
91fd307 
…d. (#7)

* [chore] remove what looks to be debugging code from instanaexporter (open-telemetry#16258)

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>

* [receiver/elasticsearch]: add fielddata memory size metrics on index level (open-telemetry#14922)

feat: add fielddata memory size metrics on index level

* Register featuregate, otherwise does not work (open-telemetry#16269)

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>

* [pkg/ottl] Rename ottllogs to ottllog (open-telemetry#16242)

* Rename ottllogs to ottllog

* add changelog entry

* fix lint

* [pkg/stanza] Fix issue where specifying a non-existent storage extension caused panic during shutdown. (open-telemetry#16213)

* [chore] update jaeger dep (open-telemetry#16287)

Moving dependency to version in main to move open-telemetry/opentelemetry-collector#6511 in core forward.

* [receiver/mongodbatlas] Alerts poll mode check hostname and port (open-telemetry#16286)

* check hostname and port before converting polled logs

* [Docker Stats Receiver] Add @jamesmoessis as code owner (open-telemetry#16253)

Adding @jamesmoessis as code owner of Docker Stats Receiver

* [receiver/rabbitmq] Fix flaky integration test (open-telemetry#16240)

This test fails periodially with a non-zero exit code returned from
the setup script. The likely cause is that the wait strategy was not
robust enough, in that the rabbitmqadmin command was not ready before
associated commands were run.

This fix incorporates the setup script into a custom wait strategy,
which allows the script to be rerun until it succeeds.

* [chore] dependabot updates Tue Nov 15 00:17:56 UTC 2022 (open-telemetry#16300)

dependabot updates Tue Nov 15 00:17:54 UTC 2022
Bump github.com/aerospike/aerospike-client-go/v6 from 6.4.0 to 6.6.0 in /receiver/aerospikereceiver
Bump github.com/aws/aws-sdk-go-v2/config from 1.17.10 to 1.18.0 in /extension/sigv4authextension
Bump go.mongodb.org/atlas from 0.18.0 to 0.19.0 in /receiver/mongodbatlasreceiver
Bump google.golang.org/api from 0.102.0 to 0.103.0 in /receiver/googlecloudspannerreceiver
Bump k8s.io/api from 0.25.3 to 0.25.4 in /processor/k8sattributesprocessor
Bump k8s.io/apimachinery from 0.25.3 to 0.25.4 in /internal/k8sconfig
Bump k8s.io/apimachinery from 0.25.3 to 0.25.4 in /processor/resourcedetectionprocessor
Bump k8s.io/client-go from 0.25.3 to 0.25.4 in /processor/k8sattributesprocessor

Co-authored-by: codeboten <codeboten@users.noreply.github.com>

* [exporter/splunkhecexporter] close HTTP servers used during testing (open-telemetry#16305)

* [exporter/datadog] Bump Datadog Agent Go modules (open-telemetry#16294)

* fix: decouple mongoDB integration tests (open-telemetry#16306)

The container requests where using same fixed, exposed port, instead of
using the random one. In this commit we are calculating the exposed port
in the getContainer method, returning the endpoint for the container
in order to be used by each test

* [receiver/elasticsearch]: add segment memory metric on node level (open-telemetry#16172)

feat: add segments memory metric on node level

* [chore]: fix port feature gate's description in apachereceiver (open-telemetry#16280)

* fix: fix port feature gate's description

* remove changelog

* [operator/trace_parser] Example corrected - body changed to attributes (open-telemetry#16309)

Example corrected :- body changed to attributes

* chore: use testcontainers-go instead of containertest (open-telemetry#16264)

* chore: use testcontainers-go in zookeper receiver

* chore: use testcontainers-go in memcached receiver

* chore: use testcontainers-go in docker stats receiver

* chore: use testcontainers-go in aerospike receiver

* chore: use testcontainers-go in kafka metrics receiver

* chore: use testcontainers-go in docker observer extension

* [processor/transform] Allow specifying functions per context (open-telemetry#16251)

* Allow specifying functions per context

* [Receiver/redisreceiver] Support redis.cpu.time with more label values for Redis versions >= 6.0 (open-telemetry#14943)

Update Redis Receiver to Support CPU Metrics After 6.0

Add more state values to redis.cpu.time metrics to support sys_main_thread and user_main_thread cpu usage time.

* [chore] Don't use deprecated [Span|Trace]ID.HexString method (open-telemetry#16301)

The methods were deprecated since there is no establish convention on how to decode empty trace or span IDs at this point. The current method behavior is preserved in functions added in internal/coreinternal/traceutil. This commit updates all the usages of HexString where the result is used as identifier to the new internal methods.

* [exporter/tanzuobservability] Don't encode/decode TraceID as string (open-telemetry#16302)

Avoid unnecessary string encoding and decoding of trace ID. Create UUID from bytes directly

* [pkg/ottl] Commonize logic for handling slice and non-slice args (open-telemetry#16298)

* Commonize logic for handling slice and non-slice args

* Add PR number for issue reference

Co-authored-by: Evan Bradley <evan-bradley@users.noreply.github.com>

* [chore] Delete changelog entry for a test change (open-telemetry#16318)

* [chore] Update collector core dependency (open-telemetry#16317)

1. In adoption to deprecation of [Span|Trace]ID.HexString method, this change includes migration to [Span|Trace]ID implementing fmt.Stringer interface for display purposes.
2. Adopt to service.State enum values renames

* chore: retire containertest (open-telemetry#16316)

It's not used anymore, in favor of testcontainers-go

* [chrony receiver] Fixing flake (open-telemetry#16323)

* [chore] [exporter/splunkhecexporter] make large_batch test less demanding (open-telemetry#16322)

Make flaky test less demanding, and add more checks to make sure the test actually tests a behavior of the exporter

* [exporter/splunkhecexporter] Do not log a warning on mapping empty metrics (open-telemetry#16324)

[exporter/splunkhecexporter] Do not log a warning on mapping empty metrics.

* [examples/demo] fix demo could not stat up (open-telemetry#16169)

* fix demo could not stat up

Signed-off-by: Jared Tan <jian.tan@daocloud.io>

* remove unnecessary GOOS=linux

Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>

* [pkg/stanza] Support to Customize bufio.SplitFunc (open-telemetry#16272)

* add a new method BuildWithSplitFunc, this method can directly pass in a user-defined splitFunc

* [receiver/elasticsearch]: add query cache metrics on index level (open-telemetry#14873)

* feat: add query cache metrics on index level

* feat: add cache.size metric

* fix: remove cache name attribute from size metric

* [chore] Add 'goporto' and 'multimod-verify' to default make target (open-telemetry#16328)

Also adds prerequisite targets to a couple targets so they can be run
independently.

* Propose @evan-bradley as approver (open-telemetry#16329)

cc: @open-telemetry/collector-contrib-approvers

Signed-off-by: Alex Boten <aboten@lightstep.com>

* [docs] [chore] Update Development stability links in the Readme headers (open-telemetry#16334)

* Propose @MovieStoreGuy as approver (open-telemetry#16330)

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
Co-authored-by: Alex Boten <aboten@lightstep.com>

* [chore] Disable flaky windows file rotation tests (open-telemetry#16332)

* [receiver/vcenter]Fix nil pointer exception when vcenter receiver collectVMs. (open-telemetry#16278)

Fix nil pointer exception when vcenter receiver collectVMs.

* [pkg/stanza] Fix serverity range unmarshaling (open-telemetry#16339)

This functionality was broken during refactoring of the unmarshaling logic.

Previously, various yaml types could be used to represent a severity level.
This was designed to be as open as possible to future implementations. However,
it is now clear that only some keys are valid anyways. Namely, string and ints
that represent OTel's native severity levels. Both strings and ints are now treated
as strings when unmarshaling, which is fully compatible with the sev map lookup code.

* [receiver/elasticsearch]: add document count metrics on index level (open-telemetry#14874)

feat: add document count metrics on index level

* [receiver/elasticsearch]: add cluster health metrics for two more shard types (open-telemetry#14875)

* feat: add cluster health metrics for two more shard types

* Add Cumulative Delta Include All Unit Tests (open-telemetry#16350)

* Upgrade core, fix usage of deprecated Config.Validate (open-telemetry#16362)


Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>

* [kafka Exporter] refactor tests (open-telemetry#16265)

These tests needed a little love to make it easier to contribute to the
component

Co-authored-by: Alex Boten <alex@boten.ca>

* [receiver/mongo] Fix flaky test TestScraperLifecycle (open-telemetry#16364)

by increasing the expected time to start and stop

* [chore] Add new approvers to remaining lists (open-telemetry#16367)

Add new approvers to remaining lists

Co-authored-by: Evan Bradley <evan-bradley@users.noreply.github.com>

* [chore] Update autoassign.yaml with new approvers (open-telemetry#16360)

Update autoassign.yaml

* [processor/transform] Use proper context name in readme (open-telemetry#16351)

Use proper context name in readme

* [exporter/signalfxexporter] allow user to add their own CA to the cert pool (open-telemetry#16250)

Adding new feature to allow the user to add their own CA to the ingest and api client cert pool.
This is needed when the exporter is pointing to a TLS enabled signalfx receiver or/and TLS enabled http_forwarder
and the CA is not in the system cert pool (ex: agent <--> GW setup)

Signed-off-by: Dani Louca <dlouca@splunk.com>
Co-authored-by: Dmitrii Anoshin <anoshindx@gmail.com>

* [receiver/elasticsearch]: add missing datapoints for operation count and operation time metrics (open-telemetry#16170)

* feat: add missing operation count datapoints with total aggregation

* feat: add missing operation count datapoints with primary shards aggregation

* feat: add missing operation time datapoints with total shard aggregation

* feat: add missing operation time datapoints with primary shards aggregations

* Update the jmx-metrics version to 1.20.0-alpha (open-telemetry#16356)

* Update the jmx-metrics version to

* Update add-jmx-metrics-gatherer-1.20.0-alpha.yaml

Co-authored-by: Alex Boten <alex@boten.ca>

* [exporter/azuremonitor] Support span and exception events (open-telemetry#16354)

* [exporter/azuremonitor] Support span and exception events

Exception events flow to the AppInsights Exception table
Span events flow to the Traces table, when enabled.

* Add changelog entry

* Exporting span events is off by default

* [receiver/prometheusreceiver]: trim type and unit suffixes from metric name (open-telemetry#16071)

 Trim type's and unit's suffixes from metric name as per otel specs.

There was a lot of changes required to pass down a featuregate.Registry in order to be testable. The important bit in the receiver code path is the metricFamily.appendMetric which trims the suffixes when creating an OTLP metric.

Units suffixes will only take effect when using the OpenMetrics format, since the prometheus parser doesn't support units.

This is being guarded by the gate: --feature-gates=pkg.translator.prometheus.NormalizeName

* [pkg/ottl] Add support for setting maps in Values (open-telemetry#16352)

 Adds support for setting maps in Values. The primary impact is the ability for contexts to set maps as attribute values.

* Remove redundant dependency on Jaeger collector (open-telemetry#16370)

We used only one constant for error message from "github.com/jaegertracing/jaeger/cmd/collector/app/handler"

* [receiver/jaeger] Fix an error message in thrift HTTP message decoder (open-telemetry#16372)

Return full error instead of format string "Unable to process request body: %v"

* [chore] update more tests to bind to 127.0.0.1 (open-telemetry#16341)

Signed-off-by: Alex Boten <aboten@lightstep.com>

Signed-off-by: Alex Boten <aboten@lightstep.com>

* [chore]: rename files in filterprocessor, use RemoveIf once (open-telemetry#16376)

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>

* enhancement servicegraph processor virtual node winthin edge.

Signed-off-by: Jared Tan <jian.tan@daocloud.io>

* update peer virtual node.

Signed-off-by: Jared Tan <jian.tan@daocloud.io>

* update the match attributes.

Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Signed-off-by: Alex Boten <aboten@lightstep.com>
Signed-off-by: Dani Louca <dlouca@splunk.com>
Co-authored-by: Bogdan Drutu <bogdandrutu@gmail.com>
Co-authored-by: Adam Boguszewski <108867528+aboguszewski-sumo@users.noreply.github.com>
Co-authored-by: Tyler Helmuth <12352919+TylerHelmuth@users.noreply.github.com>
Co-authored-by: Daniel Jaglowski <jaglows3@gmail.com>
Co-authored-by: Alex Boten <aboten@lightstep.com>
Co-authored-by: Keith Schmitt <32067685+schmikei@users.noreply.github.com>
Co-authored-by: Sean Marciniak <30928402+MovieStoreGuy@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: codeboten <codeboten@users.noreply.github.com>
Co-authored-by: Antoine Toulme <antoine@lunar-ocean.com>
Co-authored-by: Yang Song <songy23@users.noreply.github.com>
Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>
Co-authored-by: Nityananda Gohain <nityanandagohain@gmail.com>
Co-authored-by: Andrew Tang <tangliang551@gmail.com>
Co-authored-by: Dmitrii Anoshin <anoshindx@gmail.com>
Co-authored-by: Evan Bradley <github@evanbradley.org>
Co-authored-by: Evan Bradley <evan-bradley@users.noreply.github.com>
Co-authored-by: look <lookchen@tencent.com>
Co-authored-by: wzy531a <wu.zhaoyin@yottabyte.cn>
Co-authored-by: Seth L <81644108+sethAmazon@users.noreply.github.com>
Co-authored-by: Alex Boten <alex@boten.ca>
Co-authored-by: Dani Louca <59848726+dloucasfx@users.noreply.github.com>
Co-authored-by: OpenTelemetry Bot <107717825+opentelemetrybot@users.noreply.github.com>
Co-authored-by: Peter Wiese <pwiese@microsoft.com>
Co-authored-by: Gustavo Paiva <gustavo@lightstep.com>
shalper2 pushed a commit to shalper2/opentelemetry-collector-contrib that referenced this pull request Dec 6, 2022
@paivagustavo @shalper2
[receiver/prometheusreceiver]: trim type and unit suffixes from metri…
29cff4f 
…c name (open-telemetry#16071)

 Trim type's and unit's suffixes from metric name as per otel specs.

There was a lot of changes required to pass down a featuregate.Registry in order to be testable. The important bit in the receiver code path is the metricFamily.appendMetric which trims the suffixes when creating an OTLP metric.

Units suffixes will only take effect when using the OpenMetrics format, since the prometheus parser doesn't support units.

This is being guarded by the gate: --feature-gates=pkg.translator.prometheus.NormalizeName
@plantfansam plantfansam mentioned this pull request Jul 21, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@codeboten codeboten codeboten approved these changes

@Aneurysm9 Aneurysm9 Aneurysm9 approved these changes

@dashpole dashpole dashpole approved these changes

@jmacd jmacd jmacd approved these changes

@dmitryax dmitryax Awaiting requested review from dmitryax

Assignees

@codeboten codeboten

Labels
pkg/translator/prometheus receiver/prometheus Prometheus receiver receiver/prometheusexec receiver/simpleprometheus
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@paivagustavo @dashpole @codeboten @Aneurysm9 @jmacd