[otlp] Update OTel Collector PRW translate package to 0.73.0 #4063

kovrus · 2023-01-24T11:16:30Z

What this PR does

Updates the OTel Collector PRW translate package to 0.73.0. The PR contains the following user-facing changes:

Add support for converting OTLP Exponential Histograms to Prometheus Native Histograms.
Export _created metric for Summary, Histogram, and Monotonic Sum metric points if StartTimeUnixNano is set. It is disabled by default, the configuration parameter can be exposed in the future.
Do not drop exemplars of the OTLP Monotonic Sum metric.

Which issue(s) this PR fixes or relates to

Fixes #

Checklist

Tests updated
Documentation added
CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

CLAassistant · 2023-01-24T11:16:42Z

All committers have signed the CLA.

gouthamve · 2023-01-24T12:05:22Z

Can you update the end to end test to include an exponential histogram?

pracucci · 2023-01-24T13:39:22Z

Please also remember to sign the CLA. Thanks!

CHANGELOG.md

kovrus · 2023-02-17T15:47:23Z

Can you please take a look at PR again? I've added a simple test case https://github.com/grafana/mimir/pull/4063/files#diff-57b597f7e1c988306aeac3c21175be693dd901c035761195212f24ca0a4a96b6R79 and updated the change log. (sorry, squashed the commits with addressing the feedback)

integration/otlp_ingestion_test.go

pracucci

I think I don't understand how this PR can adds supports to ingest exponential histograms as prometheus native histograms. Shouldn't we map them in promToMimirTimeseries() defined in pkg/util/push/otel.go?

pracucci · 2023-02-23T16:46:45Z

CHANGELOG.md

+* [ENHANCEMENT] OTLP: Add support for converting OTel Exponential Histograms to Prometheus Native Histograms. #4063
+* [BUGFIX] OTLP: Do not drop exemplars of the OTLP Monotonic Sum metric. #4063


Please move these CHANGELOG entries below, under "### Grafana Mimir".

pstibrany · 2023-03-09T09:11:54Z

The CHANGELOG has just been cut to prepare for the next Mimir release. Please rebase main and eventually move the CHANGELOG entry added / updated in this PR to the top of the CHANGELOG document. Thanks!

pracucci · 2023-03-09T09:14:12Z

@kovrus Any update on this PR?

kovrus · 2023-03-09T11:09:59Z

@pracucci didn't have a chance to work on it. I'll implement the missing mimir to prom ts conversion today or tomorrow.

kovrus · 2023-03-10T14:35:05Z

The CHANGELOG has just been cut to prepare for the next Mimir release. Please rebase main and eventually move the CHANGELOG entry added / updated in this PR to the top of the CHANGELOG document. Thanks!

Under ## main / unreleased or ### Grafana Mimir?

kovrus · 2023-03-15T16:10:36Z

@pracucci @pstibrany can you take another look?

pkg/util/push/otel.go

integration/otlp_ingestion_test.go

CHANGELOG.md

pkg/util/push/otel.go

pstibrany · 2023-03-20T13:50:17Z

kovrus requested a review from pstibrany

Just FYI: waiting for updated integration test that queries native histograms back after ingesting them.

kovrus · 2023-03-22T08:56:48Z

kovrus requested a review from pstibrany

Just FYI: waiting for updated integration test that queries native histograms back after ingesting them.

updated

pstibrany

LGTM, thank you.

pkg/util/push/otel.go

It contains the following user facing changes: - Add support for converting OTLP Exponential Histograms to Prometheus Native Histograms. - Export `_created` metric for Summary, Histogram and Monotonic Sum metric points if StartTimeUnixNano is set. It is disabled by default, the configuration parameter can be exposed in future. - Do not drop exemplars of the OTLP Monotonic Sum metric.

Updated prw to 0.73

pracucci · 2023-03-31T12:38:50Z

pkg/util/push/otel.go

 	ts.Exemplars = exemplars

 	return mimirpb.PreallocTimeseries{TimeSeries: ts}
 }

+func promToMimirHistogram(h *prompb.Histogram) mimirpb.Histogram {
+	pSpans := make([]mimirpb.BucketSpan, len(h.PositiveSpans))


The preallocation done here and below is incorrect. I opened a PR to fix it: #4639

kovrus changed the title ~~[otlp] Update OTel Collector PRW translate dependency to 0.70.0~~ [otlp] Update OTel Collector PRW translate package to 0.70.0 Jan 24, 2023

kovrus force-pushed the otlp-dep-update branch from 1fe94c8 to 7c663fc Compare January 24, 2023 11:25

kovrus marked this pull request as ready for review January 24, 2023 11:35

kovrus requested a review from a team as a code owner January 24, 2023 11:35

kovrus requested a review from gouthamve January 24, 2023 11:36

pracucci reviewed Jan 24, 2023

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

pracucci reviewed Jan 24, 2023

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

kovrus force-pushed the otlp-dep-update branch 4 times, most recently from 5e9e76e to 367386c Compare February 17, 2023 13:22

kovrus changed the title ~~[otlp] Update OTel Collector PRW translate package to 0.70.0~~ [otlp] Update OTel Collector PRW translate package to 0.71.0 Feb 17, 2023

kovrus requested review from pracucci and krajorama February 17, 2023 15:47

codesome reviewed Feb 23, 2023

View reviewed changes

integration/otlp_ingestion_test.go Outdated Show resolved Hide resolved

codesome reviewed Feb 23, 2023

View reviewed changes

integration/otlp_ingestion_test.go Outdated Show resolved Hide resolved

pracucci reviewed Feb 23, 2023

View reviewed changes

pstibrany added the release/notified-changelog-cut label Mar 9, 2023

kovrus force-pushed the otlp-dep-update branch 2 times, most recently from 527af5c to 9c79819 Compare March 10, 2023 14:33

kovrus force-pushed the otlp-dep-update branch from 9c79819 to cfab216 Compare March 10, 2023 14:35

kovrus changed the title ~~[otlp] Update OTel Collector PRW translate package to 0.71.0~~ [otlp] Update OTel Collector PRW translate package to 0.73.0 Mar 10, 2023

kovrus requested a review from pracucci March 15, 2023 16:10

kovrus force-pushed the otlp-dep-update branch 3 times, most recently from b83a3ca to d3a5bc6 Compare March 16, 2023 15:11

pstibrany reviewed Mar 17, 2023

View reviewed changes

kovrus force-pushed the otlp-dep-update branch from 381c462 to 6183cae Compare March 17, 2023 10:55

pstibrany reviewed Mar 17, 2023

View reviewed changes

pkg/util/push/otel.go Outdated Show resolved Hide resolved

kovrus requested a review from pstibrany March 17, 2023 12:35

kovrus force-pushed the otlp-dep-update branch 8 times, most recently from a3978be to e52f8c7 Compare March 21, 2023 20:04

kovrus force-pushed the otlp-dep-update branch from e52f8c7 to f709355 Compare March 22, 2023 09:48

pstibrany approved these changes Mar 23, 2023

View reviewed changes

pkg/util/push/otel.go Outdated Show resolved Hide resolved

kovrus added 4 commits March 24, 2023 12:27

Implement prom to mimir series conversion.

4d9c2ae

Updated prw to 0.73

Address comments.

694c604

Add buckets layouts translation.

803c210

kovrus force-pushed the otlp-dep-update branch from 27302c2 to 803c210 Compare March 24, 2023 11:28

pstibrany merged commit 18b7f91 into main Mar 27, 2023

pstibrany deleted the otlp-dep-update branch March 27, 2023 10:24

pracucci mentioned this pull request Mar 31, 2023

Fix promToMimirHistogram() slices pre-allocation #4639

Merged

3 tasks

pracucci reviewed Mar 31, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[otlp] Update OTel Collector PRW translate package to 0.73.0 #4063

[otlp] Update OTel Collector PRW translate package to 0.73.0 #4063

kovrus commented Jan 24, 2023 •

edited

Loading

CLAassistant commented Jan 24, 2023 •

edited

Loading

gouthamve commented Jan 24, 2023

pracucci commented Jan 24, 2023

kovrus commented Feb 17, 2023 •

edited

Loading

pracucci left a comment

pracucci Feb 23, 2023

pstibrany commented Mar 9, 2023

pracucci commented Mar 9, 2023 •

edited

Loading

kovrus commented Mar 9, 2023 •

edited

Loading

kovrus commented Mar 10, 2023

kovrus commented Mar 15, 2023

pstibrany commented Mar 20, 2023 •

edited

Loading

kovrus commented Mar 22, 2023

pstibrany left a comment

pracucci Mar 31, 2023

		* [ENHANCEMENT] OTLP: Add support for converting OTel Exponential Histograms to Prometheus Native Histograms. #4063
		* [BUGFIX] OTLP: Do not drop exemplars of the OTLP Monotonic Sum metric. #4063

[otlp] Update OTel Collector PRW translate package to 0.73.0 #4063

[otlp] Update OTel Collector PRW translate package to 0.73.0 #4063

Conversation

kovrus commented Jan 24, 2023 • edited Loading

What this PR does

Which issue(s) this PR fixes or relates to

Checklist

CLAassistant commented Jan 24, 2023 • edited Loading

gouthamve commented Jan 24, 2023

pracucci commented Jan 24, 2023

kovrus commented Feb 17, 2023 • edited Loading

pracucci left a comment

Choose a reason for hiding this comment

pracucci Feb 23, 2023

Choose a reason for hiding this comment

pstibrany commented Mar 9, 2023

pracucci commented Mar 9, 2023 • edited Loading

kovrus commented Mar 9, 2023 • edited Loading

kovrus commented Mar 10, 2023

kovrus commented Mar 15, 2023

pstibrany commented Mar 20, 2023 • edited Loading

kovrus commented Mar 22, 2023

pstibrany left a comment

Choose a reason for hiding this comment

pracucci Mar 31, 2023

Choose a reason for hiding this comment

kovrus commented Jan 24, 2023 •

edited

Loading

CLAassistant commented Jan 24, 2023 •

edited

Loading

kovrus commented Feb 17, 2023 •

edited

Loading

pracucci commented Mar 9, 2023 •

edited

Loading

kovrus commented Mar 9, 2023 •

edited

Loading

pstibrany commented Mar 20, 2023 •

edited

Loading