Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failure exporting _created metrics #24915

Closed
ikstewa opened this issue Aug 5, 2023 · 6 comments
Closed

Failure exporting _created metrics #24915

ikstewa opened this issue Aug 5, 2023 · 6 comments
Labels
bug Something isn't working exporter/prometheusremotewrite priority:needed Triagers reviewed the issue but need code owner to set priority

Comments

@ikstewa
Copy link

ikstewa commented Aug 5, 2023

Component(s)

exporter/prometheusremotewrite

What happened?

Description

When attempting to enable the export_created_metric feature introduced in #17417 the metrics are generated however they're being rejected due to missing a timestamp.

Steps to Reproduce

Enabled the export_created_metric on the prometheusremotewrite exporter and attempt to export to a Cortex instance.

Expected Result

New metric my_metric_created is visible in Cortex.

Actual Result

Metric is rejected with the following error: HTTP code 400 (timestamp too old: 0 metric: \"my_metric_created\""

Collector version

v0.82.0

Environment information

No response

OpenTelemetry Collector configuration

No response

Log output

No response

Additional context

My assumption is this is related to the Sample being created without a timestamp: https://github.com/kovrus/opentelemetry-collector-contrib/blob/main/pkg/translator/prometheusremotewrite/helper.go#L496-L516

However it's not clear to me what timestamp it should use. Current time?
@ikstewa ikstewa added bug Something isn't working needs triage New item requiring triage labels Aug 5, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Aug 5, 2023

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@bryan-aguilar bryan-aguilar added priority:needed Triagers reviewed the issue but need code owner to set priority and removed needs triage New item requiring triage labels Aug 23, 2023
@github-actions
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Oct 23, 2023
@jmichalek132
Copy link
Contributor

still relevant

@github-actions github-actions bot removed the Stale label Nov 24, 2023
@jmichalek132 jmichalek132 mentioned this issue Nov 25, 2023
Merged
@jmichalek132
Copy link
Contributor

Started to looking into fixing this, given plans within prometheus to improve counter behaviour using this metrics.
Accepted proposal within prometheus for this topic https://github.com/prometheus/proposals/blob/main/proposals/2023-06-13_created-timestamp.md.

codeboten pushed a commit that referenced this issue Dec 6, 2023
@Aneurysm9 @dashpole
chore: fix prometheus remote write exporter not setting timestamp on …
7b1ff5b 
…created metric (#29503)

As disucssed on cncf
[slack](https://cloud-native.slack.com/archives/C01LSCJBXDZ/p1700938910822949)
fixing the issue with _created metrics where prometheus backend refuses
it due to "too old" due to not setting a timestamp for the sample
created. The timestamp used is the timestamp for the original sample the
correspondent _created metrics is generated for.

**Link to tracking Issue:** #24915

Tested locally with prometheus compatible backend.

---------

Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>
Co-authored-by: David Ashpole <dashpole@google.com>
@jmichalek132
Copy link
Contributor

Should be fixed with #29503 merged.

jayasai470 pushed a commit to jayasai470/opentelemetry-collector-contrib that referenced this issue Dec 8, 2023
@Aneurysm9 @dashpole @jayasai470
chore: fix prometheus remote write exporter not setting timestamp on …
fcf373a 
…created metric (open-telemetry#29503)

As disucssed on cncf
[slack](https://cloud-native.slack.com/archives/C01LSCJBXDZ/p1700938910822949)
fixing the issue with _created metrics where prometheus backend refuses
it due to "too old" due to not setting a timestamp for the sample
created. The timestamp used is the timestamp for the original sample the
correspondent _created metrics is generated for.

**Link to tracking Issue:** open-telemetry#24915

Tested locally with prometheus compatible backend.

---------

Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>
Co-authored-by: David Ashpole <dashpole@google.com>
@jmichalek132
Copy link
Contributor

Hi, @bryan-aguilar could you please close this one. It has been fixed in the last release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working exporter/prometheusremotewrite priority:needed Triagers reviewed the issue but need code owner to set priority
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ikstewa @jmichalek132 @bryan-aguilar