Stop saying "out of bounds"

[bboreham]

Is your feature request related to a problem? Please describe.

Users get messages like this and it means nothing to them.

time="2022-05-20T15:05:31Z" level=error msg="Request failed: 400 user=1234: err: out of bounds. timestamp=2022-05-20T13:22:47.101Z, series={__name__=\"process_max_fds\", app=\"promtail\" ..."

Describe the solution you'd like

Say something like "timestamp too old".

Describe alternatives you've considered

If we could just accept older data, maybe related to #1757, then this error would not occur.

Additional context

The error comes from inside TSDB; I guess we'd have to check in the ingester before sending it to TSDB.
Or we could try changing the message upstream in Prometheus.

[gmintoco]

This would benefit us as well. We have some edge devices that push back metrics when they can (usually only down for a few hours but potentially could be days at a time) being able to ingest this data back into Mimir when they come online would be very useful

[pracucci]

This would benefit us as well. We have some edge devices that push back metrics when they can (usually only down for a few hours but potentially could be days at a time) being able to ingest this data back into Mimir when they come online would be very useful

This issue is about rephrasing the error message, while what you need is the ability to ingest samples with any timestamp: it's something we're already working on (not tracked by this issue), but will take some time to get it production grade.

[daper]

Is there any place where I could follow the progress of this feature?

[pracucci]

Is there any place where I could follow the progress of this feature?

@codesome @jesusvazquez What's the best way to follow out-of-order samples ingestion for OSS community?

[jesusvazquez]

@pracucci @daper best way is to subscribe to this issue prometheus/prometheus#8535, that is where we're giving the updates.

https://github.com/grafana/mimir/tree/out-of-order is the active branch where we're making the changes in case you want to look deeper.

[osg-grafana]

@grafana/docs-squad, any objection to flag this phrase in the linter?

[KMiller-Grafana]

@grafana/docs-squad, any objection to flag this phrase in the linter?

No objection, except that the linter would have to be run on the code base, not on docs to improve this issue's situation.

[jesusvazquez]

In the following weeks we'll merge a PR that brings out of order support where the out of bounds errors have been replaced for sample too old. Maybe we could consider enabling the linter after that?

[pracucci]

Is this fixed by #2187 and #2066?

[codesome]

With #2187, we say "too old" only if the out of order ingestion is enabled. If there is any place where "out of bounds" is used, that still stays with #2187 with out of order disabled.

[pracucci]

We remapped the "out of bounds" error into a more understandable error in #2066, so I think we can close this issue.

Feel free to reopen it anytime if think otherwise.