Prom2: weird interplay of rate and recording rules on P2

[mwitkow]

What did you do?
Ran a stack of P1 and P2 side by side. Pointed it at the same service that generates a steady 80 qps on a rate counter. There's also a ::rate1m recording rule assigned to it. The data is scraped every 15s.

This screen shows the raw data calculated with a query, and the output of the recording rule

What did you expect to see?

The same graph on P2.

What did you see instead? Under which circumstances?

Instead, the P2 behaviour seems weird, as the query data seems very similar to the one from P1, but the recording rule is different.

Environment

dev-2.0 latest head.

@fabxc

[mwitkow]

@fabxc is this an artifact of the new TSDB or did any other changes to rule evaluation happen?

[mwitkow]

@fabxc do you think this may be related to #2776 ?

[fabxc]

Hard to guess. Possible for sure. Easiest way to find out is whether it disappears with the next alpha.

[brian-brazil]

This looks like aliasing, nothing to see here I think.

[gouthamve]

Hmm, setting the resolution to 1sec should probably show if its aliasing or not.

[mwitkow]

@brian-brazil can you elaborate please?

It's still there, sadly :(

[brian-brazil]

If every time you refresh the page (and thus have slightly different start/end times), then it's aliasing. This is a standard signal processing problem, and not something we can solve in Prometheus.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.