Restart time increase gradually for each restart on Prometheus 2.0

[tejaswiniVadlamudi]

When I try to check the restart time of Prometheus, it increases gradually each time, I reboot.

For 5 iterations: I got the below values: 7, 21, 35, 57, 97

Process followed:

1. Login to Prometheus container, run:
   date; reboot
2. Check if Prometheus container is UP and check uptime
3. Measure restart time as difference between uptime and time you get on running in bullet 1

Running Prometheus 2.0 on Kubernetes 1.6.10

[brian-brazil]

Over what time period is this? The WAL is expected to require more work over time, but that should reset every 2h.

Does this also happen with 2.1.0?

[brian-brazil]

Can you also indicate if there's any indications of data corruption? I have another report of this which went away when a corrupted block was removed.

[tejaswiniVadlamudi]

Prometheus is not corrupted, I haven't verified this with 2.1 yet.
I can check and confirm.

[brian-brazil]

This was probably the 4GB index issue preventing compaction, if this is still happening with 2.2.1 please reopen.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.