This repository has been archived by the owner on Aug 23, 2023. It is now read-only.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We have noticed for a while that the mt-backfill tool never seems to flush all of the data it has. It turns out, this is due to a mix of using system time and chunk spans.
AggMetric.lastWrite
uses the system time when the data point was ingested, not the timestamp of the datapoint. This is probably the right thing to do to avoid prematurely flushing when backfilling.However,
Aggregator
won't GC a chunk until lastWriteTime+agg.span <= chunkMinTs. For rollups with longer spans (say 4h) that means the backfill tool would need to run for 4 hours before it would flush the tail datapoints.This change is to add a simple
ForceGC
function so the backfill tool can do flush everything on shutdown. I figured this was the safest way to fix this without changing the behavior of the core components.