Currently MergeRollupTask runs in a forward manner, meaning segments that were processed are not processed again even after modifying the watermark in Zookeeper.
Possible use case is that if ingestion of data is not uniform, some days receive more events than others, so rollup for days where events are less, results in let's say 10 segments, but for days where events are much higher, segments are rolled up into 1000 segments.
If MergeRollupTask can be made to run again with new configuration for those affected days it would be ideal.
Druid has this really useful feature called Reindex which does this.
Currently
MergeRollupTaskruns in a forward manner, meaning segments that were processed are not processed again even after modifying the watermark in Zookeeper.Possible use case is that if ingestion of data is not uniform, some days receive more events than others, so rollup for days where events are less, results in let's say 10 segments, but for days where events are much higher, segments are rolled up into 1000 segments.
If
MergeRollupTaskcan be made to run again with new configuration for those affected days it would be ideal.Druid has this really useful feature called Reindex which does this.