Reject offsets that are older than the group expiration time #330
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, Burrow will accept any offset for storage. Especially in the case of Zookeeper offsets, it's possible to have stale data (topics that are no longer consumed, or old offsets that have been moved over to Kafka) that will get pulled in and cause false alerts.
This PR adds a check into the storage module to reject offsets if they are older than the expire-group configuration. The reasoning is that if the offset is old enough that we would expire a group with just that offset, we shouldn't accept it at all. This should greatly reduce false alerts due to stale Zookeeper data, and it shouldn't impact current offsets at all.
This also updates the tests to support this and properly test min-distance and offset expiration.
Closes #320