rptest: fix hwm computation in topic_recovery_test #16764
Conversation
We were previously validating the HWM with the assumption that non-data records were exactly 1:1 with the number of archival segments. While such an assumption isn't unreasonable, it isn't very future proof. I ran into a test failure where we the highest_producer_id record included to archival made this condition fail: ``` AssertionError: Unexpected high watermark 5025 (min expected: 5026, max expected: 5029) ``` This commit updates the check be more future-proof, simply validating how HWM is computed: hwm = rp_last_offset + 1 - delta
There was a problem hiding this comment.
assumption that non-data records were exactly 1:1 with the number of archival segments. While such an assumption isn't unreasonable
IIUC this assumption isn't reasonable, right? Like there are non-data batches that don't correspond to segment rolls is what we are talking about?
|
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/45425#018dedcf-fa14-4b81-b4a6-2854e179b2ef ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/45425#018dedcf-fa10-4966-87c4-87f8480fa7b1 |
More or less, yeah. We were placing an upper bound on the number of non-data batches we could have that was tied to the number of archival |
|
btw, did this resolve a CI failure or something? |
Yeah, I just opened #16769, which I hit in another PR but didn't see it on GH |
We were previously validating the HWM with the assumption that non-data records were exactly 1:1 with the number of archival segments. While such an assumption isn't unreasonable, it isn't very future proof.
I ran into a test failure where we the highest_producer_id record included to archival made this condition fail:
This commit updates the check be more future-proof, simply validating how HWM is computed:
hwm = rp_last_offset + 1 - delta
Fixes #16769
Backports Required
Release Notes