fix(metadata): roll deleted partitions and topics out of parent stats - #4046
Conversation
|
please add integration test (either enhance current one if it exists or create new one) |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #4046 +/- ##
============================================
- Coverage 85.98% 85.02% -0.96%
Complexity 1431 1431
============================================
Files 1247 1247
Lines 192967 188086 -4881
Branches 158323 153443 -4880
============================================
- Hits 165915 159920 -5995
- Misses 22963 23857 +894
- Partials 4089 4309 +220
🚀 New features to boost your workflow:
|
hubcio
left a comment
There was a problem hiding this comment.
one finding sits outside the diff, so it goes here.
core/shard/src/lib.rs:8180 - fence_partition_for_rebuild tombstones and enqueues ConfirmRemove without settling the partition counters. metadata never deleted the partition, so the registry entry survives and the rebuild's get-or-create hands back the fenced incarnation's numbers, still sitting in the parent totals. build_partition_fresh only zeroes on its error path, so journal repair then counts the same data twice.
the second exit at :8160-8178, where quarantine fails, never enqueues ConfirmRemove at all, so a settle in that arm would not reach it. both arms need one.
this path predates the pr, but the invariant it breaks is the one this pr establishes.
|
/ready |
23bef90 to
38a74a5
Compare
hubcio
left a comment
There was a problem hiding this comment.
retain_from_snapshot (stream.rs:722) evicts a partition entry without zeroing its stats, so the still-mounted stale incarnation's later ConfirmRemove takes those bytes a second time out of live siblings. No clamp fires, so nothing surfaces it. Reachable on a donor delete-then-recreate installed ahead of the receiver's frontier. Breaks the invariant at stream.rs:738, and created_revision has no test.
On unchanged lines, so no inline comment:
boot/recovery.rs:184points at the zeroing loop this change deleted fromboot/mod.rs. Retarget atStreams::from_snapshot.streaming_stats.rs:313-326and:469-482:zero_out_*cascades unconditionally, so the new settle runs an amount-0 rollback per deleted partition.if prev == 0 { return; }after each swap.- Nine public
iggy_commonreturn types change from(). A major break with no semver gate - worth a release note.
|
/ready |
Update the
Statsfrom metadata layer when partitions get removed.