Use latest LLC segment (not first) for partition assignment; skip if it is on tier#18468
Merged
Jackie-Jiang merged 4 commits intoMay 11, 2026
Conversation
…Assignment In BaseStrictRealtimeSegmentAssignment.getExistingAssignment, iterate over all entries and select the LLC segment with the highest sequence number for the target partition instead of returning the first match. Then look up the chosen segment's ZK metadata: if it has already been moved to a tier, its instance pool differs from the CONSUMING pool, so return null and fall back to the assignment derived from instance partitions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…egmentAssignment Revert getExistingAssignment in the base class to its original first-match behavior; only change is visibility (private -> protected) so subclasses can override it. In MultiTierStrictRealtimeSegmentAssignment, override getExistingAssignment to pick the LLC segment with the highest sequence number for the partition, then consult ZK metadata: if that segment has been moved to a tier it lives on a different instance pool than a new CONSUMING segment, so return null and fall back to the assignment derived from instance partitions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…nd honors tier Two new tests on the dedup (multi-tier) path: * testMultiTierAssignSegmentUsesLatestExistingAssignment seeds partition 0 with an older segment on the old CONSUMING instances and a newer segment on the new CONSUMING instances, then verifies a new CONSUMING segment lands on the new instances. The "first match" behavior would have produced the old instances, so the test catches a regression of the override. * testMultiTierAssignSegmentSkipsExistingWhenLatestOnTier marks the latest partition-0 segment as living on a tier in ZK metadata and verifies the new CONSUMING segment falls back to the assignment decided by new instance partitions instead of reusing the tiered segment's instances. A small helix-manager helper accepts a set of "tiered" segment names so we can stub SegmentZKMetadata.getTier() per segment. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Jackie-Jiang
added
bug
* Drop uploaded-segment tracking from the override — only committed LLC segments matter for picking the latest segment per partition. * Return null (and log a warning) when the latest segment's ZK metadata is unavailable, in addition to the existing tier-based skip; the prior tier case is also raised from info to warn so both branches are observable. * Test nit: replace Collections.emptySet() with Set.of() in the helix-manager factory helper. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Jackie-Jiang
approved these changes
May 11, 2026
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #18468 +/- ##
=============================================
+ Coverage 34.99% 63.64% +28.64%
- Complexity 869 1684 +815
=============================================
Files 3256 3256
Lines 199548 199575 +27
Branches 30986 30994 +8
=============================================
+ Hits 69840 127020 +57180
+ Misses 123570 62418 -61152
- Partials 6138 10137 +3999
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
BaseStrictRealtimeSegmentAssignment.getExistingAssignmentreturned the first LLCsegment it found for a partition. When multi-tier dedup tables rebalance and a partition's
CONSUMING segment moves to a new instance pool, the older segment lingering in
idealStatecan be picked first — causing the next CONSUMING segment to land oncold tier instead of hot tiers.
This PR, scoped to
MultiTierStrictRealtimeSegmentAssignment, changes the behavior to:not the first one encountered. Iteration order of
currentAssignmentislexicographic on segment name (Helix
ZNRecord.getMapFields()is aTreeMap),which diverges from numeric sequence order — so explicit comparison is required.
(
SegmentZKMetadata.getTier() != null), it lives on a different instance poolthan the CONSUMING segment, so return
nulland letassignSegmentfall backto the assignment derived from instance partitions.
The base class is unchanged in behavior — the only modification is visibility
(
private→protected) to allow the subclass override. Single-tier strictrealtime assignment is untouched.