Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -180,13 +180,18 @@ public List<SegmentZKMetadata> getNonConsumingSegmentsZKMetadataForRealtimeTable
List<SegmentZKMetadata> selectedSegmentZKMetadataList = new ArrayList<>();
for (SegmentZKMetadata segmentZKMetadata : segmentZKMetadataList) {
String segmentName = segmentZKMetadata.getSegmentName();
Map<String, String> instanceStateMap = idealState.getInstanceStateMap(segmentName);
if (idealStateSegments.contains(segmentName)
&& segmentZKMetadata.getStatus().isCompleted() // skip consuming segments
&& !idealState.getInstanceStateMap(segmentName).containsValue(SegmentStateModel.CONSUMING)) {
&& instanceStateMap != null

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need this null check since we already checked idealStateSegments.contains(segmentName). No need to handle corrupted ideal state

&& !instanceStateMap.containsValue(SegmentStateModel.CONSUMING)) {
// The last check is for an edge case where
// 1. SegmentZKMetadata was updated to DONE in segment commit protocol, but
// 2. IdealState for the segment was not updated to ONLINE due to some issue in the controller.
// We avoid picking up such segments to allow RealtimeSegmentValidationManager to fix them.
// Note: instanceStateMap == null also means the segment is in a broken state (partition exists in
// IdealState but has no instance assignments). We skip it as well to let RealtimeSegmentValidationManager
// repair it first.
selectedSegmentZKMetadataList.add(segmentZKMetadata);
}
}
Expand Down