Skip to content

Optimise getPartitionGroupConsumptionStatusList in RVM#17712

Open
noob-se7en wants to merge 3 commits intoapache:masterfrom
noob-se7en:optimise_segment_read
Open

Optimise getPartitionGroupConsumptionStatusList in RVM#17712
noob-se7en wants to merge 3 commits intoapache:masterfrom
noob-se7en:optimise_segment_read

Conversation

@noob-se7en
Copy link
Contributor

@noob-se7en noob-se7en commented Feb 16, 2026

  • Optimizes fetchPartitionGroupIdToSmallestOffset in PinotLLCRealtimeSegmentManager to reuse the already-computed latestSegmentZKMetadataMap instead of calling getPartitionGroupConsumptionStatusList,
    which performs a redundant O(N) scan over all IdealState segments and N additional ZK reads.
  • getPartitionGroupConsumptionStatusList redundant calls was also repeated per stream config.

@codecov-commenter
Copy link

codecov-commenter commented Feb 16, 2026

Codecov Report

❌ Patch coverage is 50.00000% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.20%. Comparing base (4ef8e3e) to head (bdda030).
⚠️ Report is 27 commits behind head on master.

Files with missing lines Patch % Lines
.../core/realtime/PinotLLCRealtimeSegmentManager.java 50.00% 18 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #17712      +/-   ##
============================================
- Coverage     63.25%   63.20%   -0.05%     
- Complexity     1499     1501       +2     
============================================
  Files          3174     3179       +5     
  Lines        190323   190745     +422     
  Branches      29080    29159      +79     
============================================
+ Hits         120381   120563     +182     
- Misses        60606    60804     +198     
- Partials       9336     9378      +42     
Flag Coverage Δ
custom-integration1 100.00% <ø> (ø)
integration 100.00% <ø> (ø)
integration1 100.00% <ø> (ø)
integration2 0.00% <ø> (ø)
java-11 63.16% <50.00%> (-0.05%) ⬇️
java-21 63.18% <50.00%> (-0.04%) ⬇️
temurin 63.20% <50.00%> (-0.05%) ⬇️
unittests 63.20% <50.00%> (-0.05%) ⬇️
unittests1 55.62% <ø> (-0.01%) ⬇️
unittests2 34.05% <50.00%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants