Skip to content

Added SegmentGeneratorConfig as an optional param to the SegmentPurger#14872

Merged
Jackie-Jiang merged 1 commit intoapache:masterfrom
rajagopr:add-segment-gen-config
Jan 28, 2025
Merged

Added SegmentGeneratorConfig as an optional param to the SegmentPurger#14872
Jackie-Jiang merged 1 commit intoapache:masterfrom
rajagopr:add-segment-gen-config

Conversation

@rajagopr
Copy link
Contributor

@rajagopr rajagopr commented Jan 22, 2025

Description

Allow passing SegmentGeneratorCustomConfig as an optional param to the SegmentPurger class. The SegmentGeneratorConfig can be customized using the configs supplied via the SegmentGeneratorCustomConfig class. This is required to add functionality that swaps segments atomically post the purge operation using the segment replacement protocol.

@codecov-commenter
Copy link

codecov-commenter commented Jan 22, 2025

Codecov Report

Attention: Patch coverage is 57.14286% with 9 lines in your changes missing coverage. Please review.

Project coverage is 63.73%. Comparing base (59551e4) to head (e839ce0).
Report is 1639 commits behind head on master.

Files with missing lines Patch % Lines
...va/org/apache/pinot/core/minion/SegmentPurger.java 70.58% 4 Missing and 1 partial ⚠️
...ent/spi/creator/SegmentGeneratorCustomConfigs.java 0.00% 4 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #14872      +/-   ##
============================================
+ Coverage     61.75%   63.73%   +1.98%     
- Complexity      207     1470    +1263     
============================================
  Files          2436     2713     +277     
  Lines        133233   151717   +18484     
  Branches      20636    23417    +2781     
============================================
+ Hits          82274    96697   +14423     
- Misses        44911    47767    +2856     
- Partials       6048     7253    +1205     
Flag Coverage Δ
custom-integration1 100.00% <ø> (+99.99%) ⬆️
integration 100.00% <ø> (+99.99%) ⬆️
integration1 100.00% <ø> (+99.99%) ⬆️
integration2 0.00% <ø> (ø)
java-11 63.70% <57.14%> (+1.99%) ⬆️
java-21 63.61% <57.14%> (+1.98%) ⬆️
skip-bytebuffers-false 63.72% <57.14%> (+1.98%) ⬆️
skip-bytebuffers-true 63.58% <57.14%> (+35.86%) ⬆️
temurin 63.73% <57.14%> (+1.98%) ⬆️
unittests 63.73% <57.14%> (+1.98%) ⬆️
unittests1 56.29% <57.14%> (+9.40%) ⬆️
unittests2 34.04% <0.00%> (+6.30%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@rajagopr rajagopr force-pushed the add-segment-gen-config branch 3 times, most recently from 1827152 to 6c6e81a Compare January 28, 2025 20:28
@rajagopr rajagopr force-pushed the add-segment-gen-config branch from 6c6e81a to e839ce0 Compare January 28, 2025 20:34
@Jackie-Jiang Jackie-Jiang added ingestion Related to data ingestion pipeline minion Related to Pinot Minion task framework labels Jan 28, 2025
@Jackie-Jiang Jackie-Jiang merged commit 8f003bd into apache:master Jan 28, 2025
21 checks passed
@rajagopr rajagopr deleted the add-segment-gen-config branch January 28, 2025 21:31
zeronerdzerogeekzerocool pushed a commit to zeronerdzerogeekzerocool/pinot that referenced this pull request Feb 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ingestion Related to data ingestion pipeline minion Related to Pinot Minion task framework

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants