Make the segment pre-processor pluggable via SegmentPreProcessorProvider - #19369
Merged
Jackie-Jiang merged 1 commit intoAug 26, 2026
Merged
Conversation
Plugins can substitute a SegmentPreProcessor subclass to manage resources spanning every index handler of one preprocess run - e.g. a locally staged copy of a remote source file that several handlers read. Handlers run in unspecified order relative to each other, so no single handler can scope such a resource; a pre-processor subclass wrapping process() can. Providers are discovered through ServiceLoader on the default classloader and on every plugin classloader, with the highest-priority one winning (the IndexPlugin convention). Without a registered provider, behavior is unchanged.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #19369 +/- ##
============================================
+ Coverage 67.44% 67.46% +0.01%
Complexity 1430 1430
============================================
Files 3485 3486 +1
Lines 223874 223892 +18
Branches 35300 35304 +4
============================================
+ Hits 150988 151041 +53
+ Misses 60895 60857 -38
- Partials 11991 11994 +3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
xiangfu0
approved these changes
Aug 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a minimal SPI to let plugins substitute the
SegmentPreProcessorused at segment load:SegmentPreProcessorProvider: creates the pre-processor for one preprocess run. Implementations are discovered throughServiceLoader, enumerated on the default classloader and on every plugin classloader (PluginManager#getPluginClassLoaders), with the highest-priority provider winning — the same convention asIndexPlugin.SegmentPreProcessor.create(...): resolves the provider once and falls back to the plainSegmentPreProcessorwhen none is registered, so behavior is unchanged for deployments without a provider.ImmutableSegmentLoaderconstructs the pre-processor throughcreate(...)at its two call sites.The motivation is resources that span every index handler of one preprocess run — e.g. staging a local copy of a remote source file that several handlers read. Handlers run in unspecified order relative to each other, so no single handler can own such a resource's lifecycle; a pre-processor subclass wrapping
process()can open it beforesuper.process(...)and release it in afinally.