Scope overwriteTableConfigForTier Table config Jackson to sub-configs that can hold tier overrides#18563
Merged
Jackie-Jiang merged 3 commits intoMay 25, 2026
Conversation
deepthi912
added
performance
deepthi912
changed the title
deepthi912
force-pushed
the
deepthi/scope-tier-overwrite-roundtrip
branch
from
335643c to
4826430
Compare
`TableConfigUtils.overwriteTableConfigForTier` previously did a full Jackson roundtrip over the entire TableConfig — `toJsonNode()` followed by `jsonNodeToObject(_, TableConfig.class)` — even though tier overrides can only affect `IndexingConfig` (via its `tierOverwrites` JsonNode) and individual `FieldConfig` entries (via theirs). Every other field (validationConfig, tenantConfig, ingestionConfig, taskConfig, etc.) was serialized and deserialized for no reason. In hot paths (`SegmentPreProcessor.needProcess` → `IndexLoadingConfig .refreshIndexConfigs`) this function is invoked once per segment per IndexType during `checkSegmentsReload`, and the full-TableConfig roundtrip shows up as a large CPU + allocation hotspot in profiles, driving long G1 evacuation failures and Old GC pauses on servers with many tiered segments. This change refactors the function to: - Apply the override scoped to `IndexingConfig` (when its `tierOverwrites` has an entry for the tier). - Apply the override scoped to each `FieldConfig` whose `tierOverwrites` has an entry for the tier — `FieldConfig`s without overrides skip Jackson entirely. - Return the original `TableConfig` by reference when nothing changed (common case for tiers with no overrides). - Return a shallow-copy `TableConfig` (via the existing copy constructor) with only `IndexingConfig` and `FieldConfigList` replaced; all other fields are shared by reference with the original. The per-call cost drops from "serialize entire TableConfig" to "serialize IndexingConfig + serialize each FieldConfig that has an override" — typically a 20-100× reduction in serialized bytes and proportional CPU/allocation savings. Existing semantics are preserved: `testOverwriteTableConfigForTier` and `testOverwriteTableConfigForTierWithError` pass unchanged. Two new tests assert the optimization-relevant invariants: (1) the original TableConfig instance is returned by reference when no override applies, and (2) unaffected fields are shared by reference with the original after a partial overwrite.
deepthi912
force-pushed
the
deepthi/scope-tier-overwrite-roundtrip
branch
from
4826430 to
7920d75
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #18563 +/- ##
============================================
+ Coverage 64.26% 64.27% +0.01%
- Complexity 1126 1137 +11
============================================
Files 3311 3314 +3
Lines 203829 204075 +246
Branches 31722 31765 +43
============================================
+ Hits 130989 131177 +188
- Misses 62326 62362 +36
- Partials 10514 10536 +22
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Jackie-Jiang
approved these changes
May 22, 2026
…e CollectionUtils.isEmpty Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
overwriteTableConfigForTierruns on every segment during checkSegmentsReload polls. The current code does a full Jackson roundtrip on the entire TableConfig — even though tier overrides can only live onIndexingConfig._tierOverwritesandFieldConfig._tierOverwrites.Tables with no tierOverwrites defined anywhere would still pay the full serialize cost before, because the existing code serializes upfront and only then checks if there's anything to override.
overwriteTableConfigForTierallocated ~4.5 GB (319 calls × ~14 MB each). Largest single allocator on the reload-check path. This multiplies for huge number of segments and allocates unnecessary memory when not neededFix:
Only roundtrip the pieces that can have overrides:
IndexingConfig.getTierOverwrites()has an entry for this tier → roundtrip just IndexingConfig. Otherwise return the original reference.