Logical tables / Table Config validation refactoring#17761
Open
krishan1390 wants to merge 24 commits intoapache:masterfrom
Open
Logical tables / Table Config validation refactoring#17761krishan1390 wants to merge 24 commits intoapache:masterfrom
krishan1390 wants to merge 24 commits intoapache:masterfrom
Conversation
Move fromZNRecord/toZNRecord methods from TableConfigSerDeUtils and LogicalTableConfigUtils into the config classes themselves. This makes it easier to extend these classes — subclasses can override serialization/deserialization behavior directly. - Add helix-core dependency to pinot-spi for ZNRecord access - Add fromZNRecord()/toZNRecord() to TableConfig and LogicalTableConfig - Delete TableConfigSerDeUtils (all callers updated) - Remove fromZNRecord/toZNRecord from LogicalTableConfigUtils (validation stays) - Update all callers across 15 files to use the new methods directly Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move fromZNRecord/toZNRecord methods from TableConfigSerDeUtils and LogicalTableConfigUtils into the config classes themselves. This makes it easier to extend these classes — subclasses can override serialization/deserialization behavior directly. - Add helix-core dependency to pinot-spi for ZNRecord access - Add fromZNRecord()/toZNRecord() to TableConfig and LogicalTableConfig - Delete TableConfigSerDeUtils (all callers updated) - Remove fromZNRecord/toZNRecord from LogicalTableConfigUtils (validation stays) - Update all callers across 15 files to use the new methods directly - Add LogicalTableConfigSerDeTest for ZNRecord round-trip coverage Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace ZNRecord-based SerDe on TableConfig and LogicalTableConfig with a helix-agnostic ConfigRecord POJO in pinot-spi. Bridge utilities in pinot-common (TableConfigSerDeUtils, LogicalTableConfigUtils) handle ZNRecord conversion, keeping helix-core out of the SPI layer. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #17761 +/- ##
============================================
- Coverage 63.21% 63.20% -0.01%
Complexity 1454 1454
============================================
Files 3179 3179
Lines 191069 191089 +20
Branches 29220 29223 +3
============================================
- Hits 120786 120785 -1
- Misses 60860 60885 +25
+ Partials 9423 9419 -4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
abhishekbafna
approved these changes
Feb 25, 2026
| PhysicalTableConfig physicalTableConfig = entry.getValue(); | ||
| String physicalTableName = entry.getKey(); | ||
| // Skip existence validation for multi-cluster physical tables | ||
| if (!physicalTableConfig.isMultiCluster()) { |
Collaborator
There was a problem hiding this comment.
nit: We can also add a INFO log for "Physical table {} is multi cluster, skipping existence check" for more info.
| return cloned; | ||
| } | ||
|
|
||
| public static Schema cloneSchemaWithName(Schema source, String newName) { |
Collaborator
There was a problem hiding this comment.
Good addition! Necessary for reducing the overhead while creating logical tables. But I do not see this being used anywhere currently, is this for a future addition?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refactoring of logical table / table config validations.