[feature](catalog) support medium allocation mode#63011
Open
Ryan19929 wants to merge 2 commits into
Open
Conversation
Introduce MediumAllocationMode as the table and partition metadata foundation for storage medium allocation semantics. CREATE TABLE keeps the existing user syntax: explicitly specifying storage_medium maps to STRICT, while omitting it maps to ADAPTIVE. Persist the mode in DataProperty, rebuild the table-level cache in TableProperty, and migrate SystemInfoService and its callers from the old boolean flag to the enum so STRICT placement cannot silently fall back. Old images that do not contain the new field default to ADAPTIVE, which matches the previous behavior because storageMediumSpecified was not persisted. Add FE unit coverage for enum parsing, DataProperty compatibility, TableProperty cache behavior, and STRICT/ADAPTIVE backend selection, plus a regression test covering persistence across FE restart. Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
1 similar comment
Contributor
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 29442 ms |
Contributor
TPC-DS: Total hot run time: 171204 ms |
Contributor
FE UT Coverage ReportIncrement line coverage |
Contributor
FE Regression Coverage ReportIncrement line coverage |
### What problem does this PR solve? Issue Number: close #xxx Related PR: None Problem Summary: Improve coverage for the medium allocation mode metadata change by testing DataProperty's persisted mode behavior, keeping the enum API scoped to PR-1, and adding a regression case for ADAPTIVE fallback versus STRICT failure. ### Release note None ### Check List (For Author) - Test: Unit Test attempted, but local Maven dependency resolution failed for missing FE 1.2-SNAPSHOT modules; `ReadLints` and `git diff --check` passed for changed files. - Behavior changed: No - Does this need documentation: No Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 29383 ms |
Contributor
TPC-DS: Total hot run time: 170028 ms |
Contributor
FE Regression Coverage ReportIncrement line coverage |
Contributor
Author
|
run feut |
Contributor
Author
|
/review |
1 similar comment
Contributor
Author
|
/review |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #58897
Problem Summary:
This is the first PR of a series that enables RESTORE to choose the target
storage medium in a clear and consistent way.
To support that RESTORE feature, Doris first needs a stable table/partition
metadata model for storage medium allocation. This series also makes storage
medium selection consistent between CREATE TABLE and RESTORE.
The complete series will introduce:
STRICT/ADAPTIVEmedium allocation semantics.storage_mediummeansSTRICT; omitting it meansADAPTIVE.same_with_upstream, in follow-up PRs.MediumDecisionMakerfor restore scenarios in follow-up PRs.allow_medium_fallback) in follow-up PRs.This PR only builds the table-level metadata foundation. It does not change RESTORE behavior yet.
Main changes in this PR:
MediumAllocationMode(STRICT/ADAPTIVE).mediumAllocationModeinDataProperty.TableProperty.mediumAllocationModeas a rebuilt cache from table properties.storage_mediumtoSTRICT; unspecified medium remainsADAPTIVE.SystemInfoService.selectBackendIdsForReplicaCreationand cloud override to acceptMediumAllocationModeinstead of a boolean.mediumAllocationModetoADAPTIVE.This also fixes the previous
storageMediumSpecifiedproblem: the old boolean was not persisted because it had no@SerializedName.Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)