branch-4.0: [fix](be) Handle legacy DecimalV2 segments with missing precision/frac (#63569)#63581
Open
jacktengg wants to merge 1 commit into
Open
branch-4.0: [fix](be) Handle legacy DecimalV2 segments with missing precision/frac (#63569)#63581jacktengg wants to merge 1 commit into
jacktengg wants to merge 1 commit into
Conversation
Contributor
Author
|
run buildall |
cacc79a to
f6df3b8
Compare
Contributor
Author
|
run buildall |
…recision/frac (apache#63569) Pick: apache#63569 Problem Summary: After upgrading from 3.1.4 to 4.0.4, queries on DecimalV2 columns fail with: [INTERNAL_ERROR]meet invalid precision: real_precision=0, max_decimal_precision=27, min_decimal_precision=1 Root cause: Segments written by Doris < 2.1.0 (before apache#26572) do not persist precision/frac in ColumnMetaPB; they default to 0 when read back. Since apache#35222, DataTypeFactory passes those raw values as original_precision/original_scale into DataTypeDecimalV2, which calls check_type_precision(0) and throws. 3.1.4 was unaffected because the old code hardcoded (27, 9). Fix: In _create_primitive_data_type for OLAP_FIELD_TYPE_DECIMAL, when precision is not positive (legacy segment), pass UINT32_MAX to DataTypeDecimalV2 to signal that the original precision/scale are unknown. DecimalScaleInfo<TYPE_DECIMALV2> already treats UINT32_MAX as 'unknown' and falls back to the in-memory (27, 9) representation in get_original_precision()/get_original_scale(). Fix "meet invalid precision: real_precision=0" when querying DecimalV2 columns on segments written by Doris versions older than 2.1.0. - Test: Unit Test - Added DataTypeDecimalTest.create_decimalv2_from_legacy_tablet_column covering: legacy TabletColumn (unset precision/frac), modern TabletColumn with decimal(26,6), and legacy segment_v2::ColumnMetaPB with default-0 precision/frac. - Behavior changed: No - Does this need documentation: No Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
f6df3b8 to
ebd4720
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pick: #63569
Problem Summary: After upgrading from 3.1.4 to 4.0.4, queries on DecimalV2 columns fail with:
Root cause: Segments written by Doris < 2.1.0 (before #26572) do not persist precision/frac in ColumnMetaPB; they default to 0 when read back. Since #35222, DataTypeFactory passes those raw values as original_precision/original_scale into DataTypeDecimalV2, which calls check_type_precision(0) and throws. 3.1.4 was unaffected because the old code hardcoded (27, 9).
Fix: In _create_primitive_data_type for OLAP_FIELD_TYPE_DECIMAL, when precision is not positive (legacy segment), pass UINT32_MAX to DataTypeDecimalV2 to signal that the original precision/scale are unknown. DecimalScaleInfo<TYPE_DECIMALV2> already treats UINT32_MAX as 'unknown' and falls back to the in-memory (27, 9) representation in get_original_precision()/get_original_scale().
Fix "meet invalid precision: real_precision=0" when querying DecimalV2 columns on segments written by Doris versions older than 2.1.0.
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)