Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Support complex type in column with row #41476

Merged
merged 1 commit into from Feb 28, 2024

Conversation

decster
Copy link
Contributor

@decster decster commented Feb 23, 2024

Why I'm doing:

Currently, column with row storage type does not support complex types like bitmap, array, struct, bitmap

What I'm doing:

Support those types

Fixes #38174 #38090

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 3.2
    • 3.1
    • 3.0
    • 2.5

@decster decster requested review from a team as code owners February 23, 2024 01:59
@github-actions github-actions bot added the 3.2 label Feb 23, 2024
template <typename T>
void ObjectColumn<T>::deserialize_and_append_batch(Buffer<Slice>& srcs, size_t chunk_size) {
DCHECK(false) << "Don't support object column deserialize and append";
}

template <typename T>
uint32_t ObjectColumn<T>::serialize_size(size_t idx) const {
DCHECK(false) << "Don't support object column byte size";
return 0;
return static_cast<uint32_t>(get_object(idx)->serialize_size());
}

template <typename T>
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The most risky bug in this code is:
Assuming ObjectColumn does not support certain operations originally, changing implementations to "support" them without proper checks or understanding of the overall system design might introduce serious logical errors or mismatches with expected functionality.

You can modify the code like this:
Restore the original behavior that explicitly mentions the lack of support for certain functions, ensuring that any attempts to use these functionalities knowingly fail, prompting a review of usage patterns and possibly a more suitable design adjustment.

template <typename T>
size_t ObjectColumn<T>::byte_size(size_t idx) const {
    DCHECK(false) << "Don't support object column byte size";
    return 0;
}

template <typename T>
uint32_t ObjectColumn<T>::serialize(size_t idx, uint8_t* pos) {
    DCHECK(false) << "Don't support object column serialize";
    return 0;
}

template <typename T>
void ObjectColumn<T>::serialize_batch(uint8_t* dst, Buffer<uint32_t>& slice_sizes, size_t chunk_size, uint32_t max_one_row_size) {
    DCHECK(false) << "Don't support object column serialize batch";
}

template <typename T>
void ObjectColumn<T>::deserialize_and_append_batch(Buffer<Slice>& srcs, size_t chunk_size) {
    DCHECK(false) << "Don't support object column deserialize and append";
}

template <typename T>
uint32_t ObjectColumn<T>::serialize_size(size_t idx) const {
    DCHECK(false) << "Don't support object column byte size";
    return 0;
}

This change effectively reverts modifications that assumed the introduction of operational implementations where none were intended. The principle here is to maintain explicit indication of non-support for specific methods within ObjectColumn, which could prevent misuse or misunderstanding about what the class supports. Additional design considerations and safety mechanisms should be applied if and when these functionalities are to be officially supported, including thorough validations and error handling as appropriate.

@decster decster force-pushed the cwr-object-type branch 3 times, most recently from 8b047df to 31b7971 Compare February 26, 2024 15:04
Signed-off-by: Binglin Chang <decstery@gmail.com>
Copy link

sonarcloud bot commented Feb 27, 2024

Copy link

[FE Incremental Coverage Report]

pass : 0 / 0 (0%)

Copy link

[BE Incremental Coverage Report]

pass : 27 / 31 (87.10%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 be/src/storage/row_store_encoder_simple.cpp 16 19 84.21% [160, 166, 172]
🔵 be/src/column/object_column.cpp 11 12 91.67% [205]

@decster decster merged commit 655a124 into StarRocks:main Feb 28, 2024
53 of 54 checks passed
Copy link

@Mergifyio backport branch-3.2

@github-actions github-actions bot removed the 3.2 label Feb 28, 2024
Copy link
Contributor

mergify bot commented Feb 28, 2024

backport branch-3.2

✅ Backports have been created

mergify bot pushed a commit that referenced this pull request Feb 28, 2024
Signed-off-by: Binglin Chang <decstery@gmail.com>
(cherry picked from commit 655a124)
wanpengfei-git pushed a commit that referenced this pull request Feb 28, 2024
… (#41855)

Co-authored-by: Binglin Chang <decstery@gmail.com>
Seaven pushed a commit to Seaven/starrocks that referenced this pull request Feb 28, 2024
Signed-off-by: Binglin Chang <decstery@gmail.com>
Signed-off-by: Seaven <seaven_7@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[column_with_row] pk table with bigmap column stream load failed: row encode type not supported: OBJECT
3 participants