Skip to content

Comments

[fix](storage) Fix core bug of convert to predicate column#8833

Merged
morningman merged 1 commit intoapache:masterfrom
zenoyang:0401_lowcard_debug
Apr 7, 2022
Merged

[fix](storage) Fix core bug of convert to predicate column#8833
morningman merged 1 commit intoapache:masterfrom
zenoyang:0401_lowcard_debug

Conversation

@zenoyang
Copy link
Member

@zenoyang zenoyang commented Apr 2, 2022

Proposed changes

Issue Number: close #8315 #8835
Fix core bug of convert to predicate column

Problem Summary:

recurrent:
When enable_low_cardinality_optimize = true, for the TPCH dataset, using the following SQL query will Core

select count(*) from lineitem where l_comment = 'ously even exc';

This SQL will trigger the execution of ColumnDictionary::convert_to_predicate_column_if_dictionary, and res->reserve(_codes.size()) is problematic because the current _codes.size() is smaller than its reserve value, so inserting a value into PredicateColumn will Core.

Checklist(Required)

  1. Does it affect the original behavior: (Yes/No/I Don't know)
  2. Has unit tests been added: (Yes/No/No Need)
  3. Has document been added or modified: (Yes/No/No Need)
  4. Does it need to update dependencies: (Yes/No)
  5. Are there any changes that cannot be rolled back: (Yes/No)

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@morningman morningman added kind/fix Categorizes issue or PR as related to a bug. dev/1.0.1-deprecated should be merged into dev-1.0.1 branch labels Apr 2, 2022
@wangbo
Copy link
Contributor

wangbo commented Apr 2, 2022

If this core dump has a stack info, you can open an issue and post stack in it. Then releated to this pr.

@zenoyang
Copy link
Member Author

zenoyang commented Apr 2, 2022

If this core dump has a stack info, you can open an issue and post stack in it. Then releated to this pr.

ok, i created an issue: #8835

Copy link
Contributor

@wangbo wangbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 6, 2022
@github-actions
Copy link
Contributor

github-actions bot commented Apr 6, 2022

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Apr 6, 2022

PR approved by anyone and no changes requested.

@wangbo wangbo added area/storage Issues or PRs related to storage engine area/vectorization labels Apr 6, 2022
@morningman morningman merged commit ca40552 into apache:master Apr 7, 2022
morningman pushed a commit that referenced this pull request Apr 8, 2022
recurrent:
When `enable_low_cardinality_optimize = true`, for the TPCH dataset, using the following SQL query will Core
```sql
select count(*) from lineitem where l_comment = 'ously even exc';
```

This SQL will trigger the execution of `ColumnDictionary::convert_to_predicate_column_if_dictionary`, and `res->reserve(_codes.size())` is problematic because the current `_codes.size()` is smaller than its reserve value, so inserting a value into `PredicateColumn` will Core.
@morningman morningman added dev/merged-1.0.1-deprecated PR has been merged into dev-1.0.1 and removed dev/1.0.1-deprecated should be merged into dev-1.0.1 branch labels Apr 8, 2022
weizhengte pushed a commit to weizhengte/incubator-doris that referenced this pull request Apr 22, 2022
recurrent:
When `enable_low_cardinality_optimize = true`, for the TPCH dataset, using the following SQL query will Core
```sql
select count(*) from lineitem where l_comment = 'ously even exc';
```

This SQL will trigger the execution of `ColumnDictionary::convert_to_predicate_column_if_dictionary`, and `res->reserve(_codes.size())` is problematic because the current `_codes.size()` is smaller than its reserve value, so inserting a value into `PredicateColumn` will Core.
zhengshiJ pushed a commit to zhengshiJ/incubator-doris that referenced this pull request Apr 27, 2022
recurrent:
When `enable_low_cardinality_optimize = true`, for the TPCH dataset, using the following SQL query will Core
```sql
select count(*) from lineitem where l_comment = 'ously even exc';
```

This SQL will trigger the execution of `ColumnDictionary::convert_to_predicate_column_if_dictionary`, and `res->reserve(_codes.size())` is problematic because the current `_codes.size()` is smaller than its reserve value, so inserting a value into `PredicateColumn` will Core.
@xunfeng1980
Copy link

1.1.3 rc2 version has this core bug

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. area/storage Issues or PRs related to storage engine area/vectorization dev/merged-1.0.1-deprecated PR has been merged into dev-1.0.1 kind/fix Categorizes issue or PR as related to a bug. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhancement] Low cardinality string optimization in storage layer

4 participants