-
Notifications
You must be signed in to change notification settings - Fork 6.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix actions execution during preliminary filtering (PK, partition pruning) #60196
Conversation
This is an automated comment for commit 0cb6f20 with description of existing statuses. It's updated for the latest CI running ❌ Click here to open a full report in a separate page Successful checks
|
Please check the case where input columns repeat three and more times, e.g.: select * from data final prewhere indexHint(_partition_id = 'all') or indexHint(_partition_id = 'all') or indexHint(_partition_id = 'all'); I'm getting
|
This information will be added to the exception anyway. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
…ning) Previously preliminary filtering works incorrectly if input columns had been used multiple times, because: - position initialized only for the first user - column had been moved, so the second user will get nullptr And this was the case when it had been required multiple times. Consider the following example: select * from data final prewhere indexHint(_partition_id = 'all') or indexHint(_partition_id = 'all') Actions for this PREWHERE is the following: (lldb) p actions.__ptr_->dumpActions() (std::string) $1 = "input: _partition_id LowCardinality(String) _partition_id LowCardinality(String) actions: INPUT : 0 -> _partition_id LowCardinality(String) : 0 COLUMN Const(String) -> 'all' String : 1 COLUMN Const(String) -> 'UInt8'_String String : 2 INPUT : 1 -> _partition_id LowCardinality(String) : 3 COLUMN Const(String) -> 'all' String : 4 COLUMN Const(String) -> 'UInt8'_String String : 5 FUNCTION equals(_partition_id :: 0, 'all' :: 1) -> equals(_partition_id, 'all') LowCardinality(UInt8) : 6 FUNCTION equals(_partition_id :: 3, 'all' :: 4) -> equals(_partition_id, 'all') LowCardinality(UInt8) : 1 FUNCTION _CAST(equals(_partition_id, 'all') :: 6, 'UInt8'_String :: 2) -> _CAST(equals(_partition_id, 'all'), 'UInt8'_String) UInt8 : 4 FUNCTION _CAST(equals(_partition_id, 'all') :: 1, 'UInt8'_String :: 5) -> _CAST(equals(_partition_id, 'all'), 'UInt8'_String) UInt8 : 2 FUNCTION or(_CAST(equals(_partition_id, 'all'), 'UInt8'_String) :: 4, _CAST(equals(_partition_id, 'all'), 'UInt8'_String) :: 2) -> or(indexHint(), indexHint())"... It has _partition_id column in input multiple times. So fix this by adding a flag (`allow_duplicates_in_input`), if set, columns will be fully initialized. v2: fix "Position out of bound in Block::erase()" Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
0bbd87c
to
0cb6f20
Compare
|
|
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Fix actions execution during preliminary filtering (PK, partition pruning)
Previously preliminary filtering works incorrectly if input columns had
been used multiple times, because:
And this was the case when it had been required multiple times.
Consider the following example:
Actions for this PREWHERE is the following:
It has _partition_id column in input multiple times.
So fix this by adding a flag (
allow_duplicates_in_input
), if set,columns will be fully initialized.