forked from ClickHouse/ClickHouse
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fix filtering by virtual columns with OR filter in query
The problem with the initial implementation ClickHouse#52653 was: - OR can have multiple arguments - It simply not correct to assume that if there are two arguments this is OK. Consider the following example: "WHERE (column_not_from_partition_by = 1) OR false OR false" Will be converted to: "WHERE false OR false" And it will simply read nothing. Yes, we could apply some optimization for bool, but this will not always work, since to optimize things like "0 = 1" we need to execute it. And the only way to make handle this correctly (with ability to ignore some commands during filtering) is to make is_constant() function return has it use something from the input block, so that we can be sure, that we have some sensible, and not just "false". Plus we cannot simply ignore the difference of the input and output arguments of handling OR, we need to add always-true (1/true) if the size is different, since otherwise it could break invariants (see comment in the code). This includes (but not limited to): - _part* filtering for MergeTree - _path/_file for various File/HDFS/... engines - _table for Merge - ... P.S. analyzer does not have this bug, since it execute expression as whole, and this is what filterBlockWithQuery() should do actually instead, but this will be a more complex patch. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
- Loading branch information
Showing
4 changed files
with
92 additions
and
10 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
-- { echoOn } | ||
SELECT * FROM or_bug WHERE (key = 1) OR false OR false; | ||
1 | ||
SELECT * FROM or_bug WHERE (key = 1) OR false; | ||
1 | ||
SELECT * FROM or_bug WHERE (key = 1); | ||
1 | ||
-- { echoOn } | ||
select * from forms where text_field like '%this%' or 0 = 1 or 0 = 1; | ||
5840ead423829c1eab29fa97 this is a test | ||
select * from forms where text_field like '%this%' or 0 = 1; | ||
5840ead423829c1eab29fa97 this is a test | ||
select * from forms where text_field like '%this%'; | ||
5840ead423829c1eab29fa97 this is a test |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
-- https://github.com/ClickHouse/ClickHouse/pull/52653 | ||
DROP TABLE IF EXISTS or_bug; | ||
CREATE TABLE or_bug (key UInt8) ENGINE=MergeTree ORDER BY key; | ||
INSERT INTO TABLE or_bug VALUES (0), (1); | ||
|
||
-- { echoOn } | ||
SELECT * FROM or_bug WHERE (key = 1) OR false OR false; | ||
SELECT * FROM or_bug WHERE (key = 1) OR false; | ||
SELECT * FROM or_bug WHERE (key = 1); | ||
-- { echoOff } | ||
|
||
-- https://github.com/ClickHouse/ClickHouse/issues/55288 | ||
DROP TABLE IF EXISTS forms; | ||
CREATE TABLE forms | ||
( | ||
`form_id` FixedString(24), | ||
`text_field` String | ||
) | ||
ENGINE = MergeTree | ||
PRIMARY KEY form_id | ||
ORDER BY form_id; | ||
insert into forms values ('5840ead423829c1eab29fa97','this is a test'); | ||
|
||
-- { echoOn } | ||
select * from forms where text_field like '%this%' or 0 = 1 or 0 = 1; | ||
select * from forms where text_field like '%this%' or 0 = 1; | ||
select * from forms where text_field like '%this%'; | ||
-- { echoOff } |