Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix filtering by virtual columns with OR filter in query #55418

Merged
merged 2 commits into from Oct 14, 2023

Commits on Oct 10, 2023

  1. Fix filtering by virtual columns with OR filter in query

    The problem with the initial implementation ClickHouse#52653 was:
    - OR can have multiple arguments
    - It simply not correct to assume that if there are two arguments this is OK.
      Consider the following example:
    
        "WHERE (column_not_from_partition_by = 1) OR false OR false"
    
      Will be converted to:
    
        "WHERE false OR false"
    
    And it will simply read nothing.
    
    Yes, we could apply some optimization for bool, but this will not always
    work, since to optimize things like "0 = 1" we need to execute it.
    
    And the only way to make handle this correctly (with ability to ignore
    some commands during filtering) is to make is_constant() function return
    has it use something from the input block, so that we can be sure, that
    we have some sensible, and not just "false".
    
    Plus we cannot simply ignore the difference of the input and output
    arguments of handling OR, we need to add always-true (1/true) if the
    size is different, since otherwise it could break invariants (see
    comment in the code).
    
    This includes (but not limited to):
    - _part* filtering for MergeTree
    - _path/_file for various File/HDFS/... engines
    - _table for Merge
    - ...
    
    P.S. analyzer does not have this bug, since it execute expression as
    whole, and this is what filterBlockWithQuery() should do actually
    instead, but this will be a more complex patch.
    
    Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
    azat committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    b107712 View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2023

  1. Configuration menu
    Copy the full SHA
    769ed2e View commit details
    Browse the repository at this point in the history