Skip to content

PR: Improve push down filter of join #13184 introduces correctness issues #13211

@eejbyfeldt

Description

@eejbyfeldt

Describe the bug

Recently merged PR "Improve push down filter of join #13184" introduces correctness issues.

I did not look closely at the code, but the example in the PR description seems incorrect. See reproducer for why.

To Reproduce

> CREATE TABLE t1 (c0 CHAR, c1 INT) AS VALUES ('a', 1);
CREATE TABLE t2 (c0 CHAR, c1 INT);
SELECT * FROM t1 LEFT JOIN t2 ON t1.c0 = t2.c0 WHERE abs(t2.c1) > 5;

0 row(s) fetched. 
Elapsed 0.011 seconds.

0 row(s) fetched. 
Elapsed 0.006 seconds.

+----+----+----+----+
| c0 | c1 | c0 | c1 |
+----+----+----+----+
| a  | 1  |    |    |
+----+----+----+----+

Expected behavior

Before that commit Datafusion correct produced empty output

DataFusion CLI v42.2.0
> CREATE TABLE t1 (c0 CHAR, c1 INT) AS VALUES ('a', 1);
CREATE TABLE t2 (c0 CHAR, c1 INT);
SELECT * FROM t1 LEFT JOIN t2 ON t1.c0 = t2.c0 WHERE abs(t2.c1) > 5;

0 row(s) fetched. 
Elapsed 0.015 seconds.

0 row(s) fetched. 
Elapsed 0.008 seconds.

+----+----+----+----+
| c0 | c1 | c0 | c1 |
+----+----+----+----+
+----+----+----+----+
0 row(s) fetched. 
Elapsed 0.019 seconds.

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions