[Fuzzer] Subquery error

[Maxxen]

Originally discovered in #4152

SELECT (VALUES(1 != ALL(SELECT 2)));

Actual:
src/common/types/vector.cpp:94: void duckdb::Vector::Reference(duckdb::Vector &): Assertion `other.GetType() == GetType()' failed.

Expected:
The query should output a single true value.

[Maxxen]

@taniabogatsch I think Thijs forwarded this to you after we got stuck on it, so I've assigned you to it for the time being.

[taniabogatsch]

This triggers an assertion in debug mode, namely

src/common/types/vector.cpp:94: void duckdb::Vector::Reference(duckdb::Vector &): Assertion other.GetType() == GetType()' failed.

GetType() is BOOLEAN, other.GetType() is INTEGER.

The query runs fine in release (ofc because no assertions) with the following output.

D SELECT (VALUES(1 != ALL(SELECT 2)));
┌──────────────────────────────────────────────────────────────────┐
│ (SELECT * FROM (VALUES ((NOT 1 = ANY(SELECT 2)))) AS valueslist) │
├──────────────────────────────────────────────────────────────────┤
│ true                                                             │
└──────────────────────────────────────────────────────────────────┘

If we change the query to the rewritten column name, then it runs both in debug and in release.

D (SELECT * FROM (VALUES ((NOT 1 = ANY(SELECT 2)))) AS valueslist);
┌──────┐
│ col0 │
├──────┤
│ true │
└──────┘

From my understanding this implies that the bug happens somewhere during the construction of the plan? The plans (before the optimization step) are different. If we set a breakpoint in client_context.cpp:324 and print p plan->Print(), then we get these.

Failing query.

┌───────────────────────────┐                             
│         PROJECTION        │                             
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │                             
│          SUBQUERY         │                             
└─────────────┬─────────────┘                                                          
┌─────────────┴─────────────┐                             
│         AGGREGATE         │                             
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │                             
│       first(#[8.0])       │                             
└─────────────┬─────────────┘                                                          
┌─────────────┴─────────────┐                             
│           LIMIT           │                             
└─────────────┬─────────────┘                                                          
┌─────────────┴─────────────┐                             
│         PROJECTION        │                             
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │                             
│            col0           │                             
└─────────────┬─────────────┘                                                          
┌─────────────┴─────────────┐                             
│       EXPRESSION_GET      │                             
└─────────────┬─────────────┘                                                          
┌─────────────┴─────────────┐                             
│      COMPARISON_JOIN      │                             
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │                             
│            MARK           ├──────────────┐              
│      (1) = (#[15.0])      │              │              
└─────────────┬─────────────┘              │                                           
┌─────────────┴─────────────┐┌─────────────┴─────────────┐
│         DUMMY_SCAN        ││         PROJECTION        │
│                           ││   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│                           ││             2             │
└───────────────────────────┘└─────────────┬─────────────┘                             
                             ┌─────────────┴─────────────┐
                             │         DUMMY_SCAN        │
                             └───────────────────────────┘    

Running query.

┌───────────────────────────┐                             
│         PROJECTION        │                             
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │                             
│            col0           │                             
└─────────────┬─────────────┘                                                          
┌─────────────┴─────────────┐                             
│         PROJECTION        │                             
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │                             
│            col0           │                             
└─────────────┬─────────────┘                                                          
┌─────────────┴─────────────┐                             
│       EXPRESSION_GET      │                             
└─────────────┬─────────────┘                                                          
┌─────────────┴─────────────┐                             
│      COMPARISON_JOIN      │                             
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │                             
│            MARK           ├──────────────┐              
│      (1) = (#[14.0])      │              │              
└─────────────┬─────────────┘              │                                           
┌─────────────┴─────────────┐┌─────────────┴─────────────┐
│         DUMMY_SCAN        ││         PROJECTION        │
│                           ││   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│                           ││             2             │
└───────────────────────────┘└─────────────┬─────────────┘                             
                             ┌─────────────┴─────────────┐
                             │         DUMMY_SCAN        │
                             └───────────────────────────┘

I feel like my knowledge of the Planner is a bit too limited to debug this efficiently. Is this maybe related to the aggregate + limit that we push because of the subquery? I can continue on this, but maybe @Mytherin you have an idea where this originates exactly?

@Tishj also did some initial debugging of this and I am just gonna put the findings here.

The PhysicalExpressionGet has an OperatorExpression of type OPERATOR_NOT
which has a BoundReferenceExpression as child, bound to index 0. The assertion gets triggered because it's trying to reference an INTEGER vector to a BOOLEAN vector.

This expression refers to a data chunk that comes from PhysicalComparisonJoin, which is of JoinType::MARK,
the mark join adds an extra vector of type BOOLEAN.