Skip to content

Commit

Permalink
opt: fix min cardinality calculation of EXCEPT
Browse files Browse the repository at this point in the history
Prior to this commit, we assumed that the minimum cardinality of
EXCEPT could never be lower than the minimum cardinality of the
left side minus the maximum cardinality of the right side.
However, this assumption is only true for EXCEPT ALL, not EXCEPT.

For example, VALUES (1), (1) EXCEPT VALUES (1) produces 0 rows,
even though the left side produces more rows than the right.
Because of this invalid assumption, the optimizer could make some
invalid transformations, resulting in incorrect results.

Fixes #89101

Release note (bug fix): Fixed a bug that has existed since v2.1.0
where queries containing a subquery with EXCEPT could produce
incorrect results. This could happen if the optimizer could guarantee
that the left side of the EXCEPT always returned more rows than the
right side. In this case, the optimizer made a faulty assupmtion that
the EXCEPT subquery always returned at least one row, which could cause
the optimizer to perform an invalid transformation, possibly causing
the full query to return incorrect results.
  • Loading branch information
rytaft committed Sep 30, 2022
1 parent 77e7138 commit d5c2318
Show file tree
Hide file tree
Showing 2 changed files with 60 additions and 1 deletion.
7 changes: 6 additions & 1 deletion pkg/sql/opt/memo/logical_props_builder.go
Original file line number Diff line number Diff line change
Expand Up @@ -1854,9 +1854,14 @@ func (b *logicalPropsBuilder) makeSetCardinality(
card = props.Cardinality{Min: 0, Max: left.Max}
card = card.Limit(right.Max)

case opt.ExceptOp, opt.ExceptAllOp:
case opt.ExceptOp:
// Use left Max cardinality.
card = props.Cardinality{Min: 0, Max: left.Max}

case opt.ExceptAllOp:
// Use left Max cardinality. Cardinality cannot be less than left Min minus
// right Max.
card = props.Cardinality{Min: 0, Max: left.Max}
if left.Min > right.Max {
card.Min = left.Min - right.Max
}
Expand Down
54 changes: 54 additions & 0 deletions pkg/sql/opt/memo/testdata/logprops/set
Original file line number Diff line number Diff line change
Expand Up @@ -585,3 +585,57 @@ except-all
└── eq [type=bool, outer=(6,7), constraints=(/6: (/NULL - ]; /7: (/NULL - ]), fd=(6)==(7), (7)==(6)]
├── variable: a:6 [type=int]
└── variable: b:7 [type=int]

# Regression test for #89101. EXCEPT can have cardinality 0 even if left side
# has more rows than the right.
norm
VALUES (1), (1) EXCEPT VALUES (1)
----
except
├── columns: column1:1(int!null)
├── left columns: column1:1(int!null)
├── right columns: column1:2(int)
├── cardinality: [0 - 2]
├── key: (1)
├── values
│ ├── columns: column1:1(int!null)
│ ├── cardinality: [2 - 2]
│ ├── prune: (1)
│ ├── tuple [type=tuple{int}]
│ │ └── const: 1 [type=int]
│ └── tuple [type=tuple{int}]
│ └── const: 1 [type=int]
└── values
├── columns: column1:2(int!null)
├── cardinality: [1 - 1]
├── key: ()
├── fd: ()-->(2)
├── prune: (2)
└── tuple [type=tuple{int}]
└── const: 1 [type=int]

# EXCEPT ALL cannot have cardinality lower than min left - max right.
norm
VALUES (1), (1) EXCEPT ALL VALUES (1)
----
except-all
├── columns: column1:1(int!null)
├── left columns: column1:1(int!null)
├── right columns: column1:2(int)
├── cardinality: [1 - 2]
├── values
│ ├── columns: column1:1(int!null)
│ ├── cardinality: [2 - 2]
│ ├── prune: (1)
│ ├── tuple [type=tuple{int}]
│ │ └── const: 1 [type=int]
│ └── tuple [type=tuple{int}]
│ └── const: 1 [type=int]
└── values
├── columns: column1:2(int!null)
├── cardinality: [1 - 1]
├── key: ()
├── fd: ()-->(2)
├── prune: (2)
└── tuple [type=tuple{int}]
└── const: 1 [type=int]

0 comments on commit d5c2318

Please sign in to comment.