[CALCITE-7382] The TopDownGeneralDecorrelator returns an error result when a subquery contains a LIMIT 1 by silundong · Pull Request #4754 · apache/calcite

silundong · 2026-01-18T09:45:42Z

This PR makes two changes:

During decorrelation, Sort will always be rewritten as [Filter-]Window, not only when both ORDER BY and LIMIT/OFFSET are present, because ORDER BY, LIMIT and OFFSET have to be enforced per value of the outer bindings instead of globally.
For the JIRA case, the bug was that when rewriting Sort into Filter-Window, the decorrelator forgot to set the frame clause for the window function, causing ROW_NUMBER() to be computed incorrectly. This has now been fixed.

rubenada · 2026-01-18T10:18:06Z

core/src/test/resources/sql/new-decorr.iq

+
+!ok
+
+select dname, (select empno from emp where dept.deptno = emp.deptno limit 1) from dept where deptno = 10;


Shouldn't we have an ORDER BY in the sub-query (before the LIMIT 1) to make sure that the query result is deterministic?

This test is intended to demonstrate that Sort needs to be rewritten as a [Filter-]Window regardless of whether it has an ORDER BY or LIMIT/OFFSET. As I updated in the comment

because now the order/limit/offset has to be enforced per value of the outer bindings instead of globally.

This is exactly what @asolimando was concerned about in his comment below. Perhaps in practical applications there's almost no need to write it this way; can I call this a edge case?

asolimando · 2026-01-18T11:25:09Z

core/src/test/resources/sql/new-decorr.iq


+# [CALCITE-7382] The TopDownGeneralDecorrelator returns an error result when a subquery contains a LIMIT 1
+!use scott
+SELECT dname, (SELECT emp.comm FROM "scott".emp where dept.deptno = emp.deptno ORDER BY emp.comm limit 1) FROM "scott".dept;


Suggested change

SELECT dname, (SELECT emp.comm FROM "scott".emp where dept.deptno = emp.deptno ORDER BY emp.comm limit 1) FROM "scott".dept;

SELECT dname, (SELECT emp.comm FROM "scott".emp where dept.deptno = emp.deptno ORDER BY emp.comm LIMIT 1) FROM "scott".dept;

asolimando · 2026-01-18T11:25:41Z

core/src/main/java/org/apache/calcite/sql2rel/TopDownGeneralDecorrelator.java

-      builder.filter(conditions);
-    } else {
-      builder.sortLimit(sort.offset, sort.fetch, builder.fields(shiftCollation));
+    // the Sort have to be changed during rewriting because now the order/limit/offset has to be


Q: was this a shortcoming of the original paper or an implementation issue?

The paper's original wording is:

Subqueries with ORDER BY and LIMIT or OFFSET have to be changed....

It doesn't say what to do when only LIMIT or OFFSET is present. However, I believe that even with only LIMIT or OFFSET, it should be rewritten as a window function without ORDER clause.
This matches the test case that @rubenada mentioned in his comment. I performed the same test in the umbra-db interface, and it indeed rewrite the LIMIT as a window function. It seems our approach aligns with the paper's intent.

xiedeyantu · 2026-01-18T13:19:42Z

After using the changes made in this PR, this case no longer works properly.

# [CALCITE-709] Errors with LIMIT inside scalar sub-query
!use scott
select deptno, (select sum(empno) from "scott".emp where deptno = dept.deptno limit 0) as x from "scott".dept;
+--------+---+
| DEPTNO | X |
+--------+---+
|     10 |   |
|     20 |   |
|     30 |   |
|     40 |   |
+--------+---+
(4 rows)

!ok

Caused by: java.lang.ArithmeticException: Value 38216 out of range
at org.apache.calcite.linq4j.tree.Primitive.checkRoundedRange(Primitive.java:387)
at org.apache.calcite.linq4j.tree.Primitive.numberValue(Primitive.java:564)
at org.apache.calcite.linq4j.tree.Primitive.numberValueRoundDown(Primitive.java:539)
at Baz$3.apply(Unknown Source)
at Baz$3.apply(Unknown Source)
at Baz$3.apply(Unknown Source)
at org.apache.calcite.adapter.enumerable.BasicAggregateLambdaFactory$AccumulatorAdderSeq.apply(BasicAggregateLambdaFactory.java:81)
at org.apache.calcite.linq4j.EnumerableDefaults.groupBy_(EnumerableDefaults.java:1177)
at org.apache.calcite.linq4j.EnumerableDefaults.groupBy(EnumerableDefaults.java:782)
at org.apache.calcite.linq4j.DefaultEnumerable.groupBy(DefaultEnumerable.java:312)
at Baz.bind(Unknown Source)
at org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable(CalcitePrepare.java:367)
at org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable(CalciteConnectionImpl.java:335)
at org.apache.calcite.jdbc.CalciteMetaImpl._createIterable(CalciteMetaImpl.java:609)
at org.apache.calcite.jdbc.CalciteMetaImpl.createIterable(CalciteMetaImpl.java:600)
at org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:184)
at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:64)
at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:43)

silundong · 2026-01-18T13:51:36Z

@xiedeyantu Hi, I don't think this is caused by the decorrelation algorithm. It looks like a loss of significant digits during type casting. The failure seems to come from the Aggregate when computing SUM. I tested:

SELECT SUM(empno) FROM emp GROUP BY deptno;

and it fails the same way. Perhaps a new ticket needs to be created to record it.

xiedeyantu · 2026-01-18T14:15:00Z

@xiedeyantu Hi, I don't think this is caused by the decorrelation algorithm. It looks like a loss of significant digits during type casting. The failure seems to come from the Aggregate when computing SUM. I tested:

SELECT SUM(empno) FROM emp GROUP BY deptno;

and it fails the same way. Perhaps a new ticket needs to be created to record it.

Thanks! I have file a new jira CALCITE-7384 to log this issue.

xiedeyantu · 2026-01-18T15:59:21Z

CALCITE-7384 This is an old problem that can be solved using cast(col as bigint), and it has been discussed many times. We can ignore this problem.

xiedeyantu · 2026-01-18T16:04:34Z

core/src/test/resources/sql/new-decorr.iq

 !ok

+# [CALCITE-7382] The TopDownGeneralDecorrelator returns an error result when a subquery contains a LIMIT 1
+!use scott


We can add a note to record where the currently processed case originated.

# This case comes from sub-query.iq [CALCITE-6652]

asolimando

LGTM, there is conflict to be resolved but other than that it seems good to go, nice job!

… when a subquery contains a LIMIT 1

sonarqubecloud · 2026-01-20T14:58:37Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

rubenada reviewed Jan 18, 2026

View reviewed changes

asolimando reviewed Jan 18, 2026

View reviewed changes

xiedeyantu reviewed Jan 18, 2026

View reviewed changes

silundong requested review from asolimando, rubenada and xiedeyantu January 19, 2026 14:14

xiedeyantu approved these changes Jan 19, 2026

View reviewed changes

asolimando approved these changes Jan 20, 2026

View reviewed changes

rubenada approved these changes Jan 20, 2026

View reviewed changes

[CALCITE-7382] The TopDownGeneralDecorrelator returns an error result…

1974254

… when a subquery contains a LIMIT 1

silundong force-pushed the issue-7382 branch from ac26c20 to 1974254 Compare January 20, 2026 14:44

silundong merged commit 270ba64 into apache:main Jan 20, 2026
20 checks passed

silundong deleted the issue-7382 branch February 9, 2026 13:22


		!ok

		select dname, (select empno from emp where dept.deptno = emp.deptno limit 1) from dept where deptno = 10;

	SELECT dname, (SELECT emp.comm FROM "scott".emp where dept.deptno = emp.deptno ORDER BY emp.comm limit 1) FROM "scott".dept;
	SELECT dname, (SELECT emp.comm FROM "scott".emp where dept.deptno = emp.deptno ORDER BY emp.comm LIMIT 1) FROM "scott".dept;

Comments

Conversation

silundong commented Jan 18, 2026

Uh oh!

rubenada Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

silundong Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

asolimando Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

asolimando Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

silundong Jan 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xiedeyantu commented Jan 18, 2026

Uh oh!

silundong commented Jan 18, 2026

Uh oh!

xiedeyantu commented Jan 18, 2026

Uh oh!

xiedeyantu commented Jan 18, 2026

Uh oh!

xiedeyantu Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

asolimando left a comment

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud bot commented Jan 20, 2026

Quality Gate passed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

silundong Jan 18, 2026 •

edited

Loading