ESQL: STATS query fails with ClassCastException

[jan-elastic]

Elasticsearch Version

main

Installed Plugins

No response

Java Version

n/a

OS Version

n/a

Problem Description

The query

ROW x = [1,2,3] | STATS MV_COUNT(VALUES(x)), VALUES(x), SUM(x) 

results in

class_cast_exception - class org.elasticsearch.compute.data.IntArrayBlock cannot be cast to class org.elasticsearch.compute.data.LongBlock (org.elasticsearch.compute.data.IntArrayBlock and org.elasticsearch.compute.data.LongBlock are in unnamed module of loader java.net.URLClassLoader @62cba181)

If you remove any of the three aggs, it works.

Steps to Reproduce

see above

Logs (if relevant)

[2025-09-02T16:21:25,633][DEBUG][o.e.x.e.a.EsqlResponseListener] [runTask-0] ESQL query execution failed.
Query string or async ID: [
    ROW x = [1,2,3] | STATS MV_COUNT(VALUES(x)), VALUES(x), SUM(x) 
  ]
Execution time: 8ms
[2025-09-02T16:21:25,633][WARN ][o.e.x.e.a.EsqlResponseListener] [runTask-0] ESQL request failed with status [INTERNAL_SERVER_ERROR]: java.lang.ClassCastException: class org.elasticsearch.compute.data.IntArrayBlock cannot be cast to class org.elasticsearch.compute.data.LongBlock (org.elasticsearch.compute.data.IntArrayBlock and org.elasticsearch.compute.data.LongBlock are in unnamed module of loader java.net.URLClassLoader @62cba181)
	at org.elasticsearch.compute.aggregation.SumIntAggregatorFunction.addIntermediateInput(SumIntAggregatorFunction.java:150)
	at org.elasticsearch.compute.aggregation.Aggregator.processPage(Aggregator.java:41)
	at org.elasticsearch.compute.operator.AggregationOperator.addInput(AggregationOperator.java:111)
	at org.elasticsearch.compute.operator.Driver.runSingleLoopIteration(Driver.java:286)
	at org.elasticsearch.compute.operator.Driver.run(Driver.java:186)
	at org.elasticsearch.compute.operator.Driver$1.doRun(Driver.java:420)
	at org.elasticsearch.server@9.2.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)
	at org.elasticsearch.compute.operator.DriverScheduler$1.doRun(DriverScheduler.java:57)
	at org.elasticsearch.server@9.2.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)
	at org.elasticsearch.server@9.2.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:35)
	at org.elasticsearch.server@9.2.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:1067)
	at org.elasticsearch.server@9.2.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1095)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:619)
	at java.base/java.lang.Thread.run(Thread.java:1447)

[2025-09-02T16:21:25,633][WARN ][r.suppressed             ] [runTask-0] path: /_query, params: {pretty=true, format=txt}, status: 500 java.lang.ClassCastException: class org.elasticsearch.compute.data.IntArrayBlock cannot be cast to class org.elasticsearch.compute.data.LongBlock (org.elasticsearch.compute.data.IntArrayBlock and org.elasticsearch.compute.data.LongBlock are in unnamed module of loader java.net.URLClassLoader @62cba181)
	at org.elasticsearch.compute.aggregation.SumIntAggregatorFunction.addIntermediateInput(SumIntAggregatorFunction.java:150)
	at org.elasticsearch.compute.aggregation.Aggregator.processPage(Aggregator.java:41)
	at org.elasticsearch.compute.operator.AggregationOperator.addInput(AggregationOperator.java:111)
	at org.elasticsearch.compute.operator.Driver.runSingleLoopIteration(Driver.java:286)
	at org.elasticsearch.compute.operator.Driver.run(Driver.java:186)
	at org.elasticsearch.compute.operator.Driver$1.doRun(Driver.java:420)
	at org.elasticsearch.server@9.2.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)
	at org.elasticsearch.compute.operator.DriverScheduler$1.doRun(DriverScheduler.java:57)
	at org.elasticsearch.server@9.2.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)
	at org.elasticsearch.server@9.2.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:35)
	at org.elasticsearch.server@9.2.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:1067)
	at org.elasticsearch.server@9.2.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1095)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:619)
	at java.base/java.lang.Thread.run(Thread.java:1447)

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)

[alex-spies]

Fun, you can also get NPE!

ROW x = [1.1,2,3] | STATS MV_COUNT(VALUES(x)), VALUES(x), SUM(x)

"type":"null_pointer_exception","reason":"Cannot invoke \"org.elasticsearch.compute.data.DoubleVector.getPositionCount()\" because \"value\" is null"

[alex-spies]

The CCE/NPE doesn't reproduce for me when I use an index instead of the ROW x = [1,2,3].

[alex-spies]

Got a reduced reproducer, which shows that this is not specific to VALUES but rather seems to happen when the same agg function is used twice, and the command starts with ROW:

ROW x = 1 | STATS 2*COUNT_DISTINCT(x), COUNT_DISTINCT(x), MAX(x)

[alex-spies]

Above, it seems like ReplaceAggregateAggExpressionWithEval doesn't de-duplicate the two COUNT_DISTINCT(x) instances. May be that our aggs assume that we never compute the same agg function over the same column twice.

ROW x = 1 | STATS 2*COUNT_DISTINCT(x), COUNT_DISTINCT(x), MAX(x)

[2025-09-16T11:51:38,283][TRACE][o.e.x.e.o.L.changes      ] [runTask-0] Rule logical.ReplaceAggregateAggExpressionWithEval applied with change
Limit[1000[INTEGER],false]                                                                                                  = Limit[1000[INTEGER],false]
\_Aggregate[[],[2[INTEGER] * COUNTDISTINCT(x{r}#579,true[BOOLEAN]) AS 2*COUNT_DISTINCT(x)#581, COUNTDISTINCT(x{r}#579,true[ ! \_Project[[2*COUNT_DISTINCT(x){r}#581, COUNT_DISTINCT(x){r}#583, MAX(x){r}#585]]
BOOLEAN]) AS COUNT_DISTINCT(x)#583, MAX(x{r}#579,true[BOOLEAN]) AS MAX(x)#585]]                                             !   \_Eval[[2[INTEGER] * $$COUNTDISTINCT$2*COUNT_DISTINC>$0{r$}#586 AS 2*COUNT_DISTINCT(x)#581]]
  \_Row[[1[INTEGER] AS x#579]]                                                                                              !     \_Aggregate[[],[COUNTDISTINCT(x{r}#579,true[BOOLEAN]) AS $$COUNTDISTINCT$2*COUNT_DISTINC>$0#586, COUNTDISTINCT(x{r}#579,tru
                                                                                                                            ! e[BOOLEAN]) AS COUNT_DISTINCT(x)#583, MAX(x{r}#579,true[BOOLEAN]) AS MAX(x)#585]]
                                                                                                                            !       \_Row[[1[INTEGER] AS x#579]]

In the same query with FROM test rather than ROW x = 1, the Aggregate node only computes 1 instance of COUNT_DISTINCT(x) after ReplaceAggregateAggExpressionWithEval has done its thing (and we get correct results):

from test3 | STATS 2*COUNT_DISTINCT(x), COUNT_DISTINCT(x), MAX(x)
                                                                                                                         
[2025-09-16T11:54:57,483][TRACE][o.e.x.e.o.L.changes      ] [runTask-0] Rule logical.ReplaceAggregateAggExpressionWithEval applied with change
Limit[1000[INTEGER],false]                                                                                                  = Limit[1000[INTEGER],false]
\_Aggregate[[],[2[INTEGER] * COUNTDISTINCT(x{f}#599,true[BOOLEAN]) AS 2*COUNT_DISTINCT(x)#594, COUNTDISTINCT(x{f}#599,true[ ! \_Project[[2*COUNT_DISTINCT(x){r}#594, $$COUNTDISTINCT$2*COUNT_DISTINC>$0{r$}#600 AS COUNT_DISTINCT(x)#596, MAX(x){r}#59
BOOLEAN]) AS COUNT_DISTINCT(x)#596, MAX(x{f}#599,true[BOOLEAN]) AS MAX(x)#598]]                                             ! 8]]
  \_EsRelation[test3][x{f}#599]                                                                                             !   \_Eval[[2[INTEGER] * $$COUNTDISTINCT$2*COUNT_DISTINC>$0{r$}#600 AS 2*COUNT_DISTINCT(x)#594]]
                                                                                                                            !     \_Aggregate[[],[COUNTDISTINCT(x{f}#599,true[BOOLEAN]) AS $$COUNTDISTINCT$2*COUNT_DISTINC>$0#600, MAX(x{f}#599,true[BOOLEAN]
                                                                                                                            ! ) AS MAX(x)#598]]
                                                                                                                            !       \_EsRelation[test3][x{f}#599]

[astefan]

This issue reminded me of this one #113393
It doesn't seem to be the same, but there is some resemblance.

[luigidellaquila]

A few more failures here #136584

eg.

from multivalue_geom*,no_mapping_sample_data | rename `id` AS `NfHBAZaCUm`, contains AS intersects | drop `intersects` | stats  uuHzOSMmAGp = count(NfHBAZaCUm), `shape` = median(NfHBAZaCUm) + max(NfHBAZaCUm) + max(NfHBAZaCUm), `smaller` = count(NfHBAZaCUm) + count(NfHBAZaCUm), xxGKVoyhS = count_distinct(NfHBAZaCUm), kuNvvqRtEgq = count(*)