[VL] The aggregation results mismatch between Spark3.2.2 and Gluten 1.3

[beliefer]

Backend

VL (Velox)

Bug description

There is a SQL show below.

select
  20251026 ,
  os_name ,
  count(distinct a.aid)
from
  (select ..) a
where
  a.ds = '20251026'
group by
  os_name

The subquery a have the same results between Spark3.2.2 and Gluten 1.3. But the outer query returns different results.

The output of Spark 3.2.2

20251026	1	847
20251026	2	527

The output of Spark 3.2.2 + Gluten 1.3

20251026	1	846
20251026	2	528

Gluten version

No response

Spark version

None

Spark configurations

No response

System information

No response

Relevant logs

[wForget]

Could you check if the results are consistent after adding aid to group? like:

select
  20251026 ,
  os_name ,
  a.aid,
  count(a.aid)
from
  (select ..) a
where
  a.ds = '20251026'
group by ds, os_name, a.aid

[beliefer]

As you said, I rerun the SQL, I paste the results into achievement.

[beliefer]

Gluten-result.txt
Spark-result.txt

[wForget]

Gluten-result.txt Spark-result.txt

os_name may have inconsistent, this is inconsistent line:

Spark:
20251026	1	2095193	1
Gluten:
20251026	2	2095193	1

[FelixYBW]

Any possible to try main branch?

[beliefer]

Any possible to try main branch?

Yes. I tried main branch (2025_11_01), it is still exists.

[beliefer]

@wForget @FelixYBW After my investigation and analysis, I found that is caused by the sort of subquery is different. The subquery does not have a clear sorting, so the sort of output is determined by different engines. I think the result is expected.
Users should add ORDER BY col for subquery if they want the determined results.

[FelixYBW]

Thank you for update. A result mismatch in general operator is very risky which we must solve immediately.