Skip to content

[SPARK-58095][SQL] approx_top_k_combine throws scala.MatchError on empty input#57289

Draft
jiwen624 wants to merge 2 commits into
apache:masterfrom
jiwen624:SPARK-58095
Draft

[SPARK-58095][SQL] approx_top_k_combine throws scala.MatchError on empty input#57289
jiwen624 wants to merge 2 commits into
apache:masterfrom
jiwen624:SPARK-58095

Conversation

@jiwen624

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Fix the MatchError on empty input with changes below:

  • Fall back to the statically-known item data type carried by the state struct (uncheckedItemDataType) when no input sketch resolved it, in both eval and serialize.
  • When combine size is unspecified and no input was consumed, maxItemsTracked is still the internal VOID sentinel (-1); resolve it to the default in eval only (a partial buffer must keep VOID for the final merge), so the empty output sketch carries a valid size that downstream functions such as approx_top_k_estimate accept.

Why are the changes needed?

The query aborts with an internal error instead of returning an empty result, unlike the sibling functions (approx_top_k, approx_top_k_accumulate, approx_top_k_estimate) which handle empty input correctly:

SELECT approx_top_k_combine(sketch, 5)
FROM (SELECT approx_top_k_accumulate(expr) AS sketch
      FROM VALUES (1), (2) AS t(expr))
WHERE false;
-- before: org.apache.spark.SparkException ... scala.MatchError: null
-- after:  returns a single empty sketch (estimates to [])

Does this PR introduce any user-facing change?

Yes. Fixed an query failing issue when the input is empty.

How was this patch tested?

Added UT case

Was this patch authored or co-authored using generative AI tooling?

Yes.

@jiwen624 jiwen624 changed the title [SPARK-58095][SQL] Fix approx_top_k_combine MatchError on empty input [SPARK-58095][SQL] approx_top_k_combine throws MatchError on empty input Jul 16, 2026
@jiwen624 jiwen624 changed the title [SPARK-58095][SQL] approx_top_k_combine throws MatchError on empty input [SPARK-58095][SQL] approx_top_k_combine throws scala.MatchError on empty input Jul 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant