Skip to content

[AURON #2182] Implement native support for percent_rank window function#2204

Draft
weimingdiit wants to merge 1 commit intoapache:masterfrom
weimingdiit:feat/native-window-percent_rank
Draft

[AURON #2182] Implement native support for percent_rank window function#2204
weimingdiit wants to merge 1 commit intoapache:masterfrom
weimingdiit:feat/native-window-percent_rank

Conversation

@weimingdiit
Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #2182

Rationale for this change

Auron does not currently support percent_rank() in native window execution, so queries using this function fall back to Spark. This leaves a gap in native window function coverage.

percent_rank() cannot be implemented with the existing streaming-style window processors alone, because its result depends on both the row rank and the total number of rows in the partition. To match Spark semantics, the native engine needs to evaluate it with full-partition context.

What changes are included in this PR?

This PR adds native support for percent_rank() window function end to end.
The main changes are:

  • Add PERCENT_RANK to the window function protobuf and planner conversion path so Spark plans can be serialized into native plans correctly.
  • Extend NativeWindowBase to recognize Spark's PercentRank expression and convert it to the native window function enum.
  • Introduce a native PercentRankProcessor that computes percent rank with Spark-compatible semantics:
    • rows in the same peer group share the same rank
    • the result is (rank - 1) / (partition_size - 1)
    • single-row partitions return 0.0
  • Add a full-partition execution path in native window execution for functions that require complete partition context, and use that path for percent_rank().
  • Add tests on both the native execution side and the Spark SQL side to verify correctness.

Are there any user-facing changes?

Yes.

Queries using percent_rank() window function can now stay on the native execution path instead of falling back to Spark, as long as the rest of the plan is supported by Auron. No user-facing configuration changes are introduced.

How was this patch tested?

CI.

…function

Signed-off-by: weimingdiit <weimingdiit@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement native support for percent_rank window function

1 participant