Skip to content

[FLINK-40095][table-planner] Fix JSON aggregation functions failing in group window aggregate#28682

Open
xuyangzhong wants to merge 2 commits into
apache:masterfrom
xuyangzhong:json_agg
Open

[FLINK-40095][table-planner] Fix JSON aggregation functions failing in group window aggregate#28682
xuyangzhong wants to merge 2 commits into
apache:masterfrom
xuyangzhong:json_agg

Conversation

@xuyangzhong

Copy link
Copy Markdown
Contributor

What is the purpose of the change

This pull request fixes JSON aggregation functions (JSON_OBJECTAGG / JSON_ARRAYAGG) failing when used in a group window aggregate (e.g. GROUP BY TUMBLE(...)). Previously, WrapJsonAggFunctionArgumentsRule only matched LogicalAggregate, so JSON aggregate arguments were never wrapped into JSON_STRING when the aggregation was a LogicalWindowAggregate, leading to a failure during planning. This change extends the rule to also cover window aggregates.

Brief change log

  • Generalized WrapJsonAggFunctionArgumentsRule to operate on the base Aggregate class and split the single INSTANCE into AGGREGATE_INSTANCE (for LogicalAggregate) and WINDOW_AGGREGATE_INSTANCE (for LogicalWindowAggregate), with a generic onAggregateClass(Class) config factory
  • Added a hints field to WindowAggregate and its subclasses (LogicalWindowAggregate, FlinkLogicalWindowAggregate), and implemented withHints on LogicalWindowAggregate so the rule's marker hint (used to prevent repeated transformation) is preserved
  • Registered both rule instances in FlinkBatchRuleSets and FlinkStreamRuleSets, and updated the HINT_NAME_JSON_AGGREGATE_WRAPPED excluded-rules strategy in FlinkHintStrategies

Verifying this change

New tests are added to verify this pr.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented?

@flinkbot

flinkbot commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@raminqaf raminqaf left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @xuyangzhong! I did a quick first pass.

new FlinkLogicalWindowAggregate(
tableAgg.getCluster(),
tableAgg.getTraitSet(),
Collections.emptyList(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here and everywhere else please use List.of()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beside that, shouldn't we here also pass the hint?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is because FlinkLogicalWindowTableAggregate is not a hintable node.

@github-actions github-actions Bot added the community-reviewed PR has been reviewed by the community. label Jul 8, 2026
@xuyangzhong

Copy link
Copy Markdown
Contributor Author

Thanks for your review @raminqaf ! Updated

@raminqaf raminqaf left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing! Left two nits. Should be good in the next round!

val logicalWindowAgg = new LogicalWindowAggregate(
ts.getCluster,
ts.getTraitSet,
util.List.of(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Suggested change
util.List.of(),
List.of(),

val windowAgg = new LogicalWindowAggregate(
ts.getCluster,
ts.getTraitSet,
java.util.List.of(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: import java.util

Suggested change
java.util.List.of(),
List.of(),

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-reviewed PR has been reviewed by the community.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants