Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-28247][hive] fix "no any field presented in group by" exception when over window contains grouping function in Hive dialect #20071

Closed
wants to merge 2 commits into from

Conversation

luoyuxia
Copy link
Contributor

@luoyuxia luoyuxia commented Jun 25, 2022

What is the purpose of the change

To fix the exception "no any field presented in group by" when over window contains grouping function in Hive dialect like
SELECT grouping(category), lag(live) over(partition by grouping(category)) FROM t GROUP BY category, live

Brief change log

  • Not to rewrite group function again when it has been rewritten. The main changes logic is from HIVE-15996

Verifying this change

Added test in HiveDialectQueryITCase.java#testWindowWithGrouping

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? N/A

@flinkbot
Copy link
Collaborator

flinkbot commented Jun 25, 2022

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@luoyuxia
Copy link
Contributor Author

@flinkbot run azure

@lirui-apache
Copy link
Contributor

Thanks @luoyuxia for fixing the issue. Should we also add test cases for grouping function with multiple parameters?

@luoyuxia
Copy link
Contributor Author

@lirui-apache Thanks for reminder. I add a test for it to make sure this changes won't break such case.

@luoyuxia
Copy link
Contributor Author

@flinkbot run azure

1 similar comment
@lirui-apache
Copy link
Contributor

@flinkbot run azure

@lirui-apache
Copy link
Contributor

I don't think the e2e test failure is related but I guess we need a green run to merge. @luoyuxia could you post your own CI result if it's successful?

@luoyuxia
Copy link
Contributor Author

luoyuxia commented Jul 6, 2022

@flinkbot run azure

1 similar comment
@luoyuxia
Copy link
Contributor Author

luoyuxia commented Jul 6, 2022

@flinkbot run azure

@luoyuxia
Copy link
Contributor Author

luoyuxia commented Jul 7, 2022

@lirui-apache The CI is passed now~

ericccarlson pushed a commit to ericccarlson/flink that referenced this pull request Jul 11, 2022
…n when over window contains grouping function in Hive dialect

This closes apache#20071.
liujiawinds pushed a commit to liujiawinds/flink that referenced this pull request Jul 22, 2022
…n when over window contains grouping function in Hive dialect

This closes apache#20071.
huangxiaofeng10047 pushed a commit to huangxiaofeng10047/flink that referenced this pull request Nov 3, 2022
…n when over window contains grouping function in Hive dialect

This closes apache#20071.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants