Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-32831][table-planner] RuntimeFilterProgram should aware join type when looking for the build side #23216

Closed
wants to merge 5 commits into from

Conversation

wanglijie95
Copy link
Contributor

What is the purpose of the change

Currently, runtime filter program will try to look for an Exchange as build side to avoid affecting MultiInput. It will try to push down the runtime filter builder if the original build side is not Exchange.

Currenlty, the builder-push-down does not aware the join type, which may lead to incorrect results(For example, push down the builder to the right input of left-join).

We should only support following cases:

  1. Inner join: builder can push to left + right input
  2. semi join: builder can push to left + right input
  3. left join: builder can only push to the left input
  4. right join: builder can only push to the right input

Verifying this change

Add test testBuildSideIsLeftJoinWithoutExchange

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
  • The serializers: (no)
  • The runtime per-record code paths (performance sensitive): (no)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
  • The S3 file system connector: (no)

Documentation

  • Does this pull request introduce a new feature? (no)
  • If yes, how is the feature documented? (not applicable)

@wanglijie95
Copy link
Contributor Author

Hi @lsyldliu, could you help to review this PR when you are free?

@flinkbot
Copy link
Collaborator

flinkbot commented Aug 15, 2023

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

Copy link
Contributor

@lsyldliu lsyldliu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wanglijie95 Thanks for your contribution, I left some comments.

@wanglijie95
Copy link
Contributor Author

Thanks for review @lsyldliu , I 've addressed or replied your comments, PTAL.

Copy link
Contributor

@lsyldliu lsyldliu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wanglijie95 Thanks for your contribution, I left some comments.

Copy link
Contributor

@lsyldliu lsyldliu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wanglijie95 Thanks for contribution, LGTM

@wanglijie95
Copy link
Contributor Author

CI fails due to FLINK-32751. @flinkbot run azure

wanglijie95 added a commit that referenced this pull request Aug 28, 2023
…eFilterITcase have applied runtime filter.

This closes #23216
@wanglijie95 wanglijie95 deleted the FLINK-32831 branch August 28, 2023 01:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants