Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-32513][core] Add predecessor caching #24475

Closed
wants to merge 2 commits into from

Conversation

jeyhunkarimov
Copy link
Contributor

@jeyhunkarimov jeyhunkarimov commented Mar 10, 2024

What is the purpose of the change

This pull request adds caching capability to Transformation

Brief change log

  • Change Transformation::getTransitivePredecessors() to utilize cache
  • Introduce Transformation::getTransitivePredecessorsInternal()

Verifying this change

This change added tests to org.apache.flink.api.dag.TransformationTest::testPredecessorCache()

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
  • The serializers: (no)
  • The runtime per-record code paths (performance sensitive): (no)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
  • The S3 file system connector: (no)

Documentation

  • Does this pull request introduce a new feature? (no)

@jeyhunkarimov jeyhunkarimov marked this pull request as draft March 10, 2024 14:59
@flinkbot
Copy link
Collaborator

flinkbot commented Mar 10, 2024

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@jeyhunkarimov jeyhunkarimov changed the title [WIP][FLINK-32513][core] Add predecessor caching [FLINK-32513][core] Add predecessor caching Mar 10, 2024
@jeyhunkarimov jeyhunkarimov marked this pull request as ready for review March 10, 2024 19:48
@jeyhunkarimov
Copy link
Contributor Author

Hi @zhuzhurk could you please review the PR in you available time? Thanks!

Copy link
Contributor

@zhuzhurk zhuzhurk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for creating the fix. @jeyhunkarimov
I have two comments. Please take a look.

@jeyhunkarimov
Copy link
Contributor Author

Thanks @zhuzhurk for the review. I addressed them. Could you please check again?

@jeyhunkarimov jeyhunkarimov force-pushed the FLINK-32513 branch 10 times, most recently from b07d0d3 to eaa69e3 Compare March 16, 2024 23:46
Copy link
Contributor

@zhuzhurk zhuzhurk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.
Thanks for addressing all the comments! @jeyhunkarimov
Would you squash the last two commits and rebase the changes onto the latest Flink master branch?

@jeyhunkarimov
Copy link
Contributor Author

jeyhunkarimov commented Mar 22, 2024

Looks good to me. Thanks for addressing all the comments! @jeyhunkarimov

Thanks a lot @zhuzhurk for your reviews.

Would you squash the last two commits and rebase the changes onto the latest Flink master branch?

Done.

@zhuzhurk zhuzhurk closed this in 8dcb0ae Mar 22, 2024
zhuzhurk pushed a commit that referenced this pull request Mar 22, 2024
zhuzhurk pushed a commit that referenced this pull request Mar 22, 2024
hanyuzheng7 pushed a commit to hanyuzheng7/flink that referenced this pull request May 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants