Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-15001] [table-planner-blink] The digest of sub-plan reuse should contain retraction traits for stream physical nodes #10377

Closed
wants to merge 2 commits into from

Conversation

godfreyhe
Copy link
Contributor

What is the purpose of the change

currently, the digest of sub-plan reuse does not contain retraction traits for stream physical nodes now, which maybe cause two sub-plans be reused even if they have different retraction traits and the result is wrong. The pr aims to fix this issue.

Brief change log

  • Add retraction traits to digest in RelDigestWriterImpl
  • refactor getDigest method and replace RelDigestWriterImpl with RelTreeWriterImpl

Verifying this change

This change is already covered by existing tests, such as DagOptimizationTest.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (yes / no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
  • The serializers: (yes / no / don't know)
  • The runtime per-record code paths (performance sensitive): (yes / no / don't know)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / no / don't know)
  • The S3 file system connector: (yes / no / don't know)

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

…ld contain retraction traits for stream physical nodes
…lace RelDigestWriterImpl with RelTreeWriterImpl
@flinkbot
Copy link
Collaborator

flinkbot commented Dec 2, 2019

Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.

Automated Checks

Last check on commit 71dae3a (Wed Dec 04 15:14:07 UTC 2019)

Warnings:

  • No documentation files were touched! Remember to keep the Flink docs up to date!

Mention the bot in a comment to re-run the automated checks.

Review Progress

  • ❓ 1. The [description] looks good.
  • ❓ 2. There is [consensus] that the contribution should go into to Flink.
  • ❓ 3. Needs [attention] from.
  • ❓ 4. The change fits into the overall [architecture].
  • ❓ 5. Overall code [quality] is good.

Please see the Pull Request Review Guide for a full explanation of the review process.


The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands
The @flinkbot bot supports the following commands:

  • @flinkbot approve description to approve one or more aspects (aspects: description, consensus, architecture and quality)
  • @flinkbot approve all to approve all aspects
  • @flinkbot approve-until architecture to approve everything until architecture
  • @flinkbot attention @username1 [@username2 ..] to require somebody's attention
  • @flinkbot disapprove architecture to remove an approval you gave earlier

@flinkbot
Copy link
Collaborator

flinkbot commented Dec 2, 2019

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run travis re-run the last Travis build
  • @flinkbot run azure re-run the last Azure build

@godfreyhe
Copy link
Contributor Author

cc @KurtYoung @hequn8128

Copy link
Contributor

@hequn8128 hequn8128 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@godfreyhe Thanks a lot for the fix. 👍 I have checked the retraction traits in the tests and have spotted no errors.

As I'm not familiar with the plan reuse logic, +1 to merge once @KurtYoung have no concerns.

Best, Hequn

@KurtYoung
Copy link
Contributor

@hequn8128 feel free to merge

@hequn8128
Copy link
Contributor

Merging...

hequn8128 pushed a commit to hequn8128/flink that referenced this pull request Dec 12, 2019
…ld contain retraction traits for stream physical nodes

This closes apache#10377.
hequn8128 pushed a commit to hequn8128/flink that referenced this pull request Dec 12, 2019
…ld contain retraction traits for stream physical nodes

This closes apache#10377.
@hequn8128 hequn8128 closed this in ef0a033 Dec 12, 2019
hequn8128 pushed a commit that referenced this pull request Dec 12, 2019
…ld contain retraction traits for stream physical nodes

This closes #10377.
@godfreyhe godfreyhe deleted the FLINK-15001 branch December 12, 2019 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants