Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-9074] [e2e] Add e2e for resuming from externalized checkpoints #5969

Closed
wants to merge 2 commits into from

Conversation

tzulitai
Copy link
Contributor

@tzulitai tzulitai commented May 9, 2018

What is the purpose of the change

This PR adds an end-to-end test for resuming from externalized, retained checkpoints.

The test does the following:

  • Runs the general purpose DataStream job, with externalized checkpoints enabled
  • Waits until the job has at least 1 completed checkpoints, AND has processed at least 200 records
  • Cancel the job
  • Make sure that there is exactly 1 externalized checkpoint available
  • Restore from that, wait for another 200 records to be processed to verify that exactly-once isn't violated

Brief change log

  • Allow general purpose job to be configured for externalized checkpoints
  • Add new e2e test script test_resume_externalized_checkpoint.sh

Verifying this change

This PR adds a new test, test_resume_externalized_checkpoint.sh.

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@fhueske
Copy link
Contributor

fhueske commented May 11, 2018

thanks for the PR @tzulitai.
The new e2e tests passed on my machine. The changes look good as well.
+1 to merge

@tzulitai
Copy link
Contributor Author

Thanks a lot for the review @fhueske.
Will merge this!

asfgit pushed a commit that referenced this pull request May 15, 2018
@asfgit asfgit closed this in 4d640ba May 15, 2018
sampathBhat pushed a commit to sampathBhat/flink that referenced this pull request Jul 26, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants