Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tests for checkpointing #93

Open
kkrugler opened this issue Feb 21, 2018 · 2 comments
Open

Add tests for checkpointing #93

kkrugler opened this issue Feb 21, 2018 · 2 comments

Comments

@kkrugler
Copy link
Member

kkrugler commented Feb 21, 2018

We need to verify that the DomainDB and UrlDB states are checkpointed/savepointed properly.

For checkpointing, we need a test that enables checkpointing (in memory), causes the job to fail, and the verifies that we don't replay everything that we've processed (so checkpointing worked).

For savepointing, I think we need a test that stops a job, then restarts with a different parallelism, and verifies that state is re-distributed properly.

@kkrugler kkrugler added this to the Flink Forward 2018 milestone Feb 21, 2018
@kkrugler kkrugler self-assigned this Feb 22, 2018
@kkrugler kkrugler added this to To do in Flink Forward 2018 Feb 22, 2018
@kkrugler kkrugler moved this from To do to In progress in Flink Forward 2018 Feb 22, 2018
@kkrugler
Copy link
Member Author

kkrugler commented Mar 2, 2018

See https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/testing.html, and the "Testing checkpointing and state handling" section.

@kkrugler
Copy link
Member Author

See also the test_resume_savepoint script in the Flink code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Flink Forward 2018
  
In progress
Development

No branches or pull requests

1 participant