New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-6390] [checkpoints] Add API for checkpoints that are triggered via external systems #3782

Merged
merged 1 commit into from Apr 26, 2017

Conversation

Projects
None yet
3 participants
@StephanEwen
Contributor

StephanEwen commented Apr 26, 2017

Some source systems require to be notified prior to starting a checkpoint, in order to do preparatory work for the checkpoint.

This PR adds an interface to allow sources to register hooks that are called by the checkpoint coordinator when triggering / restoring a checkpoint. These hooks may produce state that is stores with the checkpoint metadata.

Because this changes the checkpoint metadata format, the commit introduces a new metadata format version.

@tillrohrmann

Changes look really good. Great work @StephanEwen. +1 for merging.

@StephanEwen

This comment has been minimized.

Show comment
Hide comment
@StephanEwen

StephanEwen Apr 26, 2017

Contributor

Thanks a lot for the fast review. Agree with both issues raised. Will address them while merging...

Contributor

StephanEwen commented Apr 26, 2017

Thanks a lot for the fast review. Agree with both issues raised. Will address them while merging...

StephanEwen added a commit to StephanEwen/incubator-flink that referenced this pull request Apr 26, 2017

[FLINK-6390] [checkpoints] Add API for checkpoints that are triggered…
… via external systems

This includes
  - A interface for hooks that are called by the checkpoint coordinator to trigger/restore a checkpoint
  - A source extension that triggers the operator checkpoints and barrier injection on certain events

Because this changes the checkpoint metadata format, the commit introduces a new metadata format version.

This closes apache#3782

StephanEwen added a commit to StephanEwen/incubator-flink that referenced this pull request Apr 26, 2017

[FLINK-6390] [checkpoints] Add API for checkpoints that are triggered…
… via external systems

This includes
  - A interface for hooks that are called by the checkpoint coordinator to trigger/restore a checkpoint
  - A source extension that triggers the operator checkpoints and barrier injection on certain events

Because this changes the checkpoint metadata format, the commit introduces a new metadata format version.

This closes apache#3782
[FLINK-6390] [checkpoints] Add API for checkpoints that are triggered…
… via external systems

This includes
  - A interface for hooks that are called by the checkpoint coordinator to trigger/restore a checkpoint
  - A source extension that triggers the operator checkpoints and barrier injection on certain events

Because this changes the checkpoint metadata format, the commit introduces a new metadata format version.

This closes #3782

@asfgit asfgit merged commit 90ca438 into apache:master Apr 26, 2017

0 of 2 checks passed

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
continuous-integration/travis-ci/push The Travis CI build is in progress
Details

PangZhi added a commit to PangZhi/flink that referenced this pull request May 1, 2017

[FLINK-6390] [checkpoints] Add API for checkpoints that are triggered…
… via external systems

This includes
  - A interface for hooks that are called by the checkpoint coordinator to trigger/restore a checkpoint
  - A source extension that triggers the operator checkpoints and barrier injection on certain events

Because this changes the checkpoint metadata format, the commit introduces a new metadata format version.

This closes apache#3782

fanyon pushed a commit to fanyon/flink that referenced this pull request May 11, 2017

[FLINK-6390] [checkpoints] Add API for checkpoints that are triggered…
… via external systems

This includes
  - A interface for hooks that are called by the checkpoint coordinator to trigger/restore a checkpoint
  - A source extension that triggers the operator checkpoints and barrier injection on certain events

Because this changes the checkpoint metadata format, the commit introduces a new metadata format version.

This closes apache#3782
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment