Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-10185] Make ZooKeeperStateHandleStore#releaseAndTryRemove synchronous #6586

Conversation

tillrohrmann
Copy link
Contributor

What is the purpose of the change

Remove the asynchronous callback from ZooKeeperStateHandleStore#releaseAndTryRemove.
Instead we can execute the callback after having executed the releaseAndTryRemove
method successfully. This separates concerns better because we don't mix storage
with business logic. Furthermore, we can still avoid blocking operations if we use a
separate thread to call into ZooKeeperStateHandleStore#releaseAndTryRemove.

Brief change log

  • Make ZooKeeperStateHandleStore#releaseAndTryRemove synchronous
  • Introduce ZooKeeperResource which starts a ZooKeeper server

Verifying this change

  • Changes is covered by ZooKeeperStateHandleStoreTest#testRemove and ZooKeeperCompletedCheckpointStoreTest#testDiscardingSubsumedCheckpoints and ZooKeeperCompletedCheckpointStoreTest#testDiscardingCheckpointsAtShutDown

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
  • The serializers: (no)
  • The runtime per-record code paths (performance sensitive): (no)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes)
  • The S3 file system connector: (no)

Documentation

  • Does this pull request introduce a new feature? (no)
  • If yes, how is the feature documented? (not applicable)

…hronous

Remove the asynchronous callback from ZooKeeperStateHandleStore#releaseAndTryRemove.
Instead we can execute the callback after having executed the releaseAndTryRemove
method successfully. This separates concerns better because we don't mix storage
with business logic. Furthermore, we can still avoid blocking operations if we use a
separate thread to call into ZooKeeperStateHandleStore#releaseAndTryRemove.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants