New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get rid of random checkpoint name #1

Merged
merged 1 commit into from Jun 21, 2018

Conversation

Projects
None yet
1 participant
@gaohao
Collaborator

gaohao commented Jun 20, 2018

Flink's use a random name for its externalized checkpoint. Currently the checkpoint under the checkpoint root directory looks like:
checkpoint_metadata-2d00588c6398
It makes it very hard to distinguish which file belongs to which flink application.
It also make it impossible to figure out which file belong to which after the flink cluster is destroyed.

We are going to make the following changes,

  1. each of the flink apps can choose to have a dedicated sub folder for its checkpoint
  2. each checkpoint will have a timestamp attach to it
  3. user need to guarantee the sub folder's name is unique for their flink app

More reading if anyone interested:
https://medium.com/hadoop-noob/flink-externalized-checkpoint-eb86e693cfed

@gaohao gaohao merged commit 68a520b into wish-1.3.3 Jun 21, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment