Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improve][Zeta] when job finished, the checkpoint won't write to file #6674

Merged
merged 3 commits into from
Apr 26, 2024

Conversation

liunaijie
Copy link
Contributor

Purpose of this pull request

close #6670

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

@Hisoka-X
Copy link
Member

Please add test case to verify no file in checkpoint directory when job finished, also need add test case to make sure file would generate when job done with savepoint.

@liunaijie
Copy link
Contributor Author

Please add test case to verify no file in checkpoint directory when job finished, also need add test case to make sure file would generate when job done with savepoint.

Yes, will do it

@liunaijie
Copy link
Contributor Author

Please add test case to verify no file in checkpoint directory when job finished, also need add test case to make sure file would generate when job done with savepoint.

@Hisoka-X PTAL

Hisoka-X
Hisoka-X previously approved these changes Apr 25, 2024
Copy link
Member

@Hisoka-X Hisoka-X left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @liunaijie !

.pipelineId(pipelineId)
.states(states)
.build());
if (completedCheckpoint.getCheckpointType().notCompletedCheckpoint()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add an annotation to this method indicating that it represents a batch type

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the judgment here is for batch processing, but currently the checkpoint type cannot be associated with batch processing

@Hisoka-X Hisoka-X merged commit 7763541 into apache:dev Apr 26, 2024
5 checks passed
@liunaijie liunaijie deleted the zeta-checkpoint branch April 27, 2024 01:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Improve][Zeta] Does not save checkpoint file when job finished.
3 participants