-
Notifications
You must be signed in to change notification settings - Fork 137
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug] previous data is not deleted when re-register shuffle after re-assignment for stage recomputing #1579
Closed
3 tasks done
Closed
3 tasks done
Milestone
Comments
zuston
added a commit
to zuston/incubator-uniffle
that referenced
this issue
Mar 15, 2024
zuston
added a commit
to zuston/incubator-uniffle
that referenced
this issue
Mar 15, 2024
zuston
added a commit
to zuston/incubator-uniffle
that referenced
this issue
Mar 22, 2024
…ronously for stage resubmit
zuston
added a commit
to zuston/incubator-uniffle
that referenced
this issue
Mar 22, 2024
…ronously for stage resubmit
zuston
added a commit
to zuston/incubator-uniffle
that referenced
this issue
Mar 26, 2024
yl09099
pushed a commit
to zuston/incubator-uniffle
that referenced
this issue
May 12, 2024
…at all previous data is cleared for stage retry
yl09099
pushed a commit
to zuston/incubator-uniffle
that referenced
this issue
May 12, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 12, 2024
…at all previous data is cleared for stage retry
yl09099
pushed a commit
to zuston/incubator-uniffle
that referenced
this issue
May 12, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 13, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 13, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 13, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 13, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 13, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 13, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 14, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 15, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 15, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 15, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 16, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 16, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 16, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 16, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 16, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 17, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 17, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 17, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 18, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 21, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 23, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 24, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 24, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 24, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 24, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 24, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 24, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
May 24, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
Jun 3, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
Jun 3, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
Jun 3, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
Jun 3, 2024
…at all previous data is cleared for stage retry
yl09099
added a commit
to zuston/incubator-uniffle
that referenced
this issue
Jun 3, 2024
yl09099
added a commit
to yl09099/incubator-uniffle
that referenced
this issue
Jun 3, 2024
…at all previous data is cleared for stage retry.
yl09099
added a commit
to yl09099/incubator-uniffle
that referenced
this issue
Jun 3, 2024
…at all previous data is cleared for stage retry.
yl09099
added a commit
to yl09099/incubator-uniffle
that referenced
this issue
Jun 3, 2024
…at all previous data is cleared for stage retry.
yl09099
added a commit
to yl09099/incubator-uniffle
that referenced
this issue
Jun 3, 2024
…at all previous data is cleared for stage retry.
yl09099
added a commit
to yl09099/incubator-uniffle
that referenced
this issue
Jun 4, 2024
… the data in the previous Stage fails to be cleared.
yl09099
added a commit
to yl09099/incubator-uniffle
that referenced
this issue
Jun 4, 2024
… the data in the previous Stage fails to be cleared.
yl09099
added a commit
to yl09099/incubator-uniffle
that referenced
this issue
Jun 5, 2024
yl09099
added a commit
to yl09099/incubator-uniffle
that referenced
this issue
Jun 5, 2024
yl09099
added a commit
to yl09099/incubator-uniffle
that referenced
this issue
Jun 8, 2024
…at all previous data is cleared for stage retry.
yl09099
added a commit
to yl09099/incubator-uniffle
that referenced
this issue
Jun 8, 2024
…at all previous data is cleared for stage retry.
yl09099
added a commit
to yl09099/incubator-uniffle
that referenced
this issue
Jun 12, 2024
…at all previous data is cleared for stage retry.
yl09099
added a commit
to yl09099/incubator-uniffle
that referenced
this issue
Jun 12, 2024
…at all previous data is cleared for stage retry.
zuston
pushed a commit
that referenced
this issue
Jun 13, 2024
…tage retry (#1762) ### What changes were proposed in this pull request? 1. clear out previous stage attempt data synchronously when registering the re-assignment shuffleIds. 2. rework the stage retry interface and rpc 3. introducing the stage version to avoid accepting the older data ### Why are the changes needed? Fix: #1579 If the previous stage attempt is in the purge queue in shuffle-server side, the retry stage writing will cause unknown exceptions, so we'd better to clear out all previous stage attempt data before re-registering This PR is to sync remove previous stage data when the first attempt writer is initialized. Does this PR introduce any user-facing change? No. ### How was this patch tested? Existing tests.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Code of Conduct
Search before asking
Describe the bug
previous data is not deleted when re-register shuffle after re-assignment for stage recomputing
Affects Version(s)
master
Uniffle Server Log Output
No response
Uniffle Engine Log Output
No response
Uniffle Server Configurations
No response
Uniffle Engine Configurations
No response
Additional context
No response
Are you willing to submit PR?
The text was updated successfully, but these errors were encountered: