Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] shuffle server may hang when restart worker due to multi require-momery and no require-momery release #1045

Closed
3 tasks done
lifeSo opened this issue Jul 27, 2023 · 0 comments · Fixed by #1058
Closed
3 tasks done

[Bug] shuffle server may hang when restart worker due to multi require-momery and no require-momery release #1045

lifeSo opened this issue Jul 27, 2023 · 0 comments · Fixed by #1058
Assignees
@lifeSo
Labels
0.8 For 0.8 version release

Comments

@lifeSo
Copy link
Collaborator

lifeSo commented Jul 27, 2023

Code of Conduct

Search before asking

  • I have searched in the issues and found no similar issues.

Describe the bug

When Tez Task is doing big shuffle and restart worker, the total-require-memory metric will increase to very high then performance down。

Affects Version(s)

master

Uniffle Server Log Output

No response

Uniffle Engine Log Output

No response

Uniffle Server Configurations

No response

Uniffle Engine Configurations

No response

Additional context

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!
@jerqi jerqi added the 0.8 For 0.8 version release label Jul 29, 2023
@lifeSo lifeSo mentioned this issue Jul 31, 2023
Merged
jerqi pushed a commit that referenced this issue Aug 8, 2023
@lifeSo
[#1045] fix(server): shuffle server may hang when restart worker due …
ca24531 
…to multi require-momery and no require-momery release (#1058)


### What changes were proposed in this pull request?

shuffle server may hang when restart worker due to multi require-momery and no require-momery release.
So, fix the bug

### Why are the changes needed?

shuffle server may hang when restart worker due to multi require-momery and no require-momery release.
So, fix the bug

Fix:  #1045

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?
unit test
This was referenced Nov 20, 2023
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lifeSo lifeSo

Labels
0.8 For 0.8 version release
Projects
None yet
Milestone
No milestone
2 participants
@jerqi @lifeSo