Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#669] improvement: refresh application when reading memory data #741

Merged
merged 3 commits into from
Mar 19, 2023

Conversation

xianjingfeng
Copy link
Member

What changes were proposed in this pull request?

Refresh application when reading memory data.

Why are the changes needed?

In the current logic, application will not be refreshed when reading memory data. In this case, If timeout occurs when spark driver sends heartbeat to shuffle server, the application will be expired.

Fix: #669

Does this PR introduce any user-facing change?

No.

How was this patch tested?

No need,

@codecov-commenter
Copy link

codecov-commenter commented Mar 19, 2023

Codecov Report

Merging #741 (0507d7a) into master (e99eade) will increase coverage by 2.13%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##             master     #741      +/-   ##
============================================
+ Coverage     60.67%   62.81%   +2.13%     
  Complexity     1895     1895              
============================================
  Files           238      224      -14     
  Lines         12999    11036    -1963     
  Branches       1090     1090              
============================================
- Hits           7887     6932     -955     
+ Misses         4676     3728     -948     
+ Partials        436      376      -60     
Impacted Files Coverage Δ
.../org/apache/uniffle/server/ShuffleTaskManager.java 76.40% <0.00%> (-0.23%) ⬇️

... and 14 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Contributor

@jerqi jerqi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, merged to Master & 0.7

@jerqi jerqi merged commit d60d675 into apache:master Mar 19, 2023
jerqi pushed a commit that referenced this pull request Mar 19, 2023
### What changes were proposed in this pull request?

Refresh application when reading memory data.

### Why are the changes needed?

In the current logic, application will not be refreshed when reading memory data. In this case, If timeout occurs when spark driver sends heartbeat to shuffle server, the application will be expired.

Fix: #669 

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

No need,
advancedxy pushed a commit to advancedxy/incubator-uniffle that referenced this pull request Mar 21, 2023
apache#741)

### What changes were proposed in this pull request?

Refresh application when reading memory data.

### Why are the changes needed?

In the current logic, application will not be refreshed when reading memory data. In this case, If timeout occurs when spark driver sends heartbeat to shuffle server, the application will be expired.

Fix: apache#669 

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

No need,
xianjingfeng added a commit to xianjingfeng/incubator-uniffle that referenced this pull request Apr 5, 2023
apache#741)

### What changes were proposed in this pull request?

Refresh application when reading memory data.

### Why are the changes needed?

In the current logic, application will not be refreshed when reading memory data. In this case, If timeout occurs when spark driver sends heartbeat to shuffle server, the application will be expired.

Fix: apache#669 

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

No need,
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Improvement] Refresh application when reading memory data
3 participants