-
Notifications
You must be signed in to change notification settings - Fork 135
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[#1068] improvement(tez): Fail fast in WriteBufferManager when failed to send shuffle data to shuffle sever. #1069
[#1068] improvement(tez): Fail fast in WriteBufferManager when failed to send shuffle data to shuffle sever. #1069
Conversation
Codecov Report
@@ Coverage Diff @@
## master #1069 +/- ##
============================================
+ Coverage 54.20% 55.43% +1.23%
- Complexity 2550 2552 +2
============================================
Files 386 366 -20
Lines 21867 19510 -2357
Branches 1813 1812 -1
============================================
- Hits 11852 10816 -1036
+ Misses 9315 8066 -1249
+ Partials 700 628 -72
... and 20 files with indirect coverage changes 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
Could you correct your title? |
done. |
@zhengchenyu Could you help me review this pr? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. This is meaningful like Spark uniffle client did
LGTM +1 |
Thanks @zhuyaogai and @zhengchenyu review. Merged |
What changes were proposed in this pull request?
Currently, it only checks for blocks that failed to send after all buffer data has been sent. This check also needs to be moved forward into the addRecord method, allowing it to fail fast.
Why are the changes needed?
Fix: # (1068)
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Add more test case in WriteBufferManager.