-
Notifications
You must be signed in to change notification settings - Fork 28.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Revert "[SPARK-35011][CORE] Avoid Block Manager registrations when StopExecutor msg is in-flight" #33942
Conversation
…opExecutor msg is in-flight" This reverts commit b9e53f8.
The original PR has been merged into other branches(3.2, 3.1, 3.0) We should revert them as well. |
Kubernetes integration test starting |
Kubernetes integration test status failure |
Test build #143105 has finished for PR 33942 at commit
|
Thank you for pinging me, @Ngone51 . |
@dongjoon-hyun, you had asked about adding a test case to check for this (I think I saw that in the mail, but cant find that comment here) - that is an excellent idea, and we should test for it. |
Ya, I wrote it and took back my comment, @mridulm . :) |
Could you make reverting PRs to the old branches please, @Ngone51 ? We had better pass CIs at those branches. |
Sure, thanks! |
This reverts commit b9e53f8.
What changes were proposed in this pull request?
Revert #32114
Why are the changes needed?
It breaks the expected
BlockManager
re-registration (e.g., heartbeat loss of an active executor) due to deferred removal ofBlockManager
, see the check:spark/core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala
Line 551 in 9cefde8
Does this PR introduce any user-facing change?
No
How was this patch tested?
Pass existing tests.