[SPARK-58268][PYTHON] Disable GC for Config gRPC to avoid deadlock#57416
[SPARK-58268][PYTHON] Disable GC for Config gRPC to avoid deadlock#57416gaogaotiantian wants to merge 1 commit into
Conversation
|
Thank you @gaogaotiantian! |
There was a problem hiding this comment.
Hi, @gaogaotiantian .
You cannot make a follow-up for the released JIRA ID because this PR has a different Fix Version (4.2.1) even we land this to branch-4.2.
SPARK-55020 has the Fix Version = 4.2.0.
It seems that you slightly misunderstand the definition of FOLLOWUP in the Apache Spark community. Please use a new JIRA ID for your important PySpark fix.
|
@dongjoon-hyun ah okay. So even it's the same issue, I should create a new JIRA ticket for this PR because it's easier to track for different spark versions? |
|
Yes, correct, it's for trace-ability. For different |
|
For the record, |
|
@dongjoon-hyun got it, thanks! I'll create a new JIRA for this PR. |
|
Thank you so much! |
gaogaotiantian
changed the title
|
@dongjoon-hyun I created a new JIRA and updated the PR title. |
|
I also approved. Feel free to merge~ |
### What changes were proposed in this pull request? Disable GC when we do `_stub.Config()`. ### Why are the changes needed? This is the same thing for #53783 . We need to disable GC when we do gRPC to avoid deadlock. We only recently found this method due to https://github.com/apache/spark/actions/runs/29869961676 . ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? CI. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #57416 from gaogaotiantian/fix-config-deadlock. Authored-by: Tian Gao <gaogaotiantian@hotmail.com> Signed-off-by: Tian Gao <gaogaotiantian@hotmail.com> (cherry picked from commit 377a00e) Signed-off-by: Tian Gao <gaogaotiantian@hotmail.com>
|
Thank you all! |
4 participants
What changes were proposed in this pull request?
Disable GC when we do
_stub.Config().Why are the changes needed?
This is the same thing for #53783 . We need to disable GC when we do gRPC to avoid deadlock. We only recently found this method due to https://github.com/apache/spark/actions/runs/29869961676 .
Does this PR introduce any user-facing change?
No.
How was this patch tested?
CI.
Was this patch authored or co-authored using generative AI tooling?
No.