Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SPARK-48218][CORE] TransportClientFactory.createClient may NPE cause…
… FetchFailedException ### What changes were proposed in this pull request? This PR aims to add a check for `TransportChannelHandler` to be non-null in the `TransportClientFactory.createClient` method. ### Why are the changes needed? Line 178 synchronized (handler) , handler == null org.apache.spark.network.client.TransportClientFactory#createClient(java.lang.String, int, boolean) ```java TransportChannelHandler handler = cachedClient.getChannel().pipeline() .get(TransportChannelHandler.class); synchronized (handler) { handler.getResponseHandler().updateTimeOfLastRequest(); } ``` ```java org.apache.spark.shuffle.FetchFailedException at org.apache.spark.storage.ShuffleBlockFetcherIterator.throwFetchFailedException(ShuffleBlockFetcherIterator.scala:1180) at org.apache.spark.storage.ShuffleBlockFetcherIterator.next(ShuffleBlockFetcherIterator.scala:913) at org.apache.spark.storage.ShuffleBlockFetcherIterator.next(ShuffleBlockFetcherIterator.scala:84) at org.apache.spark.util.CompletionIterator.next(CompletionIterator.scala:29) Caused by: java.lang.NullPointerException at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:178) at org.apache.spark.network.shuffle.ExternalBlockStoreClient.lambda$fetchBlocks$0(ExternalBlockStoreClient.java:128) at org.apache.spark.network.shuffle.RetryingBlockTransferor.transferAllOutstanding(RetryingBlockTransferor.java:154) at org.apache.spark.network.shuffle.RetryingBlockTransferor.start(RetryingBlockTransferor.java:133) at org.apache.spark.network.shuffle.ExternalBlockStoreClient.fetchBlocks(ExternalBlockStoreClient.java:139) ``` ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? ### Was this patch authored or co-authored using generative AI tooling? No Closes #46506 from cxzl25/SPARK-48218. Authored-by: sychen <sychen@ctrip.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
- Loading branch information