[#1735] fix(client): MultiException class not found when reassign or stage retry is enabled #1725
Conversation
|
@dingshun3016 Could you create an issue for this? |
|
Why we need jetty package in client side? @dingshun3016 |
dingshun3016
changed the title
|
You can take a look at the stack information above, mainly using the MultiException class in jetty to determine whether it is a port conflict exception. |
dingshun3016
force-pushed
the
shade-jetty
branch
from
bd45f9f
to
89ab7f2
Compare
|
I moved MultiException from the jetty package to uniffle and renamed it to RssMultiException |
After consideration, it seems that MultiException cannot be ported to uniffle, because the exception thrown is |
|
I think we could remove this. @dingshun3016 |
Without using the |
rickyma
changed the title
|
@dingshun3016 Thanks for your contributions. And I have another fix seems can resolve more conflict and CNFE, PTAL |
What changes were proposed in this pull request?
include jetty-util and shade it
Why are the changes needed?
Fix: #1735
When reassign or stag retry is enabled, GrpcServer needs to be started. If there is a port conflict during startup and retry is performed, MultiException will be thrown and the following exception will occur, causing the job to fail.The root cause is that uniffle did not include jetty-util related packages.
ERROR [main] SparkSubmit$$anon$2: spark submit throw error: org/eclipse/jetty/util/MultiException java.lang.NoClassDefFoundError: org/eclipse/jetty/util/MultiException at org.apache.uniffle.common.util.RssUtils.isServerPortBindCollision(RssUtils.java:219) at org.apache.uniffle.common.util.RssUtils.startServiceOnPort(RssUtils.java:197) at org.apache.uniffle.common.rpc.GrpcServer.start(GrpcServer.java:188) at org.apache.spark.shuffle.RssShuffleManager.<init>(RssShuffleManager.java:262) at org.apache.spark.shuffle.DelegationRssShuffleManager.createShuffleManagerInDriver(DelegationRssShuffleManager.java:87) at org.apache.spark.shuffle.DelegationRssShuffleManager.<init>(DelegationRssShuffleManager.java:60) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423)Does this PR introduce any user-facing change?
No.
How was this patch tested?
No need.