Skip to content

Commit

Permalink
Executors connect to wrong port when collision occurs
Browse files Browse the repository at this point in the history
This commit fixes this by setting "spark.driver.port" to the actual
port the akka system bound to. Note that we already do this for port
0, where the original port is not the same as the bound port. However,
we still face the same issue if we end up using port n + 1.
  • Loading branch information
andrewor14 committed Aug 5, 2014
1 parent d502e5f commit 93d359f
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions core/src/main/scala/org/apache/spark/SparkEnv.scala
Original file line number Diff line number Diff line change
Expand Up @@ -150,10 +150,10 @@ object SparkEnv extends Logging {
val (actorSystem, boundPort) = AkkaUtils.createActorSystem("spark", hostname, port, conf = conf,
securityManager = securityManager)

// Bit of a hack: If this is the driver and our port was 0 (meaning bind to any free port),
// figure out which port number Akka actually bound to and set spark.driver.port to it.
if (isDriver && port == 0) {
conf.set("spark.driver.port", boundPort.toString)
// Figure out which port Akka actually bound to in case the original port is 0 or occupied.
// This is so that we tell the executors the correct port to connect to.
if (isDriver) {
conf.set("spark.driver.port", boundPort.toString)
}

// Create an instance of the class named by the given Java system property, or by
Expand Down

0 comments on commit 93d359f

Please sign in to comment.