Skip to content

Commit

Permalink
Document new port configuration options
Browse files Browse the repository at this point in the history
- spark.fileserver.port
- spark.broadcast.port
- spark.replClassServer.port
- spark.blockManager.port
  • Loading branch information
ash211 committed Jul 25, 2014
1 parent 066dc7a commit 5d84e0e
Showing 1 changed file with 30 additions and 11 deletions.
41 changes: 30 additions & 11 deletions docs/spark-standalone.md
Original file line number Diff line number Diff line change
Expand Up @@ -322,7 +322,7 @@ configure those ports.
</tr>
<tr>
<td>Browser</td>
<td>Driver</td>
<td>Application</td>
<td>4040</td>
<td>Web UI</td>
<td><code>spark.ui.port</code></td>
Expand Down Expand Up @@ -372,18 +372,37 @@ configure those ports.

<!-- Other misc stuff -->
<tr>
<td>Driver and other Workers</td>
<td>Worker</td>
<td>Application</td>
<td>(random)</td>
<td>
<ul>
<li>File server for file and jars</li>
<li>Http Broadcast</li>
<li>Class file server (Spark Shell only)</li>
</ul>
</td>
<td>None</td>
<td>Jetty-based. Each of these services starts on a random port that cannot be configured</td>
<td>File server for files and jars</td>
<td><code>spark.fileserver.port</code></td>
<td>Jetty-based</td>
</tr>
<tr>
<td>Worker</td>
<td>Application</td>
<td>(random)</td>
<td>HTTP Broadcast</td>
<td><code>spark.broadcast.port</code></td>
<td>Jetty-based. Not used by TorrentBroadcast, which sends data through the block manager
instead</td>
</tr>
<tr>
<td>Worker</td>
<td>Spark Shell</td>
<td>(random)</td>
<td>Class file server (Spark Shell only)</td>
<td><code>spark.replClassServer.port</code></td>
<td>Jetty-based</td>
</tr>
<tr>
<td>Worker</td>
<td>Other Workers</td>
<td>(random)</td>
<td>Block Manager port</td>
<td><code>spark.blockManager.port</code></td>
<td>Raw socket via ServerSocketChannel</td>
</tr>

</table>
Expand Down

0 comments on commit 5d84e0e

Please sign in to comment.