Skip to content

Commit

Permalink
Updated a few lines in the Launching Spark on YARN Section
Browse files Browse the repository at this point in the history
  • Loading branch information
Neelesh Srinivas Salian committed Jun 21, 2015
1 parent 8e8db7f commit 9cbc072
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions docs/running-on-yarn.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,10 @@ Java system properties or environment variables not managed by YARN, they should
Spark application's configuration (driver, executors, and the AM when running in client mode).

There are two deploy modes that can be used to launch Spark applications on YARN. In yarn-cluster mode, the Spark driver runs inside an application master process which is managed by YARN on the cluster, and the client can go away after initiating the application. In yarn-client mode, the driver runs in the client process, and the application master is only used for requesting resources from YARN.
(Default: --deploy-mode client)

Unlike in Spark standalone and Mesos mode, in which the master's address is specified in the "master" parameter, in YARN mode the ResourceManager's address is picked up from the Hadoop configuration. Thus, the master parameter is yarn.

Unlike in Spark standalone and Mesos mode, in which the master's address is specified in the "master" parameter, in YARN mode the ResourceManager's address is picked up from the Hadoop configuration. Thus, the master parameter is simply “yarn-client” or “yarn-cluster”.

To launch a Spark application in yarn-cluster mode:

./bin/spark-submit --class path.to.your.Class --master yarn-cluster [options] <app jar> [app options]
Expand Down

0 comments on commit 9cbc072

Please sign in to comment.