Skip to content

Commit

Permalink
[DOCUMENTATION] doc fix of job scheduling
Browse files Browse the repository at this point in the history
spark.shuffle.service.enabled is spark application related configuration, it is not necessary to set it in yarn-site.xml

Author: Jeff Zhang <zjffdu@apache.org>

Closes #10657 from zjffdu/doc-fix.
  • Loading branch information
zjffdu authored and Marcelo Vanzin committed Jan 8, 2016
1 parent ea104b8 commit 00d9261
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/job-scheduling.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ pre-packaged distribution.
2. Add this jar to the classpath of all `NodeManager`s in your cluster.
3. In the `yarn-site.xml` on each node, add `spark_shuffle` to `yarn.nodemanager.aux-services`,
then set `yarn.nodemanager.aux-services.spark_shuffle.class` to
`org.apache.spark.network.yarn.YarnShuffleService` and `spark.shuffle.service.enabled` to true.
`org.apache.spark.network.yarn.YarnShuffleService`.
4. Restart all `NodeManager`s in your cluster.

All other relevant configurations are optional and under the `spark.dynamicAllocation.*` and
Expand Down

0 comments on commit 00d9261

Please sign in to comment.