From 5fa9c46967b5427bd586403b2fd935f80f29ec78 Mon Sep 17 00:00:00 2001 From: WangTaoTheTonic Date: Tue, 3 Feb 2015 17:32:14 +0800 Subject: [PATCH 1/2] add doc for spark.executor.instances --- docs/running-on-yarn.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index 68ab127bcf087..67521aab01ef4 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -104,6 +104,13 @@ Most of the configs are the same for Spark on YARN as for other deployment modes Comma-separated list of files to be placed in the working directory of each executor. + + spark.executor.instances + 2 + + The number of executors. + + spark.yarn.executor.memoryOverhead executorMemory * 0.07, with minimum of 384 From 4c3913add23b39e4c5a5120d8a56917f972e1b4b Mon Sep 17 00:00:00 2001 From: WangTaoTheTonic Date: Wed, 4 Feb 2015 09:19:00 +0800 Subject: [PATCH 2/2] not compatible with dynamic allocation --- docs/running-on-yarn.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index 67521aab01ef4..431a240d19f85 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -108,7 +108,7 @@ Most of the configs are the same for Spark on YARN as for other deployment modes spark.executor.instances 2 - The number of executors. + The number of executors. Don't set this when dynamic allocation is enabled as they are not compatible.