From a4b2e27b0c2d2345a60ba66943b219968465b48a Mon Sep 17 00:00:00 2001 From: CrazyJvm Date: Wed, 2 Jul 2014 14:59:48 +0800 Subject: [PATCH 1/6] fix configuration spark.yarn.max.executor.failures --- docs/running-on-yarn.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index 5d8d603aa3e37..fc5c58ec9b34b 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -55,8 +55,11 @@ Most of the configs are the same for Spark on YARN as for other deployment modes spark.yarn.max.executor.failures - 2*numExecutors + 2*numExecutors or 3, whichever is larger + + + It's the same to `spark.yarn.max.worker.failures`, The maximum number of executor failures before failing the application. From 2900d234c6ebb90a5c4601083ddf8d329a2ee99d Mon Sep 17 00:00:00 2001 From: CrazyJvm Date: Wed, 2 Jul 2014 15:04:51 +0800 Subject: [PATCH 2/6] fix style --- docs/running-on-yarn.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index fc5c58ec9b34b..4a2a163e1d0d3 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -60,7 +60,7 @@ Most of the configs are the same for Spark on YARN as for other deployment modes It's the same to `spark.yarn.max.worker.failures`, - The maximum number of executor failures before failing the application. + the maximum number of executor failures before failing the application. From 211f1302aa6d57b07a7b2d3b7cd4ab21e6d50bbd Mon Sep 17 00:00:00 2001 From: CrazyJvm Date: Wed, 2 Jul 2014 15:06:28 +0800 Subject: [PATCH 3/6] fix html tag --- docs/running-on-yarn.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index 4a2a163e1d0d3..39a3cbf575b7c 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -59,7 +59,7 @@ Most of the configs are the same for Spark on YARN as for other deployment modes 2*numExecutors or 3, whichever is larger - It's the same to `spark.yarn.max.worker.failures`, + It's the same to spark.yarn.max.worker.failures, the maximum number of executor failures before failing the application. From 86effa612d2ec9ae991b43e229c4ed266e6605a6 Mon Sep 17 00:00:00 2001 From: CrazyJvm Date: Wed, 2 Jul 2014 15:15:08 +0800 Subject: [PATCH 4/6] change expression --- docs/running-on-yarn.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index 39a3cbf575b7c..ad7a171b86c3c 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -56,7 +56,7 @@ Most of the configs are the same for Spark on YARN as for other deployment modes spark.yarn.max.executor.failures - 2*numExecutors or 3, whichever is larger + numExecutors * 2, with minimum of 3 It's the same to spark.yarn.max.worker.failures, From c438aecdec8ce90cb839b7c9aa8260ff4d3c62ba Mon Sep 17 00:00:00 2001 From: CrazyJvm Date: Wed, 2 Jul 2014 15:18:18 +0800 Subject: [PATCH 5/6] fix style --- docs/running-on-yarn.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index ad7a171b86c3c..0da705d144731 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -55,9 +55,7 @@ Most of the configs are the same for Spark on YARN as for other deployment modes spark.yarn.max.executor.failures - - numExecutors * 2, with minimum of 3 - + numExecutors * 2, with minimum of 3 It's the same to spark.yarn.max.worker.failures, the maximum number of executor failures before failing the application. From 1a5f25b66b18c45870effebd182735181d906cab Mon Sep 17 00:00:00 2001 From: CrazyJvm Date: Tue, 8 Jul 2014 13:35:53 +0800 Subject: [PATCH 6/6] remove deprecated config --- docs/running-on-yarn.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index 0da705d144731..0362f5a223319 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -57,8 +57,7 @@ Most of the configs are the same for Spark on YARN as for other deployment modes spark.yarn.max.executor.failures numExecutors * 2, with minimum of 3 - It's the same to spark.yarn.max.worker.failures, - the maximum number of executor failures before failing the application. + The maximum number of executor failures before failing the application.