From 6ce0179b13dc1b3c0db56c00b43580e9fba3738e Mon Sep 17 00:00:00 2001 From: Tom Graves Date: Wed, 20 Jul 2016 12:57:25 -0500 Subject: [PATCH 1/2] [SPARK-16650] Improve documentation of spark.task.maxFailures --- docs/configuration.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/configuration.md b/docs/configuration.md index 1e95b862441f5..33b6f12fcfe20 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -1188,7 +1188,9 @@ Apart from these, the following properties are also available, and may be useful spark.task.maxFailures 4 - Number of individual task failures before giving up on the job. + Number of failures of any particular task before giving up on the job. The same + task has to fail this number of attempts, it isn't affected by the number of failures + across different tasks. Should be greater than or equal to 1. Number of allowed retries = this value - 1. From ffef4abb3a69ef791705f6811fc525a5b5ed8865 Mon Sep 17 00:00:00 2001 From: Tom Graves Date: Wed, 20 Jul 2016 13:58:58 -0500 Subject: [PATCH 2/2] Change wording --- docs/configuration.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/configuration.md b/docs/configuration.md index 33b6f12fcfe20..86a9bd97d3342 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -1188,9 +1188,9 @@ Apart from these, the following properties are also available, and may be useful spark.task.maxFailures 4 - Number of failures of any particular task before giving up on the job. The same - task has to fail this number of attempts, it isn't affected by the number of failures - across different tasks. + Number of failures of any particular task before giving up on the job. + The total number of failures spread across different tasks will not cause the job + to fail; a particular task has to fail this number of attempts. Should be greater than or equal to 1. Number of allowed retries = this value - 1.