From 1473f877523116647c5cd1f07e97390b53bccd46 Mon Sep 17 00:00:00 2001 From: keliang Date: Wed, 3 Aug 2016 16:43:52 +0800 Subject: [PATCH 1/2] Summary:add "spark.sql.broadcastTimeout" into docs/sql-programming-guide.md JIRA_ID:SPARK-16870 Description:default value for spark.sql.broadcastTimeout is 300s. and this property do not show in any docs of spark. so add "spark.sql.broadcastTimeout" into docs/sql-programming-guide.md to help people to how to fix this timeout error when it happenned Test:done --- docs/sql-programming-guide.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md index d8c8698e31d39..5eace815969fb 100644 --- a/docs/sql-programming-guide.md +++ b/docs/sql-programming-guide.md @@ -790,6 +790,15 @@ Configuration of Parquet can be done using the `setConf` method on `SparkSession

+ + spark.sql.broadcastTimeout + 300 + +

+ Timeout in seconds for the broadcast wait time in broadcast joins +

+ + ## JSON Datasets From 52911bf0f288270c11ba7d59ff70c459f7eabb4e Mon Sep 17 00:00:00 2001 From: keliang Date: Sat, 6 Aug 2016 16:04:56 +0800 Subject: [PATCH 2/2] Summary:Move it to broadcast join secsion JIRA_ID:SPARK-16870 Description:Move "spark.sql.broadcastTimeout" to broadcast join secsion Test:done --- docs/sql-programming-guide.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md index 5eace815969fb..da292bdc13313 100644 --- a/docs/sql-programming-guide.md +++ b/docs/sql-programming-guide.md @@ -790,15 +790,6 @@ Configuration of Parquet can be done using the `setConf` method on `SparkSession

- - spark.sql.broadcastTimeout - 300 - -

- Timeout in seconds for the broadcast wait time in broadcast joins -

- - ## JSON Datasets @@ -1196,6 +1187,15 @@ that these options will be deprecated in future release as more optimizations ar scheduled first). + + spark.sql.broadcastTimeout + 300 + +

+ Timeout in seconds for the broadcast wait time in broadcast joins +

+ + spark.sql.autoBroadcastJoinThreshold 10485760 (10 MB)