diff --git a/docs/configuration.md b/docs/configuration.md index 500f980455b0e..011d583d6ea70 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -536,15 +536,17 @@ Apart from these, the following properties are also available, and may be useful - spark.reducer.maxBlocksInFlightPerAddress - Int.MaxValue - - This configuration limits the number of remote blocks being fetched per reduce task from a - given host port. When a large number of blocks are being requested from a given address in a - single fetch or simultaneously, this could crash the serving executor or Node Manager. This - is especially useful to reduce the load on the Node Manager when external shuffle is enabled. - You can mitigate this issue by setting it to a lower value. - + spark.reducer.maxBlocksInFlightPerAddress + Int.MaxValue + + This configuration limits the number of remote blocks being fetched per reduce task from a + given host port. When a large number of blocks are being requested from a given address in a + single fetch or simultaneously, this could crash the serving executor or Node Manager. This + is especially useful to reduce the load on the Node Manager when external shuffle is enabled. + You can mitigate this issue by setting it to a lower value. + + + spark.reducer.maxReqSizeShuffleToMem Long.MaxValue @@ -1081,7 +1083,7 @@ Apart from these, the following properties are also available, and may be useful - spark.storage.replication.proactive + spark.storage.replication.proactive false Enables proactive block replication for RDD blocks. Cached RDD block replicas lost due to