diff --git a/docs/configuration.md b/docs/configuration.md index e287591f3fda1..0b1a273916314 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -447,9 +447,12 @@ Apart from these, the following properties are also available, and may be useful
spark.shuffle.manager
sort
and hash
. Sort-based shuffle is more memory-efficient and is
- the default option starting in 1.2.
+ Implementation to use for shuffling data. There are three implementations available:
+ sort
, hash
and the new (1.5+) tungsten-sort
.
+ Sort-based shuffle is more memory-efficient and is the default option starting in 1.2.
+ Tungsten-sort is similar to the sort based shuffle, with a direct binary cache-friendly
+ implementation with a fall back to regular sort based shuffle if its requirements are not
+ met.