diff --git a/docs/configuration.md b/docs/configuration.md index 29a36bd67f28b..aeb42201c8b77 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -458,9 +458,12 @@ Apart from these, the following properties are also available, and may be useful
spark.shuffle.manager
sort
and hash
. Sort-based shuffle is more memory-efficient and is
- the default option starting in 1.2.
+ Implementation to use for shuffling data. There are three implementations available:
+ sort
, hash
and the new (1.5+) tungsten-sort
.
+ Sort-based shuffle is more memory-efficient and is the default option starting in 1.2.
+ Tungsten-sort is similar to the sort based shuffle, with a direct binary cache-friendly
+ implementation with a fall back to regular sort based shuffle if its requirements are not
+ met.