add some shuffle configurations in doc

[CrazyJvm]

add some shuffle configurations into documentation

[SparkQA]

QA tests have started for PR 2031 at commit 8ad9955.

• This patch merges cleanly.

[SparkQA]

QA tests have started for PR 2031 at commit 876256d.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2031 at commit 8ad9955.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

QA tests have finished for PR 2031 at commit 876256d.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[colorant]

the config value is case irrelevant , if you notice that there is a toLowerCase transform later ;)

[CrazyJvm]

@colorant thanks, I've not noticed toLowerCase before : ) . already modified

[SparkQA]

QA tests have started for PR 2031 at commit 181c512.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2031 at commit 181c512.

• This patch fails unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

QA tests have started for PR 2031 at commit 117e9bf.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2031 at commit 117e9bf.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[mateiz]

Looks pretty cool, though I'll have to try it out. One thing: please add D3's license to our LICENSE and NOTICE files, similar to how we've included Boostrap and jquery.

[SparkQA]

QA tests have started for PR 2031 at commit 117e9bf.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2031 at commit 117e9bf.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[JoshRosen]

/cc @aarondav @andrewor14, is this a configuration that we want to document?

[andrewor14]

Hey @CrazyJvm I believe these options are intentionally left undocumented. These are fairly advanced settings that are meant to provide a backdoor in case the default value turns out to be erroneous, which doesn't seem to be the case. There is also an equivalent spark.storage.safetyFraction that we do not document for the same reason.

I would recommend that we close this issue.