[SPARK-25504][Docs] Update doc about retained tasks, jobs and executions

[shahidki31]

What changes were proposed in this pull request?

Updated the documentation about spark.ui.retainedTasks, spark.ui.retainedJobs, spark.ui.retainedStages

How was this patch tested?

NA

[shahidki31]

Hi @srowen , Some users seems getting confused with the threshold values, is it worth to update the docs? Thanks

[SparkQA]

Test build #99240 has finished for PR 23134 at commit ca62b7e.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #99241 has finished for PR 23134 at commit 44d2c58.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[srowen]

I am not sure it clarifies. Soft threshold implies it can retain more than the max. I also don't think we want to say it's definitely at least 90%. What is an example of this creating confusion?

[shahidki31]

In the JIRA, when the user want retainedTasks as 60000, actually retained around 58000.
As per the code, it retained atleast 0.9*retainedTasks to retainedTasks. if we can clearly specify about the threshold, the confusion doesn't arises.
If it is okay with the documentation, I can close the PR.

[srowen]

I think the existing docs are fine.