[SPARK-9911] [DOC] [ML] Update Userguide for Evaluator

[MechCoder]

I added a small note about the different types of evaluator and the metrics used.

[MechCoder]

@mengxr

I thought it is unnecessary to add a separate guide for evaluators and hence added a note within the existing example.

[SparkQA]

Test build #41221 has finished for PR 8304 at commit 4a4d7a6.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[feynmanliang]

nit: Oxford comma ("...binary data, or a...")

[feynmanliang]

+1 for @mengxr 's suggestion on JIRA to break this example into a separate ml-evaluation.md section; the examples in ml-guide are becoming monolithic and breaking another section will allow us to better document the features available in evaluators (e.g. all the metrics supported, whether to maximize or minimize the metric a la #8290)

[MechCoder]

That sounds like a great idea. But do you have better suggestions for an example? I would have to use a cross-validator and tune it anyway and I'm afraid there will be code repetition.

[feynmanliang]

I don't think we need code examples for every possible combination, otherwise combinatorial complexity is going to bite us hard.

We can:

• List all the evaluators and their features
• List the (currently two?) validators, CV and TrainTestSplit, and their features
• Provide the current code example going through with the binary classification metric

[MechCoder]

otherwise combinatorial complexity is going to bite us hard.

lol. That is not what I had meant. Sorry I should be more clear from here on.

In other words I was asking if it would be sufficient to move the Model Selection Example to the new ml-evalation.md or if you had anything else in mind?

[feynmanliang]

Oh, sorry!

I think moving the example and providing some guidance about how to choose evaluators/validators is sufficient for now.

[MechCoder]

ping @jkbradley ?

[jkbradley]

+1 for "moving the example and providing some guidance about how to choose evaluators/validators"

Something simple, but separate, which we can build upon later on.

[mengxr]

@MechCoder is busy this week. I will make a PR based on this.

[mengxr]

Since all comments are minor (or beyond this PR), I'm going to merge this into master and branch-1.5 first and then send another PR to fix issues here and some others.