SPARK-10759 [MLlib] Add python example to model selection via cross-validation

[JeremyNixon]

Python example in the same form as the Scala and Java examples. Successfully rendered on my local server and run through pyspark, results match those from the Scala example.

[sethah]

One question is whether we should be using {% include_example %} when adding new examples to the documentation. We could separate it out into different PRs, but then we are duplicating code (this example already exists here).

@yinxusen could you advise?

[yinxusen]

It's better to reuse current cross validator with {% include_example %} other than writing it directly in markdown file.

[yinxusen]

Refer to #11126 and JIRA https://issues.apache.org/jira/browse/SPARK-11337

[JeremyNixon]

yinxusen, I have a PR here (#11240) with the solution to just this example that takes advantage of {% include_example %}, though accepting it would make this example inconsistent with the rest of the examples in this doc.

I'm also going to go ahead and create a branch with all of the examples fixed with {% include_example %} to make them consistent with one another and take advantage of the automatic testing and dryer code. Would you prefer that I submit that PR here, or create a new JIRA stating that all are fixed?

[JeremyNixon]

yinxusen, I went ahead and changed all of the examples in this doc to use {% include_example %} as well as updated the python cross validation example. Let me know if it's acceptable to make all of the changes within this JIRA, or if you'd prefer they be separated.

I also notice that a JIRA exists (https://issues.apache.org/jira/browse/SPARK-13012) for converting the original examples to use {% include_example %}. The PR makes the same changes that are here, except does not include the change to the python example for cross-validation that this JIRA is supposed to be for.

[mengxr]

ok to test

[SparkQA]

Test build #51791 has finished for PR 11202 at commit 14208ad.

• This patch fails Scala style tests.
• This patch does not merge cleanly.
• This patch adds no public classes.

[yinxusen]

@JeremyNixon Could you merge the PR with the master?

[JeremyNixon]

@yinxusen Will do. But I assume you mean change the commits to make this consistent with https://issues.apache.org/jira/browse/SPARK-13012 which was merged in yesterday?

[yinxusen]

Yes. Can we close the issue? Since you have #11240

[JeremyNixon]

@yinxusen Two things -

I finished an implementation of train-validation-split so that we could finish the examples for this doc here: master...JeremyNixon:train_val_split_python, not sure if it belongs in this PR or should be done separately. #11240 requires one small update, which I can push now.

[yinxusen]

@JeremyNixon Give a PR under JIRA https://issues.apache.org/jira/browse/SPARK-12877 of the train-validation-split in PySpark. You can add the train-validation-split along with that JIRA. I think we can merge #11240 first.

[MLnick]

@JeremyNixon is this PR still alive?

[JeremyNixon]

@MLnick Dead PR, work completed with a combination of 230bbea and 02b1fef. Thanks for the heads up.