[SPARK-4614][MLLIB] Slight API changes in Matrix and Matrices

[mengxr]

Before we have a full picture of the operators we want to add, it might be safer to hide Matrix.transposeMultiply in 1.2.0. Another update we want to change is Matrix.randn and Matrix.rand, both of which should take a Random implementation. Otherwise, it is very likely to produce inconsistent RDDs. I also added some unit tests for matrix factory methods. All APIs are new in 1.2, so there is no incompatible changes.

@brkyvz

[SparkQA]

Test build #23870 has started for PR 3468 at commit 6bfd8a4.

• This patch merges cleanly.

[brkyvz]

Small, but:
mat.values.toSeq === Seq(1.0, 0.0, 0.0, 1.0) -> mat.values === Array(1.0, 0.0, 0.0, 1.0)
is there any reason the other wouldn't work?

[mengxr]

Array.equals only check the reference.

scala> Array(1.0, 2.0) == Array(1.0, 2.0)
res0: Boolean = false

[brkyvz]

Looks good to me! Just made one comment, no biggie though, it's fine as is (but if you decide to change it, there are 4 exact copies of it).
One comment/question about using Random: What would be the problem if we just pass a seed?
Since users wouldn't be able to use XORShiftRandom, which has better performance than
java.util.Random, wouldn't passing the seed be enough to solve the inconsistency issue?

[SparkQA]

Test build #23870 has finished for PR 3468 at commit 6bfd8a4.

• This patch fails MiMa tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23870/
Test FAILed.

[mengxr]

@brkyvz XORShiftRandom implements Random. So users can use it directly. Using seed will create problems when we want to generate a sequence of matrices, e.g., generate a random block matrices, because randn and rand do not return the next seed. If we using a predefined sequence of seeds, the generated sequence won't have good quality.

[SparkQA]

Test build #23884 has started for PR 3468 at commit 3b0e4e2.

• This patch merges cleanly.

[SparkQA]

Test build #23884 has finished for PR 3468 at commit 3b0e4e2.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23884/
Test PASSed.

[mengxr]

@brkyvz Thanks! I've merged this into master. Will submit another PR for branch-1.2.