Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-4023] [MLlib] [PySpark] convert rdd into RDD of Vector #2870

Closed
wants to merge 2 commits into from
Closed

[SPARK-4023] [MLlib] [PySpark] convert rdd into RDD of Vector #2870

wants to merge 2 commits into from

Conversation

davies
Copy link
Contributor

@davies davies commented Oct 21, 2014

Convert the input rdd to RDD of Vector.

cc @mengxr

@SparkQA
Copy link

SparkQA commented Oct 21, 2014

QA tests have started for PR 2870 at commit 0871576.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Oct 21, 2014

QA tests have finished for PR 2870 at commit 0871576.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21961/
Test PASSed.

mengxr
mengxr reviewed Oct 21, 2014
View reviewed changes
python/pyspark/mllib/tests.py
summary = Statistics.colStats(data)
self.assertEqual(1000, summary.count())
mean = summary.mean()
self.assertTrue(all(abs(v) < 0.1 for v in mean))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a non-deterministic test. For SPARK-4023, we only need to test colStats and other methods for RDDs of numpy arrays and python arrays.

@SparkQA
Copy link

SparkQA commented Oct 21, 2014

QA tests have started for PR 2870 at commit 1eac767.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Oct 21, 2014

QA tests have finished for PR 2870 at commit 1eac767.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21977/
Test PASSed.

@mengxr
Copy link
Contributor

mengxr commented Oct 21, 2014

LGTM. Merged into master. Thanks!

@asfgit asfgit closed this in 8570816 Oct 21, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@davies @SparkQA @AmplabJenkins @mengxr