Skip to content

Conversation

@mengxr
Copy link
Contributor

@mengxr mengxr commented May 11, 2015

@nchammas Just found that we don't need to patch Pyrolite because useMemo is exposed in the constructor in Pyrolite 4.4. So the changes in the PR should be sufficient to see the erformance difference caused by useMemo.

@AmplabJenkins
Copy link

Merged build triggered.

@AmplabJenkins
Copy link

Merged build started.

@SparkQA
Copy link

SparkQA commented May 11, 2015

Test build #32431 has started for PR 6065 at commit c8e69c1.

@SparkQA
Copy link

SparkQA commented May 12, 2015

Test build #32431 has finished for PR 6065 at commit c8e69c1.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Merged build finished. Test FAILed.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32431/
Test FAILed.

@AmplabJenkins
Copy link

Merged build triggered.

@AmplabJenkins
Copy link

Merged build started.

@SparkQA
Copy link

SparkQA commented May 12, 2015

Test build #32546 has started for PR 6065 at commit c721a23.

@SparkQA
Copy link

SparkQA commented May 13, 2015

Test build #32546 has finished for PR 6065 at commit c721a23.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Merged build finished. Test FAILed.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32546/
Test FAILed.

@nchammas
Copy link
Contributor

@mengxr I'm gonna start PySpark performance tests for before/after this PR now.

@mengxr
Copy link
Contributor Author

mengxr commented May 15, 2015

I'm closing this PR after discussing with @davies. useMemo is useful to avoid repeatedly serializing class objects. However, other than this case, it might bring overhead for Spark use cases. @davies is going to test a hybrid version that only memorizes class instances.

@mengxr mengxr closed this May 15, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants