Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minor change which fixes some problems with running dill under doctest #29

Merged
merged 1 commit into from
Apr 1, 2014

Conversation

matsjoyce
Copy link
Contributor

Possible fix for #18. Can @JoshRosen see if this solves the test suite problems?

mmckerns added a commit that referenced this pull request Apr 1, 2014
Minor change which fixes some problems with running dill under doctest
@mmckerns mmckerns merged commit cca07cb into uqfoundation:master Apr 1, 2014
@matsjoyce matsjoyce deleted the doctest_fixes branch April 1, 2014 17:35
@nchammas
Copy link

@JoshRosen @davies Are we still planning to replace cloudpickle with dill? Looks like it's ready for another look.

@davies
Copy link

davies commented Sep 14, 2014

Right now, is there any problem which cloudpickle have? I does not like cloudpickle either, but if we make this change, maybe will meet some unknown situations.

@nchammas
Copy link

@davies I think the reasons for switching are (1) dill is more active and better supported, and (2) it handles some cases better than cloudpickle (see #18 and the comments on SPARK-791).

@mmckerns
Copy link
Member

This is the status of requests from SPARK: #49
Create any issues for what works for you in cloudpickle that doesn't in dill. They'll at least be on the radar here.

@JoshRosen
Copy link

I'm still interested in using Dill in PySpark. I'm sorry for not following up on my earlier Dill issues (such as this one); at the time, I was still in grad school and this fell by the wayside. Now that I'm working on Spark full-time at Databricks, I should be able to follow through on figuring out what's needed from Dill.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants