Research dill vs. cloudpickle for pickling functions

[Ark-kun]

We now use function pickling to capture python code and it's dependencies when creating components form python code. Fairing and Pipelines use the cloudpickle library for that, but some other projects use dill. We need to research the differences and possibly switch to a different library.

Cloudpickle: https://github.com/cloudpipe/cloudpickle
Dill: http://trac.mystic.cacr.caltech.edu/project/pathos/wiki/dill

Some relevant discussions:
piskvorky/gensim#558 (comment)
uqfoundation/multiprocess#22 (comment)

[vicaire]

@Ark-kun, could you please add details about the feature that this research is related to?

[Ark-kun]

@Ark-kun, could you please add details about the feature that this research is related to?

Done.

[gaoning777]

I think you are working on the codegen part of the component authoring, I'll let you lead the efforts.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.