Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Issues with dill & celery #2526
We'd like to be able to pass objects that aren't pickle-able, and so we're using dill.
In a few places in celery & kombu, pickle is used, regardless of what is registered in the
This stops us using dill for chains & chords. We can monkey patch, but are having some issues doing that, and I'm wondering whether this was an oversight or someone has thought about this already and settled on forcing pickle.
We got this working with the following. You need to monkey patch the pickle in
def add_dill(): registry = kombu.serialization.registry kombu.serialization.pickle = dill registry.unregister('pickle') def pickle_loads(s, load=dill.load): # used to support buffer objects return load(BytesIO(s)) def pickle_dumps(obj, dumper=dill.dumps): return dumper(obj, protocol=kombu.serialization.pickle_protocol) registry.register('pickle', pickle_dumps, pickle_loads, content_type='application/x-python-serialize', content_encoding='binary')