TypeError: set([]) is not JSON serializable for kombu 2.5.0 + celery + django

[riklaunim]

I manage few Django projects using celery which uses kombu and after 2.5.0 got installed it starter throwing TypeErrors. On 2.4.8 everything is working ok. Is it some backward incompatible change and I have to change something or is it a kombu problem?

The example traceback:

...
  File "/home/jenkins/workspace/tweating/.tox/py27/local/lib/python2.7/site-packages/christmas_table/views.py", line 224, in send_facebook_notification
    sender_name)
  File "/home/jenkins/workspace/tweating/.tox/py27/local/lib/python2.7/site-packages/celery/app/task.py", line 357, in delay
    return self.apply_async(args, kwargs)
  File "/home/jenkins/workspace/tweating/.tox/py27/local/lib/python2.7/site-packages/celery/app/task.py", line 478, in apply_async
    **options)
  File "/home/jenkins/workspace/tweating/.tox/py27/local/lib/python2.7/site-packages/celery/app/amqp.py", line 229, in publish_task
    **kwargs)
  File "/home/jenkins/workspace/tweating/.tox/py27/local/lib/python2.7/site-packages/kombu/messaging.py", line 164, in publish
    routing_key, mandatory, immediate, exchange, declare)
  File "/home/jenkins/workspace/tweating/.tox/py27/local/lib/python2.7/site-packages/kombu/connection.py", line 423, in _ensured
    return fun(*args, **kwargs)
  File "/home/jenkins/workspace/tweating/.tox/py27/local/lib/python2.7/site-packages/kombu/messaging.py", line 179, in _publish
    mandatory=mandatory, immediate=immediate,
  File "/home/jenkins/workspace/tweating/.tox/py27/local/lib/python2.7/site-packages/kombu/transport/virtual/__init__.py", line 463, in basic_publish
    exchange, routing_key, **kwargs)
  File "/home/jenkins/workspace/tweating/.tox/py27/local/lib/python2.7/site-packages/kombu/transport/virtual/exchange.py", line 59, in deliver
    _put(queue, message, **kwargs)
  File "/home/jenkins/workspace/tweating/.tox/py27/local/lib/python2.7/site-packages/kombu/transport/django/__init__.py", line 29, in _put
    Queue.objects.publish(queue, dumps(message))
  File "/home/jenkins/workspace/tweating/.tox/py27/local/lib/python2.7/site-packages/anyjson/__init__.py", line 141, in dumps
    return implementation.dumps(value)
  File "/home/jenkins/workspace/tweating/.tox/py27/local/lib/python2.7/site-packages/anyjson/__init__.py", line 87, in dumps
    return self._encode(data)
  File "/usr/lib/python2.7/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 201, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/json/encoder.py", line 264, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python2.7/json/encoder.py", line 178, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: set([]) is not JSON serializable

[rslinckx]

There is a change introduced in kombu 2.5.0 that adds this "bindings" option in the message json.
When using a json serializer for messages it cannot serialize the set.

Quick fix for you is to downgrade to 2.4.10 as I did. I image the set should be casted to a list when being passed on the wire...

[galuszkak]

I have the same problem.

[ask]

Thanks for reporting this, I'm on it now, will produce a fix asap

[srusskih]

Noticed this bug for Django 1.3
Works for Django 1.4

[ask]

2.5.1 is out on PyPI now, please upgrade!

The real bug is in Celery though, because it sends fields in the message that are not needed. I will fix that as well, but it's not critical, just a waste of message size.

[riklaunim]

Seems to be working :)

[vmdx]

Thanks for addressing this!