MongoDB Replicaset

[metzlar]

Hi,

I noticed Celery breaks when supplying a MongoDB replicaset inside the connection string. This patch should fix this. My config reads:

MONGODB_HOST = 'mongodb://localhost:27170,localhost:27171'

CELERY_MONGODB_BACKEND_SETTINGS = {
"host": MONGODB_HOST
}
BROKER_HOST = MONGODB_HOST

pymongo can deal with a valid MongoDB connection URI as first argument (host) to pymongo.Connect() thats why I chose to use the HOST config value instead of creating a new one.

I have also made a patch for Celery's mongodb backend for which I will submit a pull-request shortly after this one. I do not have much experience with Kombu so please let me know if there is a more proper way to deal with this.

Cheers,

--Ivan

[flaper87]

Seems correct.

Did you try running tests?

thanks

[metzlar]

Hi, yes all Kombu tests pass. 6 Celery tests fail but they seem not
related to my changes.

On Fri, Nov 11, 2011 at 5:01 PM, Flavio Percoco Premoli
reply@reply.github.com
wrote:

Seems correct.

Did you try running tests?

---

Reply to this email directly or view it on GitHub:
https://github.com/ask/kombu/pull/81#issuecomment-2709619

[flaper87]

Ok, ready to merge.

@ask, do you agree?

[ask]

@flaper87, yeah, looks good!

[DeaconDesperado]

I might be doing something wrong, but I still seem to be having trouble trying to get celery to connect to mongodb using mongouris representing replicasets. I tried to copy @metzlar verbatim. Here is my config and the error it raises:

MONGODB_HOST = 'mongodb://localhost:27017,localhost:27018,localhost:27019/celery'

CELERY_MONGODB_BACKEND_SETTINGS = {
    "host": MONGODB_HOST
}
BROKER_HOST = MONGODB_HOST

CELERY_IMPORTS = ("tasks",)

And the exception:

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/celery/worker/__init__.py", line 230, in start
    component.start()
  File "/usr/local/lib/python2.7/dist-packages/celery/worker/consumer.py", line 338, in start
    self.reset_connection()
  File "/usr/local/lib/python2.7/dist-packages/celery/worker/consumer.py", line 596, in reset_connection
    on_decode_error=self.on_decode_error)
  File "/usr/local/lib/python2.7/dist-packages/celery/app/amqp.py", line 335, in get_task_consumer
    **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/kombu/compat.py", line 187, in __init__
    super(ConsumerSet, self).__init__(self.backend, queues, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/kombu/messaging.py", line 285, in __init__
    self.declare()
  File "/usr/local/lib/python2.7/dist-packages/kombu/messaging.py", line 295, in declare
    queue.declare()
  File "/usr/local/lib/python2.7/dist-packages/kombu/entity.py", line 388, in declare
    self.queue_declare(nowait, passive=False)
  File "/usr/local/lib/python2.7/dist-packages/kombu/entity.py", line 408, in queue_declare
    nowait=nowait)
  File "/usr/local/lib/python2.7/dist-packages/kombu/transport/virtual/__init__.py", line 380, in queue_declare
    return queue, self._size(queue), 0
  File "/usr/local/lib/python2.7/dist-packages/kombu/transport/mongodb.py", line 74, in _size
    return self.client.messages.find({"queue": queue}).count()
  File "/usr/local/lib/python2.7/dist-packages/kombu/transport/mongodb.py", line 171, in client
    self._client = self._open()
  File "/usr/local/lib/python2.7/dist-packages/kombu/transport/mongodb.py", line 97, in _open
    mongoconn = Connection(host=conninfo.hostname, port=conninfo.port)
  File "/usr/local/lib/python2.7/dist-packages/pymongo/connection.py", line 325, in __init__
    nodes.update(uri_parser.split_hosts(entity, port))
  File "/usr/local/lib/python2.7/dist-packages/pymongo/uri_parser.py", line 198, in split_hosts
    nodes.append(parse_host(entity, default_port))
  File "/usr/local/lib/python2.7/dist-packages/pymongo/uri_parser.py", line 134, in parse_host
    raise ConfigurationError("Port number must be an integer.")
ConfigurationError: Port number must be an integer.