attempt to fix #1965 analogously for epoll

[tbg]

this should hopefully take care of the error's I've recently seen with a prefork based worker, which seemed analogous to #1965 in gevent

[stevedh]

We're also hitting this using the redis broker on Ubuntu 12.04 -- I haven't dug in enough to check if the proposed patch.

>>> celery.__version__, kombu.__version__
('3.1.11', '3.0.15')

  File "/home/ubuntu/develop/venv/local/lib/python2.7/site-packages/celery/beat.py", line 203, in maybe_due
    result = self.apply_async(entry, publisher=publisher)
...
  File "/home/ubuntu/develop/venv/local/lib/python2.7/site-packages/redis/client.py", line 1130, in smembers
    return self.execute_command('SMEMBERS', name)
  File "/home/ubuntu/develop/venv/local/lib/python2.7/site-packages/redis/client.py", line 396, in execute_command
    connection.disconnect()
  File "/home/ubuntu/develop/venv/local/lib/python2.7/site-packages/kombu/transport/redis.py", line 800, in disconnect
    channel._on_connection_disconnect(self)
  File "/home/ubuntu/develop/venv/local/lib/python2.7/site-packages/kombu/transport/redis.py", line 461, in _on_connection_disconnect
    self.connection.cycle._on_connection_disconnect(connection)
  File "/home/ubuntu/develop/venv/local/lib/python2.7/site-packages/kombu/transport/redis.py", line 259, in _on_connection_disconnect
    self.poller.unregister(connection._sock)
  File "/home/ubuntu/develop/venv/local/lib/python2.7/site-packages/kombu/utils/eventio.py", line 85, in unregister
    self._epoll.unregister(fd)
SchedulingError: Couldn't apply scheduled task monitoring-daily-email: argument must be an int, or have a fileno() method.

FWIW, downgrading to

>>> celery.__version__, kombu.__version__
('3.1.9', '3.0.13')

Seems to work around the problem.

[tspike]

I was getting similar intermittent errors using 3.1.11 and 3.0.15 and the redis broker:

Traceback (most recent call last):
  File "/code/flask/local/lib/python2.7/site-packages/celery/canvas.py", line 167, in delay
    return self.apply_async(partial_args, partial_kwargs)
  File "/code/flask/local/lib/python2.7/site-packages/celery/canvas.py", line 241, in apply_async
    return _apply(args, kwargs, **options)
  File "/code/flask/local/lib/python2.7/site-packages/celery/app/task.py", line 555, in apply_async
    **dict(self._get_exec_options(), **options)

...

  File "/code/flask/local/lib/python2.7/site-packages/kombu/transport/redis.py", line 461, in _on_connection_disconnect
    self.connection.cycle._on_connection_disconnect(connection)
  File "/code/flask/local/lib/python2.7/site-packages/kombu/transport/redis.py", line 259, in _on_connection_disconnect
    self.poller.unregister(connection._sock)
  File "/code/flask/local/lib/python2.7/site-packages/kombu/utils/eventio.py", line 85, in unregister
    self._epoll.unregister(fd)
TypeError: argument must be an int, or have a fileno() method.

From limited testing, downgrading to 3.1.9 and 3.0.13 per @stevedh's suggestion appears to have worked for me as well.

[zachsnow]

Also recently started running into this. It seems like in this case connection._sock might be None?

[ask]

@zachsnow does the proposed patch work for you?

[zachsnow]

@ask I haven't had a chance yet; the only place that I can reproduce is production unfortunately.

I would expect that changing this code (https://github.com/celery/kombu/blob/master/kombu/transport/redis.py#L257) to check for connection._sock is None before unregistering might swallow less errors but still work; if I manage to reproduce somewhere that I can test changes I will let you know / generate a pull request.

[ask]

Thanks! Merged into master

[tbg]

thanks @ask! I'm closing this as it's likely not needed any more. Will give the new code a shot asap.

[ask]

Fixed in Kombu 3.0.16 out on PyPI now!

[zachsnow]

@ask great I'll give it a try. Thanks!