Kombu 4.2.0 breaks with celery 4.1.0

[nicolaszein]

Hi,

I saw that yesterday kombu released version 4.2.0.
After this release celery 4.1.0 starts to break. I guess it should have a release with this change in celery too.

Traceback (most recent call last):
  File "/usr/local/bin/celery", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/celery/__main__.py", line 14, in main
    _main()
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 326, in main
    cmd.execute_from_commandline(argv)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 488, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 281, in execute_from_commandline
    return self.handle_argv(self.prog_name, argv[1:])
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 480, in handle_argv
    return self.execute(command, argv)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 412, in execute
    ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
  File "/usr/local/lib/python3.6/site-packages/celery/bin/worker.py", line 221, in run_from_argv
    return self(*args, **options)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 244, in __call__
    ret = self.run(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/worker.py", line 255, in run
    **kwargs)
  File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 99, in __init__
    self.setup_instance(**self.prepare_args(**kwargs))
  File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 122, in setup_instance
    self.should_use_eventloop() if use_eventloop is None
  File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 241, in should_use_eventloop
    self._conninfo.transport.implements.async and
  File "/usr/local/lib/python3.6/site-packages/kombu/transport/base.py", line 125, in __getattr__
    raise AttributeError(key)
AttributeError: async

My solution was to force kombu 4.1.0 in my requirements.

[grzegorzk]

Hi, I'm observing similar behaviour however freezing version numbers did not work for me. Even with kombu 4.1.0 and celery 4.1.0 I get same stack trace as posted by original poster.

If I install celery 4.1.1 and kombu 4.2.0 then I get missing module error:

pip list | grep "kombu\|celery"
celery 4.1.1
kombu 4.2.0

Traceback (most recent call last):
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
File "/usr/lib/python3.6/site-packages/celery/__main__.py", line 20, in <module>
    main()
File "/usr/lib/python3.6/site-packages/celery/__main__.py", line 16, in main
    _main()
File "/usr/lib/python3.6/site-packages/celery/bin/celery.py", line 322, in main
    cmd.execute_from_commandline(argv)
File "/usr/lib/python3.6/site-packages/celery/bin/celery.py", line 484, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
File "/usr/lib/python3.6/site-packages/celery/bin/base.py", line 275, in execute_from_commandline
    return self.handle_argv(self.prog_name, argv[1:])
File "/usr/lib/python3.6/site-packages/celery/bin/celery.py", line 476, in handle_argv
    return self.execute(command, argv)
File "/usr/lib/python3.6/site-packages/celery/bin/celery.py", line 408, in execute
    ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
File "/usr/lib/python3.6/site-packages/celery/bin/worker.py", line 223, in run_from_argv
    return self(*args, **options)
File "/usr/lib/python3.6/site-packages/celery/bin/base.py", line 238, in __call__
    ret = self.run(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/celery/bin/worker.py", line 257, in run
    **kwargs)
File "/usr/lib/python3.6/site-packages/celery/worker/worker.py", line 101, in __init__
    self.setup_instance(**self.prepare_args(**kwargs))
File "/usr/lib/python3.6/site-packages/celery/worker/worker.py", line 132, in setup_instance
    self.pool_cls = _concurrency.get_implementation(self.pool_cls)
File "/usr/lib/python3.6/site-packages/celery/concurrency/__init__.py", line 23, in get_implementation
    return symbol_by_name(cls, ALIASES)
File "/kombu/utils/imports.py", line 56, in symbol_by_name
    module = imp(module_name, package=package, **kwargs)
File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/usr/lib/python3.6/site-packages/celery/concurrency/prefork.py", line 18, in <module>
    from celery.concurrency.base import BasePool
File "/usr/lib/python3.6/site-packages/celery/concurrency/base.py", line 15, in <module>
    from celery.utils import timer2
File "/usr/lib/python3.6/site-packages/celery/utils/timer2.py", line 16, in <module>
    from kombu.asynchronous.timer import Entry
ModuleNotFoundError: No module named 'kombu.asynchronous.timer'

[cixuuz]

Same thing happens with celery 4.0.2 and airflow 1.9.

[auvipy]

could you please use celery 4.1.1 or 4.2rc4 and let us know what error you got?

[wplct]

Traceback (most recent call last): File "C:/Users/wplct/PycharmProjects/spiderAll3/run_celery.py", line 6, in <module> spider.worker_main() File "C:\Users\wplct\PycharmProjects\spiderAll3\venv\lib\site-packages\celery\app\base.py", line 359, in worker_main ).execute_from_commandline(argv) File "C:\Users\wplct\PycharmProjects\spiderAll3\venv\lib\site-packages\celery\bin\base.py", line 275, in execute_from_commandline return self.handle_argv(self.prog_name, argv[1:]) File "C:\Users\wplct\PycharmProjects\spiderAll3\venv\lib\site-packages\celery\bin\base.py", line 362, in handle_argv return self(*args, **options) File "C:\Users\wplct\PycharmProjects\spiderAll3\venv\lib\site-packages\celery\bin\base.py", line 238, in __call__ ret = self.run(*args, **kwargs) File "C:\Users\wplct\PycharmProjects\spiderAll3\venv\lib\site-packages\celery\bin\worker.py", line 257, in run **kwargs) File "C:\Users\wplct\PycharmProjects\spiderAll3\venv\lib\site-packages\celery\worker\worker.py", line 101, in __init__ self.setup_instance(**self.prepare_args(**kwargs)) File "C:\Users\wplct\PycharmProjects\spiderAll3\venv\lib\site-packages\celery\worker\worker.py", line 132, in setup_instance self.pool_cls = _concurrency.get_implementation(self.pool_cls) File "C:\Users\wplct\PycharmProjects\spiderAll3\venv\lib\site-packages\celery\concurrency\__init__.py", line 23, in get_implementation return symbol_by_name(cls, ALIASES) File "C:\Users\wplct\PycharmProjects\spiderAll3\venv\kombu\utils\imports.py", line 56, in symbol_by_name module = imp(module_name, package=package, **kwargs) File "C:\Users\wplct\PycharmProjects\spiderAll3\venv\lib\importlib\__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 994, in _gcd_import File "<frozen importlib._bootstrap>", line 971, in _find_and_load File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 665, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 678, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "C:\Users\wplct\PycharmProjects\spiderAll3\venv\lib\site-packages\celery\concurrency\prefork.py", line 18, in <module> from celery.concurrency.base import BasePool File "C:\Users\wplct\PycharmProjects\spiderAll3\venv\lib\site-packages\celery\concurrency\base.py", line 15, in <module> from celery.utils import timer2 File "C:\Users\wplct\PycharmProjects\spiderAll3\venv\lib\site-packages\celery\utils\timer2.py", line 16, in <module> from kombu.asynchronous.timer import Entry ModuleNotFoundError: No module named 'kombu.asynchronous.timer'

[grzegorzk]

Hi, in my case when I have seen this error I noticed a "kombu" folder was created in the root of my venv. In the same time I still could see kombu in venv-related site-packages. As soon as I removed this weird "kombu" from venv root everything worked fine for me.

[thedrow]

@auvipy Is it possible we made a mistake? See #873 (comment)

[auvipy]

let me check

[auvipy]

Omer, I checked locally land it works for me, not sure if they are working on production and didn't restart after updating?

[ponpondev]

I can also confirm that removing the venv root's kombu folder will fix ModuleNotFoundError: No module named 'kombu.asynchronous.timer'

Perhaps there is something fishy during the installation of kombu?

[auvipy]

may be it's a local env cache issue?

[ponpondev]

I tried uninstalling kombu and then pip install kombu
The installation process adds a kombu folder to venv root.

My package is:
kombu 4.2.0 and celery 4.1.1

I just started learning Celery today and ran into this problem. Feelsbad hahaa

[thedrow]

@auvipy I think that at this point we can consider this another blocker.

[auvipy]

what do you think could be the issue?

[thedrow]

I really don't know. I checked the spelling of the import and it looks fine.
Tests also pass.

[auvipy]

Omer I still thing this isn't a blocker since it's working fine locally on my machine, even with py3.7-dev, I guess local pycache of specific local env is causing this. but who knows?

[wtayyeb]

Hi, I found the problem in data_files in setup.py, it create a data folder for kumbo containing .pyc files for python 3.6 from one who build the wheel. i think he is @thedrow
i found this in there

/home/omer/Documents/Projects/kombu/kombu/__init__

to verify you could download the wheel from pypi and extract it to see the problem

I install it in vrtualenv with python2.7, python3.6 and both fail to work.

as I noted before I think there is problem in setup.py -> data_files

[auvipy]

thanks for digging it!

[thedrow]

Thanks!
Strange, I 'git clean -f -x -d' before each release.
I'll fix it today.

[violetaria]

Hello,

I have been trying to get my airflow (1.9.0) worker fixed and none of the solutions have worked. I continue to get this error after pinning celery to 4.1.0 and kombu to 4.1.0

Is there a version combination that does work?

[2018-07-16 15:20:58,862] {__init__.py:45} INFO - Using executor CeleryExecutor
/usr/local/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/kombu/transport/base.py", line 123, in __getattr__
    return self[key]
KeyError: 'async'
 During handling of the above exception, another exception occurred:
 Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 27, in <module>
    args.func(args)
  File "/usr/local/lib/python3.6/site-packages/airflow/bin/cli.py", line 891, in worker
    worker.run(**options)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/worker.py", line 255, in run
    **kwargs)
  File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 99, in __init__
    self.setup_instance(**self.prepare_args(**kwargs))
  File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 122, in setup_instance
    self.should_use_eventloop() if use_eventloop is None
  File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 241, in should_use_eventloop
    self._conninfo.transport.implements.async and
  File "/usr/local/lib/python3.6/site-packages/kombu/transport/base.py", line 125, in __getattr__
    raise AttributeError(key)
AttributeError: async

[thedrow]

Celery 4.2 and Kombu 4.2.
Please clean up your virtualenv and try again.

[violetaria]

I'll try those versions but I do not think we are using virtualenv. We are using Dockerfiles to build images and use those images to run everything in Kubernetes. Should we be using virtualenv?

[violetaria]

that version combination worked! thanks for the help