Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KeyError async #870

Closed
Sonictherocketman opened this Issue May 21, 2018 · 15 comments

Comments

Projects
None yet
10 participants
@Sonictherocketman
Copy link

commented May 21, 2018

Was doing some late night builds when I noticed that none of my workers ran anymore.

After this version was pushed I started getting errors

AttributeError async
KeyError async

I think it has to do with some missing changes during this commit 7569520

Downgrading to 4.1 solved the issue.

@nathan-jones

This comment has been minimized.

Copy link

commented May 21, 2018

Same here:

$ celery -A rtp worker -l info
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/kombu/transport/base.py", line 123, in __getattr__
    return self[key]
KeyError: 'async'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/celery", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/celery/__main__.py", line 14, in main
    _main()
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 326, in main
    cmd.execute_from_commandline(argv)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 488, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 281, in execute_from_commandline
    return self.handle_argv(self.prog_name, argv[1:])
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 480, in handle_argv
    return self.execute(command, argv)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 412, in execute
    ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
  File "/usr/local/lib/python3.6/site-packages/celery/bin/worker.py", line 221, in run_from_argv
    return self(*args, **options)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 244, in __call__
    ret = self.run(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/worker.py", line 255, in run
    **kwargs)
  File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 99, in __init__
    self.setup_instance(**self.prepare_args(**kwargs))
  File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 122, in setup_instance
    self.should_use_eventloop() if use_eventloop is None
  File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 241, in should_use_eventloop
    self._conninfo.transport.implements.async and
  File "/usr/local/lib/python3.6/site-packages/kombu/transport/base.py", line 125, in __getattr__
    raise AttributeError(key)
AttributeError: async
@pssuils

This comment has been minimized.

Copy link

commented May 21, 2018

Same here with kombu==4.2.0 and celery==4.1.0 however fixing the kombu==4.1.0 fixes the issue until version 4.2.0 is stable

@clokep

This comment has been minimized.

Copy link

commented May 21, 2018

I believe this is from #742 / #839.

Because of that change (+ a corresponding celery change), I believe kombu 4.2.0 is not compatible with celery 4.1.0 (and vice versa).

@auvipy

This comment has been minimized.

Copy link
Member

commented May 21, 2018

could you check celery 4.1.1? it's a hotfix release

@alex-handley

This comment has been minimized.

Copy link

commented May 21, 2018

I think this might be related - celery/celery#4679

My Airflow staging build has started failing with the same error

@auvipy

This comment has been minimized.

Copy link
Member

commented May 21, 2018

this should be fixed by 4.1.1 check and report. we already got report airflow error was fixed by 4.1.1

@auvipy auvipy closed this May 21, 2018

@alex-handley

This comment has been minimized.

Copy link

commented May 21, 2018

Correct, upgrading and builds passing again. Thanks!

@auvipy

This comment has been minimized.

Copy link
Member

commented May 21, 2018

thanks for updating!

@artinnok

This comment has been minimized.

Copy link

commented May 23, 2018

Same bug here:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/kombu/transport/base.py", line 123, in __getattr__
    return self[key]
KeyError: 'async'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/celery", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/celery/__main__.py", line 14, in main
    _main()
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 326, in main
    cmd.execute_from_commandline(argv)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 488, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 281, in execute_from_commandline
    return self.handle_argv(self.prog_name, argv[1:])
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 480, in handle_argv
    return self.execute(command, argv)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 412, in execute
    ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
  File "/usr/local/lib/python3.6/site-packages/celery/bin/worker.py", line 221, in run_from_argv
    return self(*args, **options)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 244, in __call__
    ret = self.run(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/celery/bin/worker.py", line 255, in run
    **kwargs)
  File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 99, in __init__
    self.setup_instance(**self.prepare_args(**kwargs))
  File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 122, in setup_instance
    self.should_use_eventloop() if use_eventloop is None
  File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 241, in should_use_eventloop
    self._conninfo.transport.implements.async and
  File "/usr/local/lib/python3.6/site-packages/kombu/transport/base.py", line 125, in __getattr__
    raise AttributeError(key)
AttributeError: async
@auvipy

This comment has been minimized.

Copy link
Member

commented May 23, 2018

@artinnok please update to celery 4.1.1 or 4.2rc4

@clintonb

This comment has been minimized.

Copy link

commented May 24, 2018

This issue was introduced by #839; however, I think there's another issue: versioning. Kombu appears to follow semantic versioning, but that is not documented anywhere. If semantic versioning is followed, this backwards-incompatible change should have resulted in a bump to version 5.0.0.

The third issue lies in the fact that Celery's dependencies have no upper bound: https://github.com/celery/celery/blob/master/setup.cfg#L18-L21. Even if a major version change had gone into effect, our installations would have still been broken. Celery should have an upper bound to prevent problems like this in the future.

@auvipy

This comment has been minimized.

Copy link
Member

commented May 24, 2018

i agree that should be pinned

Irio added a commit to okfn-brasil/diario-oficial that referenced this issue May 27, 2018

Irio added a commit to okfn-brasil/diario-oficial that referenced this issue May 27, 2018

Irio added a commit to okfn-brasil/diario-oficial that referenced this issue May 27, 2018

@adrienjt

This comment has been minimized.

Copy link

commented May 31, 2018

As @clintonb mentioned, this is a versioning issue in Kombu. Rather than moving to asynchronous in Celery 4.1.1 and ask everybody to upgrade to it, Kombu 4.2.1 should have reinstated the async module, to delete it only in version 5. To avoid further damage (breaking builds in many projects pinned to Celery 4.1.0), I strongly recommend that Kombu reinstate the async module in a 4.2.2. Please re-open the issue.

@auvipy

This comment has been minimized.

Copy link
Member

commented Jun 1, 2018

just pin your kombu and celery.

@celery celery locked as spam and limited conversation to collaborators Jun 1, 2018

@thedrow

This comment has been minimized.

Copy link
Member

commented Jun 3, 2018

@adrienjt I agree this was an oversight. We'll take more care in the future.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.