Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release of Kombu 4.2.0 breaks Celery 4.1.0 #4753

Closed
MattFisher opened this issue May 22, 2018 · 4 comments
Closed

Release of Kombu 4.2.0 breaks Celery 4.1.0 #4753

MattFisher opened this issue May 22, 2018 · 4 comments

Comments

@MattFisher
Copy link

Celery workers on newly-created instances in a production AWS auto-scaling group began failing after a scheduled scale-up about six hours ago, without any changes being deployed.
Our requirements file specified celery==4.2.0rc1, and the problem was not solved by installing Celery 4.1.0.

Same problem observed by others: https://stackoverflow.com/questions/50444988/celery-attributeerror-async-error

It seems to be an incompatibility with Kombu 4.2.0, which was released 19 hours ago. Celery 4.1.0's requirements specifies kombu>=4.0.2,<5.0. This means new installations of Celery 4.1.0 will install Kombu 4.2.0 unless the Kombu version is manually pinned in the project's requirements, and experience this failure.

Recommend specifying the Kombu version in celery 4.1.0 to kombu>=4.0.2, <4.2.0

Stack trace for reference:

KeyError: 'async'
  File "kombu/transport/base.py", line 123, in __getattr__
   return self[key]
AttributeError: async
  File "var/app/run/venv3/bin/celery", line 11, in <module>
    sys.exit(main())
  File "celery/__main__.py", line 16, in main
    _main()
  File "celery/bin/celery.py", line 322, in main
    cmd.execute_from_commandline(argv)
  File "celery/bin/celery.py", line 484, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "celery/bin/base.py", line 275, in execute_from_commandline
    return self.handle_argv(self.prog_name, argv[1:])
  File "celery/bin/celery.py", line 476, in handle_argv
    return self.execute(command, argv)
  File "celery/bin/celery.py", line 408, in execute
    ).run_from_argv(self.prog_name, argv[1:], command=argv[0])
  File "celery/bin/worker.py", line 223, in run_from_argv
    return self(*args, **options)
  File "celery/bin/base.py", line 238, in __call__
    ret = self.run(*args, **kwargs)
  File "celery/bin/worker.py", line 257, in run
    **kwargs)
  File "celery/worker/worker.py", line 101, in __init__
    self.setup_instance(**self.prepare_args(**kwargs))
  File "celery/worker/worker.py", line 124, in setup_instance
    self.should_use_eventloop() if use_eventloop is None
  File "celery/worker/worker.py", line 243, in should_use_eventloop
    self._conninfo.transport.implements.async and
  File "kombu/transport/base.py", line 125, in __getattr__
    raise AttributeError(key)
@hylarucoder
Copy link

hylarucoder commented May 22, 2018

ref issue celery/kombu#870

just upgrade to celery 4.1.1

@MattFisher
Copy link
Author

Didn't see the closed issue sorry

@MattFisher
Copy link
Author

We would but 4.1.1 doesn't include #4437, which we need.

@thedrow
Copy link
Member

thedrow commented May 22, 2018

I just released Celery 4.2RC4.

@thedrow thedrow closed this as completed May 22, 2018
ryanrussell added a commit to ryanrussell/docker-airflow that referenced this issue May 29, 2018
Worker gives KeyError async on fresh docker build due to Kombu 4.2. Result is a continually failing/rebooting worker.

Tracked it down to this issue.
celery/celery#4753

Should upgrade to celery 4.2 when stable
puckel pushed a commit to puckel/docker-airflow that referenced this issue Jun 7, 2018
Worker gives KeyError async on fresh docker build due to Kombu 4.2. Result is a continually failing/rebooting worker.

Tracked it down to this issue.
celery/celery#4753

Should upgrade to celery 4.2 when stable
gagejustins pushed a commit to gagejustins/snql-airflow that referenced this issue Dec 29, 2018
Worker gives KeyError async on fresh docker build due to Kombu 4.2. Result is a continually failing/rebooting worker.

Tracked it down to this issue.
celery/celery#4753

Should upgrade to celery 4.2 when stable
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants