No module named 'celery.backends.amqp' or KeyError: 'backend'

[artem-shestakov]

Problem with Python 3.8.5 / Flask and Celery 5.0

(venv) ashestakov@MacBook-Pro-Artem ToDo_Service % celery -A main.celery worker
[2020-10-01 16:31:07,749: CRITICAL/MainProcess] Unrecoverable error: ModuleNotFoundError("No module named 'celery.backends.amqp'")
Traceback (most recent call last):
  File "/Users/ashestakov/Documents/Py_projects/ToDo_Service/venv/lib/python3.8/site-packages/kombu/utils/objects.py", line 41, in __get__
    return obj.__dict__[self.__name__]
KeyError: 'backend'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ashestakov/Documents/Py_projects/ToDo_Service/venv/lib/python3.8/site-packages/celery/worker/worker.py", line 203, in start
    self.blueprint.start(self)
  File "/Users/ashestakov/Documents/Py_projects/ToDo_Service/venv/lib/python3.8/site-packages/celery/bootsteps.py", line 112, in start
    self.on_start()
  File "/Users/ashestakov/Documents/Py_projects/ToDo_Service/venv/lib/python3.8/site-packages/celery/apps/worker.py", line 136, in on_start
    self.emit_banner()
  File "/Users/ashestakov/Documents/Py_projects/ToDo_Service/venv/lib/python3.8/site-packages/celery/apps/worker.py", line 159, in emit_banner
    ' \n', self.startup_info(artlines=not use_image))),
  File "/Users/ashestakov/Documents/Py_projects/ToDo_Service/venv/lib/python3.8/site-packages/celery/apps/worker.py", line 221, in startup_info
    results=self.app.backend.as_uri(),
  File "/Users/ashestakov/Documents/Py_projects/ToDo_Service/venv/lib/python3.8/site-packages/kombu/utils/objects.py", line 43, in __get__
    value = obj.__dict__[self.__name__] = self.__get(obj)
  File "/Users/ashestakov/Documents/Py_projects/ToDo_Service/venv/lib/python3.8/site-packages/celery/app/base.py", line 1169, in backend
    return self._get_backend()
  File "/Users/ashestakov/Documents/Py_projects/ToDo_Service/venv/lib/python3.8/site-packages/celery/app/base.py", line 884, in _get_backend
    backend, url = backends.by_url(
  File "/Users/ashestakov/Documents/Py_projects/ToDo_Service/venv/lib/python3.8/site-packages/celery/app/backends.py", line 70, in by_url
    return by_name(backend, loader), url
  File "/Users/ashestakov/Documents/Py_projects/ToDo_Service/venv/lib/python3.8/site-packages/celery/app/backends.py", line 50, in by_name
    cls = symbol_by_name(backend, aliases)
  File "/Users/ashestakov/Documents/Py_projects/ToDo_Service/venv/lib/python3.8/site-packages/kombu/utils/imports.py", line 56, in symbol_by_name
    module = imp(module_name, package=package, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'celery.backends.amqp'

Python code:
init.py

from flask import Flask
from flask_mongoengine import MongoEngine
from flask_jwt_extended import JWTManager
from todo.utils.email import mail
from todo.utils.response import response_with
import todo.utils.response_code as response_code
from celery import Celery
import logging

# Init MongoDB object
mongo = MongoEngine()


def make_celery(app):
    celery = Celery(
        app.import_name,
        backend=app.config['CELERY_RESULT_BACKEND'],
        broker=app.config['CELERY_BROKER_URL']
    )
    celery.conf.update(app.config)

    class ContextTask(celery.Task):
        def __call__(self, *args, **kwargs):
            with app.app_context():
                return self.run(*args, **kwargs)

    celery.Task = ContextTask
    return celery


def create_app(config_object):
    """
    Create Flask application object with all modules

    :param config_object: Configure object from config file
    """
    from .api import create_module as api_create_module
    from .users import create_model as users_create_module
    from .board import create_module as board_create_module
    from .lists import create_module as lists_create_module
    from .auth import create_module as auth_create_module

    app = Flask(__name__)
    app.config.from_object(config_object)

    jwt = JWTManager(app)

    @app.errorhandler(404)
    def not_found(e):
        logging.error(e)
        return response_with(response_code.NOT_FOUND_404)

    @app.errorhandler(500)
    def not_found(e):
        logging.error(e)
        return response_with(response_code.SERVER_ERROR_500)

    mongo.init_app(app)
    mail.init_app(app)

    # Register Blueprints
    api_create_module(app)
    users_create_module(app)
    auth_create_module(app)
    board_create_module(app)
    lists_create_module(app)

    return app

run.py

from todo import create_app, make_celery
from todo.cli import cli_register
import os

# Getting WORK_ENV variable and create Flask object.
env = os.environ.get('WORK_ENV', 'Dev')
app = create_app(f'config.{env.capitalize()}Config')
celery = make_celery(app)
cli_register(app)

if __name__ == '__main__':
    app.run()

[thedrow]

The amqp backend was deprecated as documented in the release notes.
Please use another backend.

[delivey]

A temporary fix is to pip uninstall celery and pip install celery==4.4.6. Note this is not recommended.

[Kailaash-Balachandran]

temporary fix until its updated: use CELERY_RESULT_BACKEND = "rpc://" instead

[chrisspen]

The amqp backend was deprecated as documented in the release notes. Please use another backend.

@thedrow Your release notes link is broken. The docs still imply RabbitMQ and amqp support is available. Have the docs not been updated or am I misreading them?

I've tried changing my CELERY_RESULT_BACKEND to rpc://, but this doesn't fix the error. Does the CELERY_BROKER_URL also need to be changed as well to not use "amqp://'?

[sieira]

Hello. I'm wondering why is this issue closed? Is the backend supported or not?

It is according to the documentation.

https://docs.celeryproject.org/en/stable/reference/celery.app.amqp.html

Upgrading celery o version >5 is a requirement for our product, given CVE-2021-23727. Are we supposed to move to rpc?

Thank you for your amazing work

[sieira]

Ok, I found the answer in the 5.2 release notes nevermind. Thankyou

[eldamir]

Still listed in the docs as the default and an "Excellent choice" https://docs.celeryproject.org/en/stable/getting-started/first-steps-with-celery.html#rabbitmq

[sieira]

I'm wondering, how can we participate in updating the documentation?

[auvipy]

Still listed in the docs as the default and an "Excellent choice" https://docs.celeryproject.org/en/stable/getting-started/first-steps-with-celery.html#rabbitmq

As broaker/transport, not as result backend

[thedrow]

I'm wondering, how can we participate in updating the documentation?

Pull requests are welcome.

[Kludex]

The amqp backend is still on the BACKEND_ALIASES list. It should probably be removed. 🤔

[auvipy]

The amqp backend is still on the BACKEND_ALIASES list. It should probably be removed. thinking

If so, yes

[kevinomyonga]

I encountered this issue while running Celery on a docker container. Changing my CELERY_RESULT_BACKEND from pyamqp:// to rpc:// fixed the error.