boilerplate code to start with celery and rabbitmq in docker cluster
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
celery_main
.gitignore
README.md
docker-compose.yml
dockerfile
requirements.txt

README.md

Boilerplate code for celery rabbitmq docker cluster


Version
  • Docker-compose : docker-compose version 1.8.0, build unknown
  • Docker : Docker version 17.12.0-ce, build c97c6d6
  • Python: Python:3 docker image being used in dockerfile

How to run
  • Run command sudo docker-compose up

  • Above command will start 1 container for each worker and rabbit

  • Now go inside one worker container and run python -m celery_main.task_submitter

  • this will start pushing tasks in rabitmq and workers


Speedup the process
  • To run 5 workers and 1 rabbitmq:

    sudo docker-compose up --scale worker=5

  • Do not increase concurrency to too much in dockerfile as machine might not be able to handle it

    ENTRYPOINT celery -A test_celery worker --concurrency=10 --loglevel=info