Celery Docker Upgrades

[Nusnus]

1. Allow build, lint, test & bash with the celery container using make commands.
2. Added CI to build the docker image to make sure it doesn't get broken in a future PR without knowing.
3. Default to Python 3.11 in the Dockerfile if not set otherwise via .python-version (pyenv).

For example running make docker-lint from the celery repo root:

make docker-lint

[+] Creating 4/0
 ✔ Container docker-azurite-1   Running                                                                                                                                           0.0s
 ✔ Container docker-rabbit-1    Running                                                                                                                                           0.0s
 ✔ Container docker-dynamodb-1  Running                                                                                                                                           0.0s
 ✔ Container docker-redis-1     Running                                                                                                                                           0.0s
ROOT: will run in automatically provisioned tox, host /home/developer/.pyenv/versions/3.11.5/bin/python3.11 is missing [requires (has)]: tox-gh-actions
ROOT: recreate env because python changed version_info=[3, 9, 13, 'final', 0]->[3, 11, 5, 'final', 0] | executable='/Users/nusnus/.pyenv/versions/3.9.13/bin/python3.9'->'/home/developer/.pyenv/versions/3.11.5/bin/python3.11' | virtualenv version='20.24.4'->'20.24.5'
ROOT: remove tox env folder /home/developer/celery/.tox/.tox
ROOT: install_deps> python -I -m pip install tox tox-gh-actions
ROOT: provision> .tox/.tox/bin/python -m tox -e lint
ROOT: tox-gh-actions won't override envlist because tox is not running in GitHub Actions
.pkg: _optional_hooks> python /home/developer/celery/.tox/.tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_editable> python /home/developer/celery/.tox/.tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: build_editable> python /home/developer/celery/.tox/.tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
lint: install_package> python -I -m pip install --force-reinstall --no-deps /home/developer/celery/.tox/.tmp/package/17/celery-5.3.4-0.editable-py3-none-any.whl
lint: commands[0]> pre-commit run --all-files --show-diff-on-failure
[INFO] Initializing environment for https://github.com/asottile/pyupgrade.
[INFO] Initializing environment for https://github.com/PyCQA/flake8.
[INFO] Initializing environment for https://github.com/asottile/yesqa.
[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Initializing environment for https://github.com/pycqa/isort.
[INFO] Initializing environment for https://github.com/pre-commit/mirrors-mypy.
[INFO] Installing environment for https://github.com/asottile/pyupgrade.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/PyCQA/flake8.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/asottile/yesqa.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/pycqa/isort.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/pre-commit/mirrors-mypy.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
pyupgrade................................................................Passed
flake8...................................................................Passed
Strip unnecessary `# noqa`s..............................................Passed
check for merge conflicts................................................Passed
check toml...............................................................Passed
check yaml...............................................................Passed
mixed line ending........................................................Passed
isort....................................................................Passed
mypy.....................................................................Passed
.pkg: _exit> python /home/developer/celery/.tox/.tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
  lint: OK (35.25=setup[3.34]+cmd[31.92] seconds)
  congratulations :) (37.49 seconds)

The make help is also modified:

make help
docs                 - Build documentation.
test-all             - Run tests for all supported python versions.
distcheck ---------- - Check distribution for problems.
  test               - Run unittests using current python.
  lint ------------  - Check codebase for problems.
    apicheck         - Check API reference coverage.
    configcheck      - Check configuration reference coverage.
    readmecheck      - Check README.rst encoding.
    contribcheck     - Check CONTRIBUTING.rst encoding
    flakes --------  - Check code for syntax and style errors.
      flakecheck     - Run flake8 on the source code.
      flakepluscheck - Run flakeplus on the source code.
readme               - Regenerate README.rst file.
contrib              - Regenerate CONTRIBUTING.rst file
clean-dist --------- - Clean all distribution build artifacts.
  clean-git-force    - Remove all uncommitted files.
  clean ------------ - Non-destructive clean
    clean-pyc        - Remove .pyc/__pycache__ files
    clean-docs       - Remove documentation build artifacts.
    clean-build      - Remove setup artifacts.
bump                 - Bump patch version number.
bump-minor           - Bump minor version number.
bump-major           - Bump major version number.
release              - Make PyPI release.

Docker-specific commands:
  docker-build			- Build celery docker container.
  docker-lint        		- Run tox -e lint on docker container.
  docker-unit-tests		- Run unit tests on docker container, use '-- -k <TEST NAME>' for specific test run.
  docker-bash        		- Get a bash shell inside the container.

[Nusnus]

Updated PR message @auvipy .
PR is ready for review.

[Nusnus]

The integration tests are partially working in the docker environment so this PR doesn't fix 100% per-say the problem from #8447, but it should at least improve on the current capability, allowing the integration tests to even work with the broker and backend from the docker compose (which was 100% broken before - see changes in tox.ini).

[Nusnus]

Changed a string from '' to "" to trigger full CI to make sure this change doesn't create unwanted side effects.

[codecov]

Codecov Report

Patch and project coverage have no change.

Comparison is base (65ac2ac) 87.45% compared to head (9f331ce) 87.45%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #8531   +/-   ##
=======================================
  Coverage   87.45%   87.45%           
=======================================
  Files         148      148           
  Lines       18499    18499           
  Branches     3158     3158           
=======================================
  Hits        16179    16179           
  Misses       2032     2032           
  Partials      288      288           

Flag	Coverage Δ
unittests	87.42% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.