New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes #30010 -- Add docker-compose support #10725

Open
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
3 participants
@orf
Copy link
Contributor

orf commented Dec 5, 2018

Ticket: https://code.djangoproject.com/ticket/30010

This adds docker-compose support to Django. You can run the test suite like so:

docker-compose run postgres

or for specific Python/DB versions:

PYTHON_VERSION=3.7 POSTGRES_VERSION=9.6 docker-compose run postgres

Some thoughts/notes:

  1. We need the .env file to be called .env and be at the root of the project. See here: docker/compose#4189 (comment)

  2. The django .git directory is huge. If we include it in the .dockerignore file then we cannot use the bisect stuff in runtests. If we don't include it then images get significantly bigger, and many builds can eat disk space. Also: for some reason the .git exclusion in dockerignore is not respected on my machine Edit: Not relevant anymore

  3. I needed to include docker-specific settings under docker/settings. A lot of this feels like it could be de-duplicated, but at the same time it's annoyingly different (different DB hosts, different backends, backend specific config etc)

  4. I was not sure if we want to include wait-for-it.sh or download it at build time. I opted for including it, but we can download it easily enough as well. The reason we need this is because when using docker-compose run it does not respect healthchecks, so the database may not be ready by the time the tests are running.

@orf orf force-pushed the orf:30010-docker-box branch 9 times, most recently from 6958f76 to 1bea41b Dec 5, 2018

Show resolved Hide resolved docker/Dockerfile Outdated

ENV PYTHONPATH "${PYTHONPATH}:/django/:/django/docker/"

VOLUME /django

This comment has been minimized.

@jpic

jpic Dec 5, 2018

Contributor

Is it necessary? docker doesn't ask image permission to mount a volume in a directory. If docker compose says volumes: [.:/django/] then it will mount it anyway - whether the target been marked as VOLUME or not in the Dockerfile.

This comment has been minimized.

@orf

orf Dec 6, 2018

Contributor

It's just metadata about the expected volumes, I'd like to keep it in the image if possible

@timgraham

This comment has been minimized.

Copy link
Member

timgraham commented Dec 5, 2018

You can DRY this up by creating a base settings file (Jenkins configuration uses test_sqlite3.py) with all the common settings. The other settings files look like:

from test_sqlite3 import *  # NOQA

DATABASES = {
   ...
}

@orf orf force-pushed the orf:30010-docker-box branch 4 times, most recently from 0a3d73f to 5aedfe1 Dec 6, 2018

@orf

This comment has been minimized.

Copy link
Contributor

orf commented Dec 6, 2018

Thanks @timgraham. I've done that and I've added documentation. Can you give any advice on how you want to structure it? I've split the Quickstart into two sections: one with docker and one without.

I'm a little bit biased, but in my mind running the suite through docker-compose is more of a quickstart than the current quickstart we have (especially when it comes to installing system packages!), so I moved this to the top of the documentation.

@orf orf force-pushed the orf:30010-docker-box branch from c08f533 to 5611e63 Dec 7, 2018

@jpic

This comment has been minimized.

Copy link
Contributor

jpic commented Dec 7, 2018

Another possibility is to have only one DJANGO_SETTINGS_MODULE and in it switch over a DJANGO_SETTINGS_PROFILE variable. Then you can have only one settings file and still vary the settings on an env var. Maybe worth trying and see if you like that better.

PROFILE = os.getenv('DJANGO_SETTINGS_PROFILE')

if PROFILE in ...:
    ...

@orf orf force-pushed the orf:30010-docker-box branch 2 times, most recently from dc866b3 to 8df10dc Dec 23, 2018

@orf

This comment has been minimized.

Copy link
Contributor

orf commented Dec 29, 2018

Unfortunately this kills the isort and windows jenkins jobs, as they create a virtualenv in a directory called .env. This PR introduces a .env file, who's name cannot be changed :/

@orf orf force-pushed the orf:30010-docker-box branch 2 times, most recently from 1fdb094 to c300a31 Jan 9, 2019

@jpic

This comment has been minimized.

Copy link
Contributor

jpic commented Jan 13, 2019

PYTHON_VERSION=3.7 POSTGRES_VERSION=9.6 docker-compose run postgres works great here ! No need to maintain a django.lxc container anymore to execute tests 👍 🎸

@timgraham is it possible to change .env to .venv maybe in the jobs ?

@orf orf force-pushed the orf:30010-docker-box branch from c300a31 to 2dea11d Jan 19, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment