-
Notifications
You must be signed in to change notification settings - Fork 2.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(docker-compose): make startup more robust with deterministic services' dependencies #7880
Conversation
@gcernier-semarchy - This is really nice!!! When did these features get added to docker-compose? I recall looking for a way to do this and not finding it or something along the lines of it was ignored by docker-compose cli. |
Yes @david-leifker, those healthchecks-based dependency mechanisms are very helpful! I'm not sure of understanding exactly the whole story, but it seems that there were some back and forths, because:
This highlights in particular the mix of 2 lifecycles involved there, which complexifies understanding:
And then, it seems like:
In the meantime, when it was not supported, it seems like (even if it disappeared from the documentation history) Docker Compose suggested to perform such checks in application code or thanks to a 3rd-party tools like Anyway, the good news is that this syntax and this feature are now natively supported and since quite some time and versions of both Compose Specification and Docker Compose! 🙂 |
dfdf3ea
to
782f8e3
Compare
782f8e3
to
7fe924b
Compare
7fe924b
to
a24e095
Compare
I really like this PR, the blocker that I need to address here is that the quickstart compose files are derived from other compose files in the parent directory. I need to actually reproduce the changes there so that the automated generation of the quickstart versions are in alignment. I believe it'll be next week before I can get bandwidth to make that change here. |
Ah yes @david-leifker! Now that you point me to this, I get what is missing in my PR. |
a24e095
to
4211358
Compare
Hello @david-leifker, I finally took some time to update the PR by taking into account your previous remark, but also rebasing from very latest changes. I also realized that the automated generation (of But as you might have seen, in the Feel free to review again, so that it can go further. Thanks in advance 🙂 |
…ices' dependencies
4211358
to
869ea70
Compare
@gcernier-semarchy - Awesome!!! 🤩 Thank you so much for this! I am open to removing the alpha sorting, but not a requirement for getting this in. I am not sure why it was originally done that way, maybe the comparison logic or something. In any case, thank you for taking the time to update this! |
Purpose
Several unstable behaviours of the startup through
docker compose
were noticed ; often requiring todown
andup
again, after a first unsuccesfulup
, whether it was just after containers' creation or restart of Docker / host machine.This PR brings several improvements in corresponding
docker-compose*.yml
files (esp./quickstart
ones but then more extensively, trying to be consistent as much as possible), especially by:healthcheck
definitions (in/quickstart
ones, onlyelasticsearch
had one)service-A
onservice-B
more robust by relying on thosehealthcheck
・s, thanks to thedepends_on
's long syntaxservice-A
is started not only afterservice-B
is started (which is what short syntax only ensures), but also afterservice-B
is a:service_healthy
which means that its initialization (which can takes some time) completed, it's now running fine and therefore is actually ready to serve requests.service_completed_successfully
, which is particularly useful in the DataHub's context of*-setup
services/containersWhich somehow leads to such a dependency graph, e.g. with one of full "quickstart" examples:
Hence the re-ordering in the
docker-compose*.yml
files to somehow reflect this dependencies' ordering.This also enables to remove several
restart: on-failure
occurrences, that were poor workarounds compared to such more robust and deterministic solution (only kept the occurrences formysql
/mariadb
that appears to sometimes still fail just once at the very beginning when trying to create a socket, but not due todocker-compose*.yml
structure).Checklist