Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can this work with v3 of docker compose? #3

Closed
ghost opened this issue Sep 12, 2017 · 7 comments
Closed

Can this work with v3 of docker compose? #3

ghost opened this issue Sep 12, 2017 · 7 comments

Comments

@ghost
Copy link

ghost commented Sep 12, 2017

Since docker compose won't support the condition form of depends_on can something like healthcheck works yet?

reference:
https://docs.docker.com/compose/compose-file/#depends_on

@peter-evans
Copy link
Owner

peter-evans commented Sep 13, 2017

No, unfortunately this does not work with v3 of docker compose. From v3 they are not supporting the condition syntax, as you have pointed out. Docker want to move away from docker-compose to the integrated docker stack and Swarm services. When designing how these should work it seems that Docker have made a conscious decision not to support features that wait for containers to be in a "ready" state. They argue that applications depending on other systems should be resilient to failure.

This is a couple of paragraphs taken from the documentation here that explains their reasoning.

The problem of waiting for a database (for example) to be ready is really just a subset of a much larger problem of distributed systems. In production, your database could become unavailable or move hosts at any time. Your application needs to be resilient to these types of failures.

To handle this, your application should attempt to re-establish a connection to the database after a failure. If the application retries the connection, it should eventually be able to connect to the database.

So the options are:

  1. Keep using the v2 format
  2. Use a tool like wait-for-it or dockerize
  3. Make your application resilient to database failure and able to retry it's connection on startup

There is some discussion about this issue here and here.

jgondron added a commit to ndlib/curate_nd that referenced this issue Jul 26, 2018
… disambiguate this from the project_root/app directory

- Added sync excludes for common unneeded directories that may exist in dev workdirs
- Changed docker-sync to sync the full project directory
- Changed Dockerfile.rails to bundle in into a separate directory. This is to preserve any changes bundler will make to the Gemfile.lock, so that we can copy it back to the host once sync is started. I'm running an additional command at the end of the Dockerfile to copy of the lock file back to project_root in the case when someone is not using docker-sync
- The healthy dependency condition is no longer available in v3 of docker compose. Copied the wait-for-it script referenced in peter-evans/docker-compose-healthcheck#3 and added a rails entry script that will handle all the things associated with first starting the rails app, including waiting for mysql to start.
jgondron added a commit to ndlib/curate_nd that referenced this issue Jul 26, 2018
… disambiguate this from the project_root/app directory

- Added sync excludes for common unneeded directories that may exist in dev workdirs
- Changed docker-sync to sync the full project directory
- Changed Dockerfile.rails to bundle in into a separate directory. This is to preserve any changes bundler will make to the Gemfile.lock, so that we can copy it back to the host once sync is started. I'm running an additional command at the end of the Dockerfile to copy of the lock file back to project_root in the case when someone is not using docker-sync
- The healthy dependency condition is no longer available in v3 of docker compose. Copied the wait-for-it script referenced in peter-evans/docker-compose-healthcheck#3 and added a rails entry script that will handle all the things associated with first starting the rails app, including waiting for mysql to start.
jgondron added a commit to ndlib/curate_nd that referenced this issue Jul 27, 2018
Some notes about the rationale behind some of the decisions made in implementing this:
  - Initially got a version of this working with rails in a container with a mounted volume for the app code, but it's really really slow. Appears to be a known problem when using a mounted volume on OSX https://forums.docker.com/t/file-access-in-mounted-volumes-extremely-slow-cpu-bound/8076/276. Added docker-sync configuration and instructions on how to install/use.
  - Building the image will copy the full application code into /project_root within the container instead of /app. This is in order to disambiguate this from the app directory within the root of the project. Most of the docs for docker-sync will reference /app, so just be aware of this
  - Added sync excludes for common unneeded directories that may exist in dev workdirs. If you do begin using this, it would be a good idea to clean out any temp files you have in your directory to speed up build/syncs
  - There are some shenanigans I have to play with the bundled gems and the Gemfile. The rails container will bundle gems into a /bundle directory. This is to preserve any changes bundler will make to the Gemfile.lock, so that we can copy it back to the host once sync is started. I'm running an additional command at the end of the Dockerfile to copy of the lock file back to project_root in the case when someone is not using docker-sync
  - Had to add a USER env to the compose to reproduce the "developer is an admin" behavior that we all expect
  - Added install of fits (and the java runtime) to the rails image so that rake curatend:ci would run correctly
  - The rails_entry script will additionally create the test db for you to simplify running tests in the container. We may want to remove this in the future and separate the test scripts/compose/etc from dev
  - The rails app is not very fault tolerant if, at start time, mysql isn't ready. Initially I added a healthcheck on mariadb and a rails depends_on mysql healthy condition, thinking this should prevent rails from starting until mysql is healthy. Unfortunately, the healthy dependency condition is no longer available in v3 of docker compose. Instead, this required copying the wait-for-it script referenced in peter-evans/docker-compose-healthcheck#3 and adding a rails entry script that will handle all the things associated with first starting the rails app, including waiting for mysql to start. Left the healthy script there since it may still prove useful
  - Broke out the readme into separate readme files now that there is an entirely different set of instructions when using Docker and Docker-sync. README.md will continue to give instruction on how to install/run locally on a host. README_DOCKER.md will contain instructions on how to do these things with Docker
  - Finally, I just want to again note that this only solves the problem of using Docker in development. We need to do things differently if we want to use Docker for CI and other deploys. There are pieces of this that can be reused, but how we separate those out into a common base dockerfile/compose files would need to be discussed

Ideally, adding this would have required making no changes to existing app code, but I did have to make several changes:
  - Running rails app under compose needs to use network names for external services, ie: specs and configs can't assume things are on localhost. Changed solr/fedora/database configs to use env if available, but fallback to localhost if not there to continue supporting development on the host without Docker (such as in the Travis ci atm)
  - Had to add config.allow_http_connections_when_no_cassette to spec_support. The config was already ignoring localhost, which worked when developing locally on a host. In docker this won't work because it will make calls to the "jetty" and "mysql" hosts and will fail because it couldn't find a cassette that matches. Likewise, changed the id_service_spec to just match on path and method for the cassette so that it would successfully match when a spec made the call to jetty.
  - Added a byebug initializer so that we can debug remotely, since this is technically a remote machine
  - Added .docker-sync to gitignore for cases when someone runs the sync as a daemon
  - Now have multiple Dockerfiles, so had to do a little clean up of the file locations. Note: This docker-compose is still for development only. We'll need to break this out later if we ever decide to use this for ci/staging/etc
@peter-evans peter-evans pinned this issue Dec 14, 2018
@peter-evans peter-evans unpinned this issue Dec 14, 2018
@satrapu
Copy link

satrapu commented Aug 13, 2020

You could invoke Docker API, as documented here.

@blevine
Copy link

blevine commented Sep 17, 2020

You might want to make a note in your README that this solution doesn't work as of docker-compose V3

@peter-evans
Copy link
Owner

peter-evans commented Sep 19, 2020

Added a note to the readme about this issue. Thanks.

@shane-axiom
Copy link

shane-axiom commented Mar 4, 2021

This is supported in newer versions of docker-compose which support the compose specification. Verified with docker-compose 1.28.4 here.

cc @unparalleled-js

@peter-evans
Copy link
Owner

peter-evans commented Mar 4, 2021

@shane-axiom Thanks for letting me know! I had no idea about this change.

Docker Compose 1.27.0+ implements the format defined by the Compose Specification. Previous Docker Compose versions have support for several Compose file formats – 2, 2.x, and 3.x. The Compose specification is an unified 2.x and 3.x file format, aggregating properties across these formats.

ref: https://docs.docker.com/compose/compose-file/

It aggregates properties across all the past formats, so you can now use the 2.x condition syntax with the latest version of compose. Perhaps it only works if you don't specify the version at the top of the compose file? I see that the version property is optional now. Anyway, I'll test it out and update this repository.

@peter-evans
Copy link
Owner

peter-evans commented Mar 4, 2021

Tested and updated the readme. Basically, it doesn't matter what the version property is set to anymore because it's not used. Recent versions of Docker Compose follow the Compose Specification which allows features from previous versions of Compose to be used together.

I think we can consider this issue resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants