Skip to content

Feature Request: Init Containers for Docker Compose #6855

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
soapergem opened this issue Aug 19, 2019 · 13 comments · Fixed by #8122
Closed

Feature Request: Init Containers for Docker Compose #6855

soapergem opened this issue Aug 19, 2019 · 13 comments · Fixed by #8122

Comments

@soapergem
Copy link

soapergem commented Aug 19, 2019

Is your feature request related to a problem? Please describe.

Kubernetes has a concept called Init Containers which currently has no direct parallel in Docker Compose. There are certain applications designed to take advantage of Init Containers which cannot be correctly tested locally using Docker Compose due to this limitation.

Describe the solution you'd like

The Docker Compose docs includes an article on Startup Order for Containers but this doesn't capture the essence of this problem. That article stresses, repeatedly, that there is no good way for Docker Compose to determine when a container is truly "up," so dependent applications should be designed with checks. However Init Containers are a totally different animal, as they are designed to start up and terminate before the next container runs. Termination is, in fact, very easy to check for and should be thought of as a different use case entirely.

In other words, there should be a solution in Docker Compose to declare that a container is dependent on a special class of "Init Containers" which will start up, run, and terminate before launching the rest. All you'd really have to do is add a flag in the configuration - something like initContainer: true

Describe alternatives you've considered

The only "alternatives" aren't really alternatives at all. You can override the default entrypoint / commands of the dependent containers to wait some fixed interval, but this is sloppy guesswork and does not represent good application design. You could write your application code in the dependent containers to check that other containers (with certain names, maybe?) at one time existed and are no longer running... but you see where this is going, no one does this, and indeed no one should try to do this, because it's sloppy, unreliable, and bad design. On the other hand, it would be very easy to check, from a Docker Compose standpoint, that a container has terminated before launching the next.

Additional context

An example of an application that makes use of Kubernetes' Init Containers is vault-env, by Banzai Cloud. They have an in-depth article on it here. Essentially, they have an Init Container application which copies itself to a shared volume as a means of injecting secrets from Hashicorp Vault, and then the default entrypoint / command of the dependent container is updated to call the binary in the shared volume. Currently with Docker Compose this is not possible, because it doesn't wait for the first container to finish executing before trying to launch the second, so the application doesn't copy to the shared volume in time. Again, you can hack around this by changing the entrypoint to a long string involving bash -c sleep 3; /shared-volume/vault-env ... but this is an absolute mess.

Kubernetes has Init Containers. Docker Compose needs them too!

@cartermckinnon
Copy link

cartermckinnon commented Aug 20, 2019

I agree; init containers are an elegant way to achieve much more flexible, reliable lifecycle management for a stack. The alternatives are distasteful.

I would rather write a docker-compose.yml than a helm chart, but (in the interest of time) using a local Kubernetes cluster may be a better solution. You can set one up with a checkbox in Docker for Mac, and you'd be able to use all your k8s files instead of having to maintain a docker-compose.yml.

@ghost
Copy link

ghost commented Jan 9, 2020

Coming from Kubernetes I was just looking for this feature. The company am I working for now does not have a Kubernetes implementation yet. And for now just a Docker installation is fine. But I am missing this feature.

@cartermckinnon
Copy link

cartermckinnon commented Jan 9, 2020

Having had more experience with initContainers in production, there are some downsides. Specifically, if an initContainer is not successful, how often is it retried? In Kubernetes, the Pod enters CrashLoopBackoff (exponential backoff). This can cause your stack to be slow to stabilize when the initContainer finally succeeds. An Operator is one way to conduct a complex lifecycle without this sort of delay. I'm not sure what an "operator" would look like in a bare Docker setup -- a shell script? 😂

@blackandred
Copy link

This feature would be nice to see in compose. It is very helpful, so some kind of "Maintenance page" could be easily implemented in place of apps that are not up yet. Often applications do not wait for linked services such as database, cache or other linked application.

@AndreasGB
Copy link

I'd also love for this feature to be implemented as well.

My usecase is generating config files containing per-deployment secrets that two containers expect.

@dfcowell
Copy link

dfcowell commented May 17, 2020

I worked around this by defining a shared volume that my init container writes to and then sleeps forever, which my dependent containers poll.

version: '3.7'
services:

  bootstrap:
    build: ./
    volumes:
      - ./:/app
      - bootstrap:/bootstrap
    command: ./bootstrap.sh

  dependent-1:
    build: ./
    depends_on:
      - bootstrap
    volumes:
      - ./:/app
      - bootstrap:/bootstrap
    command: /app/start.sh dependent-1

  dependent-2:
    build: ./
    depends_on:
      - bootstrap
    volumes:
      - ./:/app
      - bootstrap:/bootstrap
    command: /app/start.sh dependent-2

volumes:
  bootstrap:

bootstrap.sh:

#! /bin/ash

if test ! -f "/bootstrap/bootstrap"; then
  lerna bootstrap

  # This provides an init-contianer-like experience on docker-compose, which
  # doesn't natively support init containers.
  today=$(date +"%Y-%m-%d")

  echo "${today}" > /bootstrap/bootstrap
fi

sleep infinity

start.sh:

#! /bin/ash

set -e

until test -f "/bootstrap/bootstrap"; do
  >&2 echo "Waiting for bootstrap - sleeping"
  sleep 3
done

cd "./packages/$1"

yarn start

Won't work for all init container use-cases, but it solves a subset of them.

@fksimon
Copy link

fksimon commented Jun 11, 2020

We have a few scenarios where we want to move to init containers.

  • certificate generation
  • database provisioning
  • common configuration

Right now we use the approach above which is to have a flag that tells the init container to not shutdown in compose environments, but run normal in kubernetes environment.

@blackandred
Copy link

blackandred commented Jun 11, 2020

I'm developing a docker (compose based) environment framework/template with features such as service discovery, automatic SSL, rolling updates (zero-downtime deployments) and I also plan to implement startup priority + init containers.

I'm preparing a 2.0 version rewritten in Python where I plan to implement init containers within month or two (possibly in 2.1) - subscribe for releases :)

https://github.com/riotkit-org/riotkit-harbor/tree/migrate_to_rkd

harbor

@stale
Copy link

stale bot commented Dec 9, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Dec 9, 2020
@Floeig
Copy link

Floeig commented Dec 9, 2020

I'm still interested specially for swarm

@stale
Copy link

stale bot commented Dec 9, 2020

This issue has been automatically marked as not stale anymore due to the recent activity.

@stale stale bot removed the stale label Dec 9, 2020
ojab added a commit to ojab/compose that referenced this issue Feb 16, 2021
Fixes docker#6855

See compose-spec/compose-spec#134

Signed-off-by: ojab <ojab@ojab.ru>
ojab added a commit to ojab/compose that referenced this issue Feb 16, 2021
Fixes docker#6855

See compose-spec/compose-spec#134

Signed-off-by: Slava Kardakov <ojab@ojab.ru>
ojab added a commit to ojab/compose that referenced this issue Feb 16, 2021
Fixes docker#6855

See compose-spec/compose-spec#134

Signed-off-by: Slava Kardakov <ojab@ojab.ru>
ojab added a commit to ojab/compose that referenced this issue Feb 16, 2021
Fixes docker#6855

See compose-spec/compose-spec#134

Signed-off-by: Slava Kardakov <ojab@ojab.ru>
ojab added a commit to ojab/compose that referenced this issue Feb 16, 2021
Fixes docker#6855

See compose-spec/compose-spec#134

Signed-off-by: Slava Kardakov <ojab@ojab.ru>
ojab added a commit to ojab/compose that referenced this issue Feb 16, 2021
Fixes docker#6855

See compose-spec/compose-spec#134

Signed-off-by: Slava Kardakov <ojab@ojab.ru>
ojab added a commit to ojab/compose that referenced this issue Feb 16, 2021
Fixes docker#6855

See compose-spec/compose-spec#134

Signed-off-by: Slava Kardakov <ojab@ojab.ru>
ojab added a commit to ojab/compose that referenced this issue Feb 16, 2021
Fixes docker#6855

See compose-spec/compose-spec#134

Signed-off-by: Slava Kardakov <ojab@ojab.ru>
ojab added a commit to ojab/compose that referenced this issue Feb 16, 2021
Fixes docker#6855

See compose-spec/compose-spec#134

Signed-off-by: Slava Kardakov <ojab@ojab.ru>
ojab added a commit to ojab/compose that referenced this issue Feb 16, 2021
Fixes docker#6855

See compose-spec/compose-spec#134

Signed-off-by: Slava Kardakov <ojab@ojab.ru>
ojab added a commit to ojab/compose that referenced this issue Feb 16, 2021
Fixes docker#6855

See compose-spec/compose-spec#134

Signed-off-by: Slava Kardakov <ojab@ojab.ru>
ojab added a commit to ojab/compose that referenced this issue Feb 16, 2021
Fixes docker#6855

See compose-spec/compose-spec#134

Signed-off-by: Slava Kardakov <ojab@ojab.ru>
ojab added a commit to ojab/compose that referenced this issue Feb 16, 2021
Fixes docker#6855

See compose-spec/compose-spec#134

Signed-off-by: Slava Kardakov <ojab@ojab.ru>
ojab added a commit to ojab/compose that referenced this issue Feb 16, 2021
Fixes docker#6855

See compose-spec/compose-spec#134

Signed-off-by: Slava Kardakov <ojab@ojab.ru>
ojab added a commit to ojab/compose that referenced this issue Feb 18, 2021
Fixes docker#6855

See compose-spec/compose-spec#134

Signed-off-by: Slava Kardakov <ojab@ojab.ru>
@shoce
Copy link

shoce commented Mar 25, 2021

I want to use this feature to provision database for staging environment. On every staging services (re)start a new copy from production database should be done, a database cleanup (from sensible client info) script should be run and only after this staging backend should start. All solutions i can think of feel weird to me as they need some bash scripting beside just docker-compose invocation. I see using an init container as the best fit for this task.

@cbrunnkvist
Copy link

cbrunnkvist commented Aug 20, 2021

@AndreasGB I think you can solve the problem by using a combination of volume from an init-container with a "are the files generated yet?"-healthcheck, and depends-on with condition: service_healthy in the service container(s).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants