Feature request: docker stack deploy pass environment variables via cli options

[dhanvi]

From @dhanvi on March 10, 2018 18:54

Few people might have a use case to pass environment variables with a flag like -e.

docker stack deploy -c file.yml -e name=value stack_name

Labels: area/cli kind/feature

Copied from original issue: moby/moby#36554

[dhanvi]

From @AkihiroSuda on March 11, 2018 14:16

CLI is out of the scope of this repo.
Please refer to https://github.com/docker/cli

[vdemeester]

@dhanvi Thanks for the issues. That said, I have a small question about the feature request

Few people might have a use case to pass environment variables with a flag like -e.

I can see two different features requested here :

• setting environment variables to be used for interpolation (${name}), this is then equivalent to env name=value docker stack deploy
• setting environment variables to each services in the stack

@marcel-steinbach-mpf @jeanadrien @mallchin it would be helpful for us to know which one is it 👼 🙏

[dhanvi]

@vdemeester I am just looking for the option docker stack deploy -e name=value stack_name, I gave an example of -c for reference here. (equivalent to env name=value docker stack deploy is what I have in mind.)

I am assuming that the variable in here docker stack deploy -c file.yml -e name=value stack_name will only be applicable to the service in the compose file. (It doesn't makes sense for to add a variable for the whole stack, my assumption is that variables are scoped to the service only, can they be scoped for the whole stack ?)

[thaJeztah]

@vdemeester's question was because there are two "levels" of environment variables:

1. Environment variables that are substituted when processing the docker compose file
2. Environment variables that are set on services.

For example, taking this docker-compose.yml:

version: "3.6"
services:
  one:
    image: "nginx:alpine"
    environment:
      foo: "bar"
      SOME_VAR:
      baz: "${OTHER_VAR}"
    labels:
      some-label: "$SOME_VAR"
  two:
    image: "nginx:alpine"
    environment:
      hello: "world"
      world: "${SOME_VAR}"
    labels:
      some-label: "$OTHER_VAR"

env SOME_VAR="I am some var" OTHER_VAR="I am other var" docker stack deploy -c docker-compose.yml envie

Inspecting environment variables on service one (envie_one):

docker service inspect --format='{{ json .Spec.TaskTemplate.ContainerSpec.Env}}' envie_one

Shows:

[
  "SOME_VAR=I am some var",
  "baz=I am other var",
  "foo=bar"
]

And its labels:

docker service inspect --format='{{ json .Spec.TaskTemplate.ContainerSpec.Labels}}' envie_one

Shows:

{
  "com.docker.stack.namespace": "envie",
  "some-label": "I am some var"
}

Inspecting environment variables on service two (envie_two)

docker service inspect --format='{{ json .Spec.TaskTemplate.ContainerSpec.Env}}' envie_two

[
  "hello=world",
  "world=I am some var"
]

And its labels:

docker service inspect --format='{{ json .Spec.TaskTemplate.ContainerSpec.Labels}}' envie_two

Shows:

{
  "com.docker.stack.namespace": "envie",
  "some-label": "I am other var"
}

My take on this

Environment variables that are substituted while processing the docker compose file are already possible;

• you can set them up-front (export MYVAR=myvalue)
• you can set them inline (env MYVAR=myvalue)

What's not supported (yet?) is the use of an .env file when deploying stacks: moby/moby#29133. I'd personally be good with adding support for .env files, so that "environment" variables are set when deploying a stack, without the requirement to export those variables up-front (or set them manually on each run).

What I don't think should be done is add an --env foo=bar option that would set the foo=bar environment variable on each service in the stack. Reasons I think we should not be adding that:

• The environment variable wil be set on every service in the stack: in many cases you may want to only set them on some services, but not on all
• Changing those values will re-deploy every service in the stack (likely not desirable)
• The docker-compose file is intended to be describing the definition of your services: by manually changing the service configuration (i.e., adding environment variables), the definition of the service in the docker compose file no longer matches the actual service. If you want an environment variable (or any other option) to be set on a service; edit the docker-compose file, and add it to the services.

What if I want an environment variable to be set on service(s), but don't store it's value in the docker-compose file?

Use variable substitusion, and set the environment-variable in your shell (having support for an .env file would help with this):

version: "3.6"
services:
  one:
    image: "nginx:alpine"
    environment:
      SOME_VAR:

Now, deploy the stack using either:

export SOME_VAR=some-value
docker stack deploy -c docker-compose.yml foobar

Or

env SOME_VAR=some-value docker stack deploy -c docker-compose.yml foobar

What if I want additional environment variables to be set (in some situations)

For example: during development, I want to set some additional environment variables, but in production, I don't want those set (or vice-versa).

Use an "override" file, by deploying multiple docker compose files:

docker-compose.yml:

version: "3.6"
services:
  one:
    image: "nginx:alpine"
    environment:
      somevar: "somevalue"
  two:
    image: "nginx:alpine"

docker-compose-dev.yml:

version: "3.6"
services:
  one:
    environment:
      debug: "1"
  two:
    environment:
      development: "yes"

Specify both files when deploying the stack:

docker stack deploy -c docker-compose.yml -c docker-compose-dev.yml mystack

And see that the additional environment variables are set;

docker service inspect --format='{{ json .Spec.TaskTemplate.ContainerSpec.Env}}' mystack_one
["debug=1","somevar=somevalue"]

docker service inspect --format='{{ json .Spec.TaskTemplate.ContainerSpec.Env}}' mystack_two
["development=yes"]

[thaJeztah]

Having said the above; I'm interested to hear what the exact use-case is that people here are looking for. Perhaps you can explain what you want to use this for, and show examples how it will be used.

[jgoodall]

My use case is the first one you describe:

1. Environment variables that are substituted when processing the docker compose file

Specifically, I have two files dev.env and prod.env for development and production that each export a bunch of variables. And when I run docker stack I do it like this:

$ . ./dev.env && docker stack deploy -c run.yml torro-run

Or,

$ . ./prod.env && docker stack deploy -c run.yml torro-run

Within those env files I also have the following for variables in common across prod and dev:

. ./shared.env

As long as the user remembers to source the file, it works great. It would be a little nicer to have support for multiple .env files, but the approach we have now seems to work.

[thaJeztah]

Thanks for the additional information!

Okay, so looks like the desired feature would be to (for a start) make docker stack deploy read an .envfile from the current directory (same as docker-compose up does).

From moby/moby#29133 (comment), I see that @dnephin was open to discuss that option, but looks like he had some reservations;

This is by design. The .env support is a feature of Compose, not of the file format.
We can discuss adding this for a future release, but I don't know if it's really the best option.

Some questions, as I know there are various limitations to the current .env files;

• Should we stick to the simple key=value syntax (no special treatment of quotes (", '") in values)?
• Same: an .env file thus will not be useful for Bash/shell (i.e., export foo=bar won't work)
• Same: variables inside the file won't be expanded (FOO=${BAR}-something will literally use ${BAR}-something as value for the FOO env-var)

In addition to the above, I think it would be useful to print an informational message that docker read variables from that file (so that it doesn't catch a user by surprise).

[mallchin]

I'm busy with GDPR right now but will revisit this next week. I had a specific use case and will see how it fits in with the above. I expect setting environment variables on the command line might work for me but will confirm shortly.

[gbjbaaha]

I have a slightly different scenario:-

I am using a standard docker-compose.yml file to define my stack. I wish to build the containers via "docker-compose build" and then deploy into my swarm via "docker stack deploy".
Due to way our CI/CD pipeline is setup, we explicitly tag all of our Docker images with the product release name and version. As Docker Compose supports environment variable substitution, I can define the "image" tags within the "build" section as follows:-

my-service-name:
    image: my-service-name:${CODENAME}-${TAG}
    build:
      context: ${GIT_ROOT}/my-service-name-repo

The result of the build is a suitably tagged image.

What I would like to happen is for the same docker-compose.yml file to support environment variable substitution via the call to "docker stack deploy" when deploying my stack such that the correct images are deployed. My workaround for now is to set those environment variables via the shell, but it would be good to see a way to set these on the command line call.

Happy to take advice if there is a better or more Docker compliant way of achieving this.

[b01]

I'm deploying from CircleCI and have the same need as @gbjbaaha. Unfortunately it not as easy as exporting the variable to the environment because in CircleCI the docker environment is remote, so I'd have to ssh in and export them there. This requires a lot of setup because again the machine where the docker commands are executed is remote, see setup_remote_docker

[kajmagnus]

I'm also using a tag like @gbjbaaha , in the .env file: VERSION_TAG=v1.2.3 and in docker-compose: image: organization-name/image-name:$VERSION_TAG. And, to upgrade all images, a cron job updates the version in .env, then does docker-compose down and then up, meaning, all services restart with the new image versions (so they don't need to be compatible with old versions of the other services API:s — since everything upgrade at once to the same new version).

@gbjbaaha if I understood you correctly, replacing a $TAG variable, works, if one does this: ?

VERSION_TAG=v1.2.3 docker stack deploy -c stack-file.yml

So, this: VERSION_TAG=v1.2.3 docker stack ...
is "the same as" an .env file with VERSION_TAG=v1.2.3, and docker-compose ...?

Hmm maybe the script could update the version number directly in the stack-file.yml as well, instead of indirectly via the .env file. The version number would then be hardcoded at many places in the stack-file.yml ... meaning, a little bit duplication, and maybe that's fine.

[gbjbaaha]

Yes, variable substitution via environment variables set in the shell as suggested by @kajmagnus works correctly.

[kajmagnus]

Ok thanks @gbjbaaha for the info :- )

[yisraeldov]

There is this work around docker-compose config | docker stack deploy --compose-file - test That will create your docker-compose.yaml but replace with the variables from .env

[ejyanezp]

Another use case:

How to force a container's (task) environment variables to receive its values from the environment variables of the host in which it is running? (This is for each task in a "docker stack deploy" running in several swarm nodes)
I have a service defined in a "docker-compose.yml" file, like:

  openlegacy-config-server:
    image: credicorpbankinnovacion/config-server:4.2.4.b1914
    environment:
      - SPRING_PROFILES_ACTIVE=docker,elk-logs
      - OL_ENCRYPTION_KEY=changeme
      - OL_ECOSYS_DB_SRV=${OL_ECOSYS_DB_SRV}
      - OL_ECOSYS_SRV=${OL_ECOSYS_SRV}
    build: openlegacy-config-server/
    networks:
      - ol-network
    # This deploy config is for QA & PROD (those envs use Docker Swarm)
    deploy:
      mode: replicated
      replicas: 2
      placement:
        constraints:
          - node.role == manager
    depends_on:
      - openlegacy-service-discovery
    restart: always

This service is to be deployed unto two nodes of a docker swarm using docker stack deploy
For each node I have defined the environment variables:

NODE A:
OL_ECOSYS_SRV=192.168.10.149
OL_ECOSYS_DB_SRV=192.168.10.123

NODE B:
OL_ECOSYS_SRV=192.168.10.143
OL_ECOSYS_DB_SRV=192.168.10.117

But the expressions ${OL_ECOSYS_SRV} and ${OL_ECOSYS_DB_SRV} are evaluated at the moment of entering the command "docker stack deploy" in node A, so the different IP address effect depending on the node if not achieved, and all containers (tasks) in both nodes are created with the same IPs of node A because the variables are evaluated at the service creation moment and not at the task creation moment.

How could I pass to the containers in node A and B a different local dns configuration (/etc/hosts) depending on the node itself? Is there another approach to solve this problem?
This must work for a single but replicated docker swarm service.

Which approach would you use?

[anshupitlia]

There is this work around docker-compose config | docker stack deploy --compose-file - test That will create your docker-compose.yaml but replace with the variables from .env

Good workaround, but will not work in cases docker-compose is not installed, only docker is

[yamac-kurtulus]

@ejyanezp

Hey man, did you manage to find an approach for your use case?

[moracabanas]

There is this work around docker-compose config | docker stack deploy --compose-file - test That will create your docker-compose.yaml but replace with the variables from .env

Good workaround, but will not work in cases docker-compose is not installed, only docker is

My entire workflow is failing at this point with external networks defined on the docker-compose giving an error only if this method is used instead hardcoding configs. So this is failing parsing external networks or in any other way:
docker-compose config -> the parsed definition contains your external-network-name: null and then docker deploy prints this error as a consecuence.

Additional property name is not allowed

I really don't know how to transition from docker-compose to Swarm, where you set a development and production scenarios.
How can you make this possible, when your docker-compose workflow is meant to reading ${VARIABLES} from .env file but you are not intended to use this approach when you are full on docker Swarm?

[moracabanas]

I just read about secrets but I want to define not secret configs from .env like hostnames and use secrets like passwords etc.. without the need of create secret manually for every ${VARIABLE} i was used to put in .env file when docker-compose times.

[moracabanas]

There is this work around docker-compose config | docker stack deploy --compose-file - test That will create your docker-compose.yaml but replace with the variables from .env

Good workaround, but will not work in cases docker-compose is not installed, only docker is

My entire workflow is failing at this point with external networks defined on the docker-compose giving an error only if this method is used instead hardcoding configs. So this is failing parsing external networks or in any other way:
docker-compose config -> the parsed definition contains your external-network-name: null and then docker deploy prints this error as a consecuence.

Additional property name is not allowed

I really don't know how to transition from docker-compose to Swarm, where you set a development and production scenarios.
How can you make this possible, when your docker-compose workflow is meant to reading ${VARIABLES} from .env file but you are not intended to use this approach when you are full on docker Swarm?

My workaround:

deploy.sh:

#!/bin/sh
export $(cat .env) > /dev/null 2>&1; 
docker stack deploy -c docker-compose.yml ${1:-STACK_NAME}

. deploy.sh <stack_name>

(stack_name optional if previously defined in .env)

[ejyanezp]

@ejyanezp

Hey man, did you manage to find an approach for your use case?

Nope :(

[membersound]

How can I pass an environment variable for the image registry from gitlab_ci to docker-compose for use with docker swarm?

In my following example, the value $APP_SECRET is successfully resolved from a .env file, while $REGISTRY is not!

.gitlab_ci.yml:

deploy:
  stage: deploy
  script:
    - ssh -i $SSH user@$SERVER "
      env REGISTRY=$REGISTRY &&
      docker stack deploy -c docker-compose.yml docker-test --with-registry-auth
    "

(even setting the env REGISTRY=... does not work here. And also not when completely removing that statement).

docker-compose.yml:

version: '3.7'
services:
  docker-test:
    image: $REGISTRY/docker-test:latest
    env_file: '.env'
    environment:
      - app.secret=$APP_SECRET

.env:

REGISTRY=my-repo.registry.com
app.secret=set by env file

When I start this: docker stack deploy -c docker-compose.yml docker-test --with-registry-auth

All I get is: Invalid reference format.

This seems to be due to the $REGISTRY in the image tag. When I manually set the correct value in the docker-compose like image: my-repo.registry.com/docker-test:latest, everything works fine. Though there is still an $APP_SECRET variable to be resolved from .env, and that actually still works!

So why is does variable replacement works in environment tag, but not in image tag?

[thaJeztah]

docker stack does not use .env files to set environment variables for the current environment (that feature is specific to Docker Compose). You need to make sure those environment variables are set when running the command, so that they're evaluated.

Using this compose file:

version: '3.7'
services:
  docker-test:
    image: $REGISTRY/nginx:alpine
    environment:
      - app.secret=$APP_SECRET

And a stack deploy with the REGISTRY and APP_SECRET environment variables set:

REGISTRY=docker.io APP_SECRET=hello docker stack deploy -c docker-compose.yaml mystack

docker service inspect --format='{{json .Spec.Labels}}' mystack_docker-test
{"com.docker.stack.image":"docker.io/nginx:alpine","com.docker.stack.namespace":"mystack"}

docker service inspect --format='{{json .Spec.TaskTemplate.ContainerSpec.Env}}' mystack_docker-test
["app.secret=hello"]

[membersound]

But why then does env_file: '.env' set the $APP_SECRET in environment tag successfully?

[thaJeztah]

The env_file option in the compose-file is the equivalent of docker run --env-file, which allows setting environment variables on the container itself (it's the equivalent to setting --env FOO=bar --env BAR=baz on a container). Those environment variables are set on the container itself, but not used to interpolate environment variables in the compose-file itself.

(docker compose unfortunately also has a top-level --env-file flag, which is used to specify an alternative location for the .env file, which makes its quite confusing, but as that flag has existed for quite some time, it's difficult to remove it without potentially breaking many users)