-
Notifications
You must be signed in to change notification settings - Fork 5.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Interpolate environment variables in docker-compose.yml #1377
Comments
How far do you want to go with these established UNIX standards? (FWIW, it's not a defacto standard, it's an actual standard.) As someone who occasionally accidentally tries to use POSIX parameter expansions in Dockerfiles, if they were at all supported in docker-compose.yml it would make me a happy camper. |
@kojiromike Hmm, so POSIX parameter expansion is actually what I was going for, but reading over the docs it looks like I misremembered the syntax/semantics. Edit: I've updated my thoughts on syntax in the description. |
I have been following the old thread and we urgently wanted to have this feature. Finally the pain was too big and we created a yaml preprocessor bahs script to substitute variables in POSIX style. it worked fine but eventually we stopped using it, because it had one issue. You have to run the preprocessor first and set all the parameters before you get the final solution. Now we are using the docker yaml extends feature. Because it allows us to checkin the actual configuration and just execute it on the target. We know better what is going to happen. Even though I was a supporter of docker-compose passing variables, I am now not so sure. As an ideal solution I would rather see docker extends done right. In a sense that would be a solution that fits both. So what is broken in docker extends? It is basically the fact that you have to write all entries in the inherited file. It is not a merge where you enter only what you want to override. Look at out actual example and how verbose it is. There are only two lines that matter.
So my recommendation fix docker extends an make it less verbose. You don't even have to write that much code as YAML provides all the functionality you need. If you stick with standard YAML the file could be analyzed or created by other tools and UIs. Take a look at YAML "node anchors" and YAML "file merge" it might be the perfect solution. FYI: this discussion continuous now on #1380 |
@Vad1mo I agree that |
Of course! I just wanted to highlight that this could be an easy and elegant alternative. |
My use case is to allow elasticsearch:
image: zinvoice/elasticsearch
volumes:
- $PWD:/app |
@mattes I believe that is already supported, I think |
@aanand As a PoC I did a dirty hackup of POSIX PE in Python. For the Saturdays. |
@kojiromike Looks great. Let me know if you plan to continue on it. |
@aanand I intend to, but it definitely has a few bugs right now (and I think it may have been a bad idea to use |
@dnephin how about |
@nafg Both of those are supported for the host path of a volume |
@dnephin interesting, b/c somehow I ended up with a directory named '$HOME'... |
@aanand Like the "${VARIABLE:default}" proposal, with global_extends (or "import") this would become rather powerful. Q: Would this allow to specify port number that is exposed to host? like - "${WEB_PORT:80}:80"? |
Yes, you'd be able to do that. |
I'd like to use vars in volumes together with
And I want to be able use
|
I hope it would be possible to interpolate variables as part of image name too |
But you can set |
@andrerom don't follow. According to docs that controls the following
|
ah, my mistake. Thought you meant <I_AM_DYNAMIC>:
image: nginx Dynamic image (and build) reference would indeed make a lot of sense. For instance switching between debug and non debug base containers for your programming language for instance would be a good use case for this. |
Additional use case (which might be what @Maxim-Filimonov has in mind): Being able to override which tag to use of an image, so you can use :latest by default, but change to easily test something else without changing yml file (needed for CI use cases basically). |
@andrerom that is exactly our use case 👍 |
Will this also work for things like??
|
@k0377 I don't think they will, because that's really something that's handled by the shell, but you could add the result in an environment variable and use that. In this particular case, the |
@aanand Why not use any of existing template engines that are already present? Jinja2 is there and works fine. As mentioned before -- implementing our own templating is non-trivial task (and regexps are not that cool) so that we should use already existing one, that proven to be solid. |
Alternatively we might use YAML ancors and references https://gist.github.com/bowsersenior/979804 But then we are limited on variables usage (inject variable name into middle of content). |
The default value should be there... Very useful... Developers and OPS are using either the docker logs or syslog... So, They usually have to create the default LOG_FORMAT shown below... We could just have the default value and only use it when switching to syslog... default:
extends:
file: base.yml
service: base-${LOG_FORMAT:docker}
labels:
- "net.company.npmjs.datacenter=${DATA_CENTER}"
- "net.company.npmjs.env=${ENV}"
- "net.company.npmjs.hostname=${HOSTNAME}"
- "net.company.npmjs.role=${NPMO_ROLE}"
- "net.company.npmjs.log=${LOG_FORMAT}"
base-syslog:
log_driver: syslog
log_opt:
tag: "{{.ImageName}}/{{.Name}}/{{.ID}}"
base-docker:
log_driver: json-file
log_opt:
max-size: "128m"
max-file: "4" When will this be available? I'm on Compose 1.7.0 and this is still not there :( |
Please please please give us default values! |
@marcellodesales: Perhaps you can take advantage of using a Also +1 on env vars. It is our major pain point with docker-compose nowadays. |
I would insist on my PR #3367 to be able to get certain values from host. :) |
@pataquets I don' think I want to create yet other override file... our |
+1 |
3 similar comments
+1 |
+1 |
+1 |
Any notice about use environment variables in docker-compose? +1 |
+1 |
FYI: Environment vars for docker-compose works as of 1.7.0. You can also set docker-compose default variables in |
Is there a way to set the service name as a variable ? Instead of writing this
We could write
My goal is to keep the same |
@LouWii you can use services:
site_db:
container_name: "${CONTAINER_NAME}"
image: mysql:5.7 or (compose-file format 2.1 and up) services:
site_db:
container_name: "${CONTAINER_NAME:-defaultname}"
image: mysql:5.7 But why not set the project-name? The project name is intended for that, as it prefixes/namespaces the container-names that are created to prevent conflicting with other projects on the same host. See https://docs.docker.com/compose/reference/envvars/#/composeprojectname |
@thaJeztah Thanks ! I'm still learning how Docker and docker compose work. Setting the project name seems to be a better option, it makes total sense in my use. |
Is there any way to script within the interpolated braces? For example |
You could pipe the file through Jinja or something...
…On Tue, Jan 24, 2017, 5:36 AM Sam A. Horvath-Hunt ***@***.***> wrote:
Is there any way to script within the interpolated braces? For example ${HOST_PORT
+ 1}.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1377 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAGAUN5ZrU39dnVVVASwIHr5mGqJFxh3ks5rVdRIgaJpZM4EMysO>
.
|
Am I able to escape
Currently this results in the host's PATH variable being interpolated and not the container |
is a docker-compose.yml file only one found ? |
@logicminds though I couldn't find it documented anywhere, I found environment:
PATH: "$$PATH:/home/appuser/.bundler/bin" |
@elquimista has a solution that I've found useful mhart/alpine-node#48 (comment) |
(I'm creating a fresh issue for this as the old one has accumulated rather a lot of baggage.)
It should be possible to pass in environment variables to the value of any* configuration entry in
docker-compose.yml
. A lot of people want to do it, it's good for portability and I'm satisfied it's not going to cause chaos.I have some reckons.
Required variables and optional defaults
It's useful to be able to specify that a variable that must be present in the environment, i.e. that Compose will refuse to run if it isn't. However, this will be a pain when you've got lots of them, so it should either be something you explicitly enable, or it should be possible to specify a default value.
The MVP implementation does not need to have either feature, but there should be a clear path to implementing both in a backwards-compatible way.
Syntax
There's a strong case for implementing an established standard, as long as it's not heavyweight - our requirements for functionality are minimal.
${VARIABLE}
- outputs empty string ifVARIABLE
is unset${VARIABLE-default}
- outputsdefault
ifVARIABLE
is unset${VARIABLE?}
- errors out ifVARIABLE
is unset#845 implemented a Bash-style
${VARIABLE:default}
syntax, which is similar to POSIX parameter expansion but slightly different.Implementation
Python's
os.path.expandvars
function implements the most basic case of POSIX parameter expansion:However, it's got at least 2 problems:
So far, #845 is the closest we've got, but I'm fundamentally wary of an implementation that relies on regular expressions. Templating is a non-trivial job, and people are going to put all kinds of broken stuff in, so we need something that's robust, strict and errors out with helpful messages. Two important requirements:
There may well be good Python implementations of Bash-like variable interpolation out there already - if not, creating something standalone would be far preferable to bloating the Compose codebase.
*Actually, are there any configuration keys for which we shouldn't allow interpolation?
The text was updated successfully, but these errors were encountered: