Variable substitution

[gponsu]

This new feature is amazing, but I had a problem. I explain:

In our dev team, to enter in docker with a user with the same UID and GID that of the host machine, we made the following:

# docker-compose.yml
app:
  image: rails:4.2.4
  working_dir: /app
  ...
  extends:
    file: docker-custom.yml
    service: app

The docker-custom.yml file we have untracked of git, so everyone can define their own settings.

# docker-custom.yml
app:
  # Set your user and group id.
  # View your user id with command 'id -u' and your group id with 'id -g'.
  user: '1000:1000'
  environment:
    BUNDLE_JOBS: 4
    BUNDLE_PATH: /app/.bundle
    BUNDLE_APP_CONFIG: /app/.bundle
  # You can use env_file option instead of environment if you are using one file
  # to save environment variables
  # env_file: .env

With the new functionality of Variable substitution, I was excited, because we could spare us all this, and minimize it to this line in docker-compose.yml:

user: "${UID}:${GID}"

Unfortunately this does not work, because UID and GID are shell variables (not env variables), and os.environ do not get them.

Would it be possible get the shell envs too? Something like (I know nothing about python 😞):

os.system("set")

He had also thought of another possibility, if the option user is user: host or similar, internally get the uid and gid with:

os.getgid()
os.getuid()

And setup docker with this config for user.

Thank you very much for everything, and apologies for my horrible English.

[oinak]

👍

I have the exact same problem, as soon as I saw variable substitutio I tried for $UID to no avail

I could make the file ownership issues of containers mounting code folders go away.

Please please please enable shellvars along with envvars to be sunstituted on docker-compose.yml files.

Thank you for the excellent work.

[dnephin]

The only different is that UID isn't exported. I don't think it's correct to go reading un-expected shell variables.

I would suggest export UID.

[deeky666]

👍 I think this is a common use-case when mouting directories from the host and accessing them inside the container. Having to export extra environment variables on the host side is unnecessary extra configuration on the host side which more or less defeats this whole purpose.
Until using docker composer, I was evaluating the user and group id via id -u and id -g in the run command, which is not (yet) possible in docker composer configuration.
Some sort of solution to this would be very welcome.

[cookandy]

+1

Another use case would be trying to set the hostname of a container to the hostname of your host:

myapp:
  hostname: ${HOSTNAME}

My workaround right now is to set it before calling docker-compose

HOSTNAME=$HOSTNAME docker-compose up

[dvddarias]

I have exactly the same problem. In our team all you need to do to setup a new developer is
to run:

git clone ....
cd ....   
export UID   
docker-compose up   

We need the same user on the container and the host because of the big amount of Ruby on Rails commands that generate new files on the project volume. The sad part is that you have to export the UID variable on every new shell session for docker-compose up to work or add the variable to your shell startup script, which is sub-optimal on such a common use case.

This problem is specific to programmers using docker-compose as a tool:

Using docker CLI directly you don't have this problem because it is assumed that you run them within a shell and there you can use the variable $UID without exporting it.

Using docker through the API is also possible getting the current user id with os.geteuid() or something similar.

When using docker-compose i haven't find the way of doing it without running commands, so the "same user as the host" service requirement can't be made effective on any of the repository files.

A solution may be to setup some specific environment variable (let's say COMPOSE_UID) to the same UID as the user running the docker-compose command with os.geteuid() and then use it within the file as ${COMPOSE_UID}.

[mknapik]

👍
Same use case, handling shared directories at CI server.

[edannenberg]

Can't spam enough 👍 for this. UID/GID of the current host process should be provided by docker-compose in some way.

[ScreamingDev]

So as my #4725 got closed due to duplicates I need to ask the remaining question here:

How to solve/bypass this?

I guess it can't be left that way it is as long as there is no alternative.
It really is something that makes work a lot easier for developers.
Doesn't matter on production server but for dev and test server it helps.

Does the world build a wrapper around docker-compose having export UID in it?
Is it feasible by docker compose to solve this?

[durkode]

I'm also going to add my voice to this. Developing with Django from within a container and would love for this to work out of the box without having to export UID and GID.

I personally am not concerned how it is done, but a way for files written to the mounted volume in the container to match the host user without having to export UID and GID is my end aim. Currently running the export isn't a big deal, it would just be nicer if this wasn't the case

[sokratisg]

To be honest we sort of worked around this issue by doing something similar to:

1. Make sure UID is exported in each user's bash shell. Just create the following file and restart:
   /etc/profile.d/docker.sh

export UID

2. Add the necessary arguments in the docker-compose.yml file which is used for local development purposes:

...
  app:
    build:
      context: .
      args:
        username: $USER
        userid: $UID
...

3. Use these arguments in the respective Dockerfile which is also used solely for local development:

...
ARG username
ARG userid
RUN useradd --no-create-home --user-group --shell /bin/bash --home-dir $APP_HOME --uid $userid $username
RUN chown -R $username: $APP_HOME
...

Only drawback: UID conflicts with images that by default use the same one with any local user. (i.e. official node images have node user with UID=1000 and that conflicts with the initial UID a typical Ubuntu desktop assigns to its first user, tha one you're going to use post-installation)

[edannenberg]

Only drawback: UID conflicts with images that by default use the same one with any local user. (i.e. official node images have node user with UID=1000 and that conflicts with the initial UID a typical Ubuntu desktop assigns to its first user, tha one you're going to use post-installation)

Just use a static user like node in the image and change it's UID on container start as needed. The username is irrelevant when it comes to permissions, all that matters is a matching UID.

[sokratisg]

That is true if you use an image from a Dockerfile. Not true if you just want to start a node app by pulling the default image without any fiddling around or any custom shell script as your command.

[edannenberg]

@sokratisg You seem to use a local Dockerfile already, replacing useradd with usermod --uid $userid nodeshould do the trick.

[sokratisg]

@edannenberg excuse me but I think I may have been misunderstood, maybe my comment wasn't very clear. I'll try to give you an example.

Let's say your desktop user has UID=1000 (default for first user, at least on Ubuntu) and that you've written a node app that outputs or generates some local files. You want them to be written with the correct ownership (UID=1000) since you're also working locally and want to have read/write access on them without using sudo for your favorite editor/IDE. If you go with the solution I described then upon startup of the node container you will end up with a UiD conflict since official node images come pre-baked with a node user that has the same UID (1000).

This is what I was trying to describe; that this workaround is not universal and comes with some possibilities of a conflict so this issue is still relevant.