Dynamically source the UID and GID when starting devcontainer on Linux

[kkom]

Hey @Chuxel, thanks a lot for this example repo!

I have a question about the UID/GID settings mentioned here:

https://github.com/microsoft/vscode-remote-try-node/blob/07cbc3ee99d49b076ac46a85b1697efd1abc1841/.devcontainer/Dockerfile#L11-L16
https://github.com/microsoft/vscode-remote-try-node/blob/07cbc3ee99d49b076ac46a85b1697efd1abc1841/.devcontainer/devcontainer.json#L12-L14

Is it possible to set up VSCode to dynamically source these from the currently running user on the host machine?

The shell should normally be aware of them I think:

➜  bema4 git:(add-devcontainer) ✗ echo $GID
20
➜  bema4 git:(add-devcontainer) ✗ echo $UID
502

It would make the devcontainers really portable across users, which I think is the main reason for using them! :)

Best wishes,
Konrad

PS: CC @theomessin who was interested in that topic too

[Chuxel]

@kkom Yeah this was something I was wondering about myself. I think if we did this, what we’d want to do is dynamically run the appropriate commands to add a non-root user during spin up of the container. We could add the concept of user into devcontainer.json and then have an automatic value dynamically figure out the right non-root user that should be used to avoid this problem.

//cc: @chrmarti

[kkom]

Thanks! Looking forward to that @Chuxel !

I'm confused about one thing though:

I think if we did this, what we’d want to do is dynamically run the appropriate commands to add a non-root user during spin up of the container.

As far as I understand it, the issue is not about whether the user (of the OS inside the container) is root or non-root. It's rather about whether their UID/GID happens to coincide with that of the user running VSCode on the host OS.

Looking at https://medium.com/@mccode/understanding-how-uid-and-gid-work-in-docker-containers-c37a01d01cf can it be as simple as passing an appropriately constructed --user flag to the docker run (or equivalent) command?

I hope that in that way it's possible for the UID/GID to be different depending on whether we look from inside or outside the container!

Nevermind, I didn't read the article carefully / test it on an actual Linux system. The same UID/GID is used inside and outside the container. I now understand the need to dynamically create the user with a specific UID/GID inside the container!

This may become quite tricky for images where a specific user with very particular settings/permissions is created...

[Chuxel]

@kkom Yeah, this is a specific behavior of Docker on Linux - the pro/con with Linux is its literally the file that is bound without any abstractions. This means perf is native, but the permissions map directly right down to the IDs. On Mac and Windows it goes through a filesystem abstraction so you don’t have the ID issue - but that adds overhead.

I suppose we could allow you to specify a user to modify or create via some syntax. See if it exists, then tweak the UID/GID if so, otherwise create. (For the vscode-remote-try-node repo, the base node container has a node user for exactly this purpose - so it modifies the existing node user’s UID/GID. However, Python doesn’t have this kind of user, so vscode-remote-try-python creates the user instead.)

[chrmarti]

Files keep their numeric UID/GID. So we might need to update existing files inside the container when we change an existing user/group's id.

[chrmarti]

Related: #1860, #538

[chrmarti]

@Chuxel I'm adding containerUser and remoteUser to the devcontainer.json as we discussed.

I assume the automatic mode you referred to would use the local user's UID when on Linux and leave things untouched on Mac and Windows?

Instead of having automatic somehow as a configuration value:

What about having a variable that is the local user's UID on Linux and empty on Mac and Windows? That can be used in other places in the devcontainer.json, e.g., in environment variables, postCreateCommand and (future) build arguments.

E.g: "remoteUser": "${linuxUID}".

That would avoid us having to change /etc/passwd or chown existing files (we would have to guess where they are anyway).

[Chuxel]

@chrmarti Yeah, here's how I was thinking that automatic mode would end up working.

We have to use usermod / groupmod because of configuration files that can sit in the non-root user's home folder. We shouldn't create a new one if it already exists - like it does for node, R, and all of the dev containers in the vscode-dev-containers repo. This also has to happen before things like .bashrc execute for the user when we connect, so postCreateCommand is too late. You also need to run this as root and you may be starting the container as non-root, which means you can't modify the ID. This is why I'm telling people to modify the Dockerfile instead of the postCreateCommand.

So I am thinking we would exec as root and:

1. Look to see if there's already a user in the container with UID/GID 1000. You can get the user name by running awk -v val=1000 -F ":" '$3==val{print $1}' /etc/passwd or just manually inspecting /etc/passwd. Every Docker image I've seen that adds a non-root user uses 1000 since that's the default user ID on most desktop Linux installs.
2. If it exists, we run usermod / groupmod on it to change the IDs to whatever has been specified. This should modify the privs of /home/$USERNAME automatically, but we could also chown -R if push came to shove. This ensures that any existing config files and .bashrc settings specific to that user continue to function as expected.
3. If no such user exists, we create a new one.

On Alpine, we would need to verify the shadow package is installed to get the user/groupmod commands, but on other distros it's been there in the base images. Once I merge microsoft/vscode-dev-containers#140, any container that uses the common setup script for Alpine will get it. (See here - possible bugs aside.)

The update is what we're telling people to do for the node images right now. The fact this is basically needed 100% of the time on Linux is what makes me think it's important to support.

As far as Mac and Windows go, WSL2 support in Docker is changing how bind mounts work so it's current behaviors will change. It's theoretically possible it will behave like Linux - I don't believe it does now, but if your mount point is in WSL2, it very well might. On macOS, it's not needed, but for consistency and debugging, it might be worth just sticking with the same code path in all three.

If we did want to support being able to pass in a local UID/GID, I actually think supporting something like "$(id -u)" might be the right way to go since this also allows the execution of other things. We could scope it to the OS like $(linux: <command string>) (There's a GH issue for this - I forget the number.)

[chrmarti]

That is making a few assumptions about the container (usermod installed, UID 1000, no other files to chown than the home folder). Ideally we could avoid these. What if we added a build argument with the UID?

[Chuxel]

@chrmarti The painful thing here is having to tweak something just to get up and running. We'd also be trying to make this easier for people who really don't understand all this detail so that the majority scenario is things work by default.

In some cases, you may not have access to the original Dockerfile to add/modify a user when the build is not setup to do it or the user doesn't have access to modify the images that are generated. With #140, we're also now going to have base images that are pre-built with these UID/GIDs locked so it's immediately a user modification since config needs to land there as well. I'm also increasingly thinking about how VSO would be able to support some of this where things like runArgs wouldn't work and images are more likely to be pre-built or at least not rebuilt.

Now, to be clear, we'd clearly let people just specify a user - this is "smart mode" where we do our best to get a good default.

So the question is how we'd be able to enable this for the majority use cases. There's really two:

1. No non-root user
2. Non-root user at 1000

For the most part, if we don't do something here it just pushes it back into all 54 dev containers, samples, and docs. What would be better is if we can document how to resolve the exception cases - but the majority works. Right now the majority doesn't work.

If you're worried about user 1000, we could tweak the syntax a bit so that someone could have more control. For example:

"remoteUser": {
    "name": "vscode", // default would be "vscode"
    "uid": "${localUID}", // default would be 1000
    "gid": "${localGID}", // default would be 1000,
    "createIfMissing": false, // default would be true
    "modifyIfExists": true, // default would be true
}

Like I said, for Alpine, the shadow package adds it the commands above. so we can install it much like libgcc/libstdc++. If we wanted to we could do the same thing for apt-get and yum based images, but the base Debian, Ubuntu, and CentOS images have it.

Failing all that, I think it's totally ok to fail with a message saying the command is missing in the container. It should be a rare occurrence.

[chrmarti]

What would ${localUID} be on Windows?

How can we start the container with that user? Wouldn't we want to support passing the local UID as a build argument for that anyway?

We could have a Dockerfile in each definition using that build argument to create the user in a distro specific way. (Creating a user is different on Alpine than on other distros, IIRC.)

[Chuxel]

@chrmarti With WSL2 on Windows, it would be the WSL user which is 1000 by default. Generally, when in doubt, use 1000. :)

Alpine is just missing adduser by default so you use useradd which isn’t as sophisticated. Useradd is also available on other distros - we could look into sticking with that.

Unfortunately Alpine is missing usermod by default unlike the others.

[Chuxel]

How can we start the container with that user? Wouldn't we want to support passing the local UID as a build argument for that anyway?

How can we start the container with that user? Wouldn't we want to support passing the local UID as a build argument for that anyway?

@chrmarti Oops forgot to respond to this part.

Yeah, I was focusing on just what I'd been thinking on automatic - that is not a total fix and something that I'd assume we'd limited to remoteUser to keep complexity down. I think remoteUser is probably going to be the most common one given this would also apply to attach config and Docker Compose as well -- anywhere we do not control the build. BUT, I also completely agree we generally do need to support containeUser and something build arg related.

(As an aside, for buildArgs - we'd need to decide if we want this to be literally --build-args. If we did that, we'd probably need at least a property for --target to support multi-stage builds as well. In either case, we could support the property if we used docker-compose build for the compose scenario... up` doesn't build args.)

Perhaps the real full fix would be ${localUID} / ${localGID} usable from remoteUser, buildArgs, and containerUser. Then remoteUser could support something like the the variation above that basically is getting at the same idea as automatic. On true Windows (not the WSL side), the UID/GID would just fall back to 1000.

For automatic mode, I'm also thinking of all the feedback we've seen with existing Dockerfiles, external build systems where the user doesn't control, etc - places where our base definitions are not in use. It's just that #140 starts to introduce a similar problem for our own images - but the build system does inject Dockerfiles currently to the definitions to allow this to happen since there's no other option today.

[chrmarti]

Ok, what about this (looking for simplicity and flexibility at the same time):

• We'll add containerUser and remoteUser properties, both taking a user name as their value. The image will have to come with that user installed.
• On Linux and WSL2 we look up the local UID and GID and build a subimage in which we update /etc/passwd directly and chown everything in the remoteUser's home folder.

We'll do that with the following limitations:

• Only for single containers. (Not Docker Compose and not the Attach case for now.)
• Not when the remoteUser is root.
• Not when UID or GID are taken by another user or group in the container.

That has several advantages:

• containerUser can be the same as remoteUser because the container starts from a fixed up image.
• We avoid having magic ${...} variables.
• We don't need to add all options for creating a user (like home path and probably others).
• This works with base images that create a user.
• No need to add support for build args or other features for now. (We might still want these, but we don't tie this feature to any other.)

[Chuxel]

@chrmarti Seems like a reasonable starting point. How different is remoteUser for the attach and compose scenarios? I could see for compose not supporting containerUser to start since that would involve creating another Docker Compose file... but for the others, isn't the code path to spin up the user essentially the same?

Clearly the compose case came up in the discussions and attach has an open issue as well. (In fact, I think we heard about this first in those two scenarios.)

[chrmarti]

@Chuxel I have remoteUser prepared for all 3 cases. containerUser only for the single container case, as you suggest.

[chrmarti]

Also adding a property updateRemoteUserUID defaulting to true, so this can be turned off. (It won't do anything on Windows / Mac for now.)

[chrmarti]

Available in VS Code Insiders with the latest version of Remote-Containers (0.86.0).

[MikeZhu92]

Hi, @chrmarti
I'm really excited to see this new feature!
Just a little bit confused by the statement:

We'll do that with the following limitations:
Only for single containers. (Not Docker Compose and not the Attach case for now.)

But issue #1860 is exactly for the attach case.

Would there be a future update to support this feature?

[MikeZhu92]

I've tried the Remote-Containers (0.86.0).
My procedure is the following (host env ubuntu 16.04):

1. start docker container with my docker scripts (which will add my current usr to the group)
2. use remote-container to connect to that container
3. open the attached configuration file and add:
   "remoteUser": "xxx"
   After reconnect to the container and open a terminal, it seems the current user is still root.

The docker scripts looks like this: https://github.com/ApolloAuto/apollo/blob/r3.0.0/docker/scripts/dev_start.sh

Any help or suggestion would be great.

[chrmarti]

@MikeZhu92 "remoteUser" is available for all setups. The limitations I listed only apply to the automatic updating of the UID and GID of the remote user (which is only needed on Linux).

I checked this works for me with the attach config. Please check the user exists and you picked the right config file (use F1 > Remote-Containers: Open Container Configuration File while attached).

[MikeZhu92]

Thank you @chrmarti for your help!
I realize this feature currently require vscode insider version.
It works great!

[Yivan]

@chrmarti Thanks for this very good news about "remoteUser". We were not able to use remote container because of this problem. Having the IDE creating files as root messed the permission inside the container.

I got to 2 mains questions please:

1-
Just to be sure, this is working like that, if a container is already runned as root user (the service inside need to be CMD as root) :

• on the local vscode we "Connect to Container"
• it install vscode-server on the container... as root so ? (this is the important point...)
• we edit the container file xxxx.json to add "remoteUser": "www-data" (this is our user for project code files)
• and after all files created/manipulated/etc. are done by www-data ? And terminal is log under www-data ? The user which install/run vscode-server has no importance ?

2-
What hapen behing the scene ? Is it vscode-server which write the files (local vscode -> remote vscode-server -> filesystem) or a "docker exec" commands (so directly from local vscode -> container filesystem) ?
What happen for extension writing directly files (i mean they do it from vscode-server i think as we install extentions on it...) ?

Side question please, why having choosed "remoteUser", instead of "execArgs" for instance (so consistant with runArgs, add more freely configurable) ?

Thanks a lot for taking time on this issue, so important to make remote-container usable !
Best regards.

[chrmarti]

On 1: You will need to reload the VS Code window after adding "remoteUser": "www-data" to the configuration. That will then install the VS Code Server under the www-data user's home folder. You might want to check if that is a regular folder the user can write to (since this is a system user).

On 2: This is the VS Code Server writing the files. The server runs under the "remoteUser" and so do all extensions. So files created by extensions will also be owned by that user.

We could still introduce "execArgs", but specific properties tend to be easier to use on the user's side.

[Yivan]

Ok thanks for explaining this and helpful details about installation.
So all should be ok ! Really great.
Best regards.