Non-root users, shared volumes, & RStudio

[cboettig]

Docker does some kinda unexpected things when it comes to handling non-root users and shared volumes. With the rstudio image, we have to create a non-root user with a password that can be set at runtime for security purposes, which we do by running a custom userconf.sh on startup.

RStudio wants to start in the users home directory (regardless of the value of --workdir). First crazy thing is: if the userconf.sh script does not run a chown command (as we have currently), that directory is not owned by the user rstudio (or other user specified with the environment variable $USER. Lacking write permissions, RStudio cannot even start up. Surprisingly, the directory /home/rstudio is owned by the user docker!

If we run chown -R $USER:$USER /home/$USER at the end of the userconf.sh script, RStudio can now log in. But if we have linked a home directory, chowning to anything other than the the docker user (Why the docker user? Just the first non-root user added to docker?) means that the permissions on the linked directory outside of the container will be changed to some other user. Attempting to match the host user name with the $USER value on the container isn't any different than just leaving the container $USER as docker.

The only way out of this that I see is to have only one user (e.g. hardwire the user docker, and just set the password interactively. breaks previous instructions and feels inelegant). Can't be sure if this problem existed before we added a default user docker, perhaps removing that would avoid these issues?

Anyway, curious if you might confirm any of this behavior, and I'll see what I can come up with.

[cboettig]

Worked up a more minimal example of the issue here: http://stackoverflow.com/questions/26500270

Should we:

• drop the default docker user?
• drop the custom user (everyone must log into rstudio as user docker, but with custom password?)
• Have both, and tell people to log in as docker if using linked volumes?

none of these feel like good solutions

[eddelbuettel]

All good questions. Maybe the Docker group / mailing list needs the Q as well...

[cboettig]

Okay good idea, putting the link here for future reference: https://groups.google.com/forum/#!topic/docker-user/VFdFuZ4Ze_A

[eddelbuettel]

Very polite of you :) The main trouble with that list may be that you get as ignored as on SO :-/

[cboettig]

yeah, shoulda started off with 'look at this stupid docker behavior'.
Kinda feel like the behavior already makes some sense to me: it seems the
kernel doesn't care about user names, files are just owned by user id
numbers like 1000, 1001, etc. The fact that I can link volumes and not get
my permissions screwed up normally seems to be pure chance that on my host
account, my user is also id 1000. If I make a new user on my host ubuntu
machine, he gets id 1001. If I log into a container as the second user
created on the container with a linked volume and write some files, all of
a sudden they are owned by the new user, not the user whom I am logged in
as!

One one hand that makes sense if the kernel just cares about the id
numbers, but it seems like this could create real problems if your host
machine had lots of user accounts configured.

On Wed, Oct 22, 2014 at 9:14 AM, Dirk Eddelbuettel <notifications@github.com

wrote:

Very polite of you :) The main trouble with that list may be that you get
as ignored as on SO :-/

—
Reply to this email directly or view it on GitHub
#50 (comment).

Carl Boettiger
UC Santa Cruz
http://carlboettiger.info/

[eddelbuettel]

Yep. Seems like an issue that shoulda coulda woulda hit other users / developers too.

[cboettig]

fwiw, I've made a few tweaks to the Dockerfiles in 43d6203 to work around this issue for the time being.

• In r-base, when we useradd docker I've also created a home directory and chown'd it to docker (Could have used the command adduser --disabled-password --gecos '' docker to do that in one go). This does a few things:
  • someone using r-base (or any image building on it) can run with --user docker, mounting volumes with -v localpath:/home/docker/somepath and not have the permissions of any file they edit/change end up owned by root instead.
  • If a user tries to starts rstudio (in rstudio and above) and tries to log in as docker, it will work now. Used to break because either /home/docker didn't exist or if it did it had the wrong permissions.
• in userconf.sh (rstudio Dockerfile), I keep the option to define a custom user, and keep the chown -R $USER:$USER /home/$USER, otherwise that user cannot log in or edit files. However, this option shouldn't be used when linking local volumes, and should perhaps be considered deprecated.
• in userconf.sh I also just set the password provided as the password for the docker user as well. This means that someone can now log into the RStudio instance as docker, with custom password, and linked volumes should probably work as expected.

[cboettig]

Okay, here's a slightly cleaner solution than requiring the user to be docker:

• We just delete the docker with userdel before creating the new user in userconf. This way, one can specify anything for the value of user (including docker, provided it's given as an environment -e USER=docker, otherwise rstudio is still the default) and still have that be the "first" users, (e.g. have UID 1000).
• Adjust userconf to take a custom UID (defaulting as before to 1000). In this way, any user can use the flag e UID=$UID to pass their UID to docker to make sure that permissions won't be impacted by linking volumes. (clearly this would mostly be of interest for any user who's id isn't 1000)

Not super-elegant, but until docker supports usernames instead of UIDs I think this will do. Some interesting feedback on both the SO and the google-discuss list above as well.

[eddelbuettel]

That looks good. One thing that is different between Debian and Ubuntu is the automagic sudo capability for uid 1000 in Ubuntu. So maybe we don't have to be that concerned with 1000 (unless something changed in Debian ...)

[cboettig]

Hmm! Just tried that on an ubuntu image and I still get:

docker is not in the sudoers file.  This incident will be reported.

[cboettig]

still good that we're on Debian. Kinda suprised Debian doesn't have more downloads: https://registry.hub.docker.com/search?q=library&f=official

[eddelbuettel]

I scratched my head about that one too -- might be another case of mind share / market share among Docker users.

[cboettig]

I think all the issues in this thread are resolved in the current build.

Anyone running our containers as a non-root user can link a directory, including ones at or above ~/, to the corresponding user's directory on the container. The container will only ever have one user.

If the host machine user has a UID other than 1000 (or 0, for root), the user should specify their UID when running docker, e.g.

docker run -d -P -v $(pwd)/home/$USER/foo \
-e USER=$USER  -e USERID = $UID rocker/rstudio

to avoid changing the permissions in the linked volume on the host. This is designed for rstudio mode and run only when the container is executed without a default command.

For interactive containers, it is sufficient to run with the flag --user docker, e.g. the interactive R container:

docker run --rm -it --user docker -v $(pwd):/home/docker/foo -w /home/docker/foo rocker/r-base R

note this command links the current working directory to /home/docker/foo on the container, where our user docker can write to, and also sets this as the working directory for the container. This is analogous to just running R in the working directory.

The only limitation is that the interactive method doesn't handle alternate UIDs. If the user running docker has a different UID, they have to do this more manually. Run a terminal in docker (using rstudio image or above):

docker run --rm -it -v $(pwd)/home/$USER/foo \
-e USER=$USER  -e USERID=$UID rocker/rstudio bash

and then in the bash shell run:

userconf.sh
su $USER
R

to setup the user with correct UID, switch to the non-root user, and then run R or whatever.

[dyndna]

@cboettig : Although a non-root user can share folders with running container, it is not a good idea to allow sharing of entire user home ~/, worse if shared folder is above ~/.

cd $HOME
docker run -d -P -v $(pwd)/home/$USER/foo \
-e USER=$USER  -e USERID = $UID rocker/rstudio

Running docker container as daemon with above command recursively and irreversibly sets owner permission to $UID and removes SUID, if set. At present with docker v1.7.0, docker stop <container_id> has no mechanism to reset original owner permission for all files and folders below working directory.

Although such behavior may be acceptable if one has specified $USER home ~/ or below as working directory, it is certainly a bad idea if user goes above ~/ which is either /home/ in linux or /Users/ in mac, or working directory is a block partition where other disks are mounted, e.g., /dev/sdb1 on /scratch. I had several system level configurations at /scratch/, where some folders were owned by root and SUID set. Following docker run command as above made my system unstable and had hard time to locate original owner level permission for several files and folders.

If a user run like following by switching to root block partition /, that is guaranteed system failure!

cd /
docker run -d -P -v $(pwd)/home/$USER/foo \
-e USER=$USER  -e USERID = $UID rocker/rstudio

I tried it on dummy system and only way to recover is to access hard drive from different system and go file by file to revert to root as owner and set SUID for at least following binaries source.

/sbin/unix_chkpwd
/usr/bin/chage
/usr/bin/passwd
/usr/bin/mail-touchlock
/usr/bin/mail-unlock
/usr/bin/gpasswd
/usr/bin/crontab
/usr/bin/chfn
/usr/bin/newgrp
/usr/bin/sudo
/usr/bin/wall
/usr/bin/mail-lock
/usr/bin/expiry
/usr/bin/dotlockfile
/usr/bin/chsh
/usr/lib/eject/dmcrypt-get-device
/usr/lib/pt_chown
/bin/ping6
/bin/su
/bin/ping
/bin/umount
/bin/mount

@cboettig: Forgot to pass big thanks for making rstudio docker. It works flawless once permissions are set correctly.

[cboettig]

@dyndna Yes, messing with the root file system can always cause trouble. If you are running docker with a non-root user, you should not have permissions to create, delete, or edit anything above $HOME (~/), just as in usual *NIX, so I'm not sure how you modified / files simply with the command shown. In any event, I wouldn't recommend ever linking host volumes above $HOME.

[dyndna]

Agree and non-root user usually has no read-write permission above $HOME. We have shared mount disk at /scratch which has sticky bits set for user-level read-write operations.

Although I have not tested, I believe your docker run command seems to instruct container root to set recursive owner permissions to host $UID at and below pwd. If that's true, then I doubt if host user permissions has any effect on read-write above user $HOME.

My 2 cents:

• I see -u flag in docker run manual to force docker container use non-root user, and that may be relevant here.

$foo:
cd /home/foo/dummy/
docker run -d -P -u $UID -v $(pwd)/home/$USER/foo \
-e USER=$USER  -e USERID = $UID rocker/rstudio

• docker run has a few options to play with SUID, SGID at https://docs.docker.com/reference/run/#runtime-privilege-linux-capabilities-and-lxc-configuration I have not tested those though.

Thanks

[cboettig]

@dyndna Yes, you're running the container as a root, so of course it has root permissions and can change the root directories. Note in the examples above we suggest running the container as a non-root user with the --user flag. I think you're confusing this with the suggestion to use things like e USER=$USER, which is for a different purpose entirely: setting the user name for the RStudio user (a non root-user, even though rstudio-server itself must be launched with root privileges on the container, hence we do not use --user for RStudio instances. Anyone logging into the RStudio console won't have root privileges and cannot generally write above $HOME.

In your example, since you run the container as root and mount a root directory on the host to the home directory of the container. If we did not change permissions on that home directory (recursively), the rstudio user would not be able to access it or the files contained in it. Like you observe, docker volume linking prevents you from modifying things above the linked volume.

Thanks for your comments, we should probably add some clarifications to the Wiki on not linking volumes above $HOME.

Maybe there's a way we can avoid any call to change permissions if volumes are linked; e.g. https://github.com/rocker-org/rocker/blob/master/rstudio/userconf.sh#L55;

[Robinlovelace]

Hi @cboettig I'm just hitting into issues with user permissions. I've read this and advice on StackOverflow but I find that setting --user makes the docker fail. The 'nuclear option' chmod a+rwx folder-to-edit-using-a-docker-container works fine but wondered if there's any better advice for people (I'm thinking of updating the advice here in case it fails: https://github.com/Robinlovelace/geocompr#running-geocompr-code-in-docker )

[cboettig]

Hi @Robinlovelace , nice to hear from you. Just taking a quick look at https://github.com/Robinlovelace/geocompr#running-geocompr-code-in-docker, seems like you're binding to /data on the container:

docker run -d -p 8787:8787 -v $(pwd):/data robinlovelace/geocompr

If I'm going to be accessing the container with RStudio, I almost always just bind to /home/rstudio/data instead:

docker run -d -p 8787:8787 -v $(pwd):/home/rstudio/data robinlovelace/geocompr

Can you consider switching to that? Otherwise you will want to change permissions of /data on the container, which is owned by root by default.

Sidenote: binding to /data is fine when running in command-line mode as root, e.g.: docker run -ti -v $(pwd):/data robinlovelace/geocompr R since you are then a root user inside the container and can happily write to /data. The downside of this is that back on the host the files you create or modify in that working directory will also now be owned by root.

[Robinlovelace]

Many thanks for this lucid explanation. Entendido!

I will update the README (and my own usage of docker) accordingly.

[Robinlovelace]

++ PR v welcome (I see you've forked the book ; )

[Robinlovelace]

Update @cboettig I've updated the readme following your advice - see here: geocompx/geocompr@a02d3f9

I'm still getting messages saying RStudio doesn't have the right permissions to open the project. This is separate from the previous issue but my solution is still to do, for example

 chmod a+rwx geocompr

It works out of the box on my windows machine so wonder if its a question of docker set-up. Thanks loads in any case!

[alexrblohm]

So I was having the same issues and it was fixed by changing the listening port (sorry, I'm new to this and may be describing it wrong). I used a docker-compose and did
ports:
- '50:8787'

But in the docker run -p 50:8787 should do it if it's the same.

That was the big fix for me! First post here, please be nice :)

My Dockerfile (note I was messing with the ENV (-e) stuff
`FROM rocker/verse
COPY . /home/rstudio/project

MAINTAINER 'Alex Blohm' alexrblohm@gmail.com

ADD install_packages.R /

RUN Rscript /install_packages.R

EXPOSE 8787

#WORKDIR /home/rstudio/project

ENV DISABLE_AUTH=true
#ENV USER=guest
#ENV USERID=
#ENV GROUPID=
#ENV UMASK=022
#ENV PASSWORD=guest`

docker-compose
version: '3' services: r_docker: build: . container_name: r_docker ports: - '50:8787' volumes: - '.:/home/rstudio/project'

Sorry formatting got all crazy!