--tmpfs with --user works for the first time. Fails in subsequent container starts.

[squeaky-pl]

I'm trying to run busybox container with --tmpfs and -u set. After starting the container the first time I can write to tmpfs volume, after stopping it and starting the second time it fails with permission denied.

I'm running Fedora 23

Linux localhost.localdomain 4.3.3-303.fc23.x86_64 #1 SMP Tue Jan 19 18:31:55 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

docker version
Client:
 Version:      1.10.1
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   9e83765
 Built:        Thu Feb 11 19:33:13 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.10.1
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   9e83765
 Built:        Thu Feb 11 19:33:13 2016
 OS/Arch:      linux/amd64

docker info
Containers: 43
 Running: 1
 Paused: 0
 Stopped: 42
Images: 194
Server Version: 1.10.1
Storage Driver: devicemapper
 Pool Name: docker-253:0-1316958-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 107.4 GB
 Backing Filesystem: ext4
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 36.48 GB
 Data Space Total: 107.4 GB
 Data Space Available: 5.229 GB
 Metadata Space Used: 29.13 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.118 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 WARNING: Usage of loopback devices is strongly discouraged for production use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning.
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.109 (2015-09-22)
Execution Driver: native-0.2
Logging Driver: json-file
Plugins: 
 Volume: local
 Network: bridge null host
Kernel Version: 4.3.3-303.fc23.x86_64
Operating System: Fedora 23 (Workstation Edition)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 3.726 GiB
Name: localhost.localdomain
ID: QOYG:UF6S:FW7E:HDNU:CTP3:M4M5:6LYZ:FCEX:Q2TH:EKDR:PXYR:FU6H

To reproduce

●  docker  create -u 1000:1000 --tmpfs /tmpfs --name testcase busybox sleep 1000
8fec7d4d3103566ac4691b1765df540e6c3a79dab6bbc6293176a79cdebe1440
●  docker start testcase
testcase
●  docker exec -ti testcase touch /tmpfs/test
●  docker stop testcase                     
testcase
●  docker start testcase                     
testcase
●  docker exec -ti testcase touch /tmpfs/test
touch: /tmpfs/test: Permission denied

Expected result: I can consistently write to tmpfs with several container restarts
Actual results: I can only write to tempfs on the first container start, I need to remove and recreate container

[thaJeztah]

/cc @rhatdan

[rhatdan]

What are the permissions on the tmpfs on the second run?

[rhatdan]

First run

/ $ ls -ld /tmpfs
drwxrwxrwt    2 root     root            40 Feb 18 14:00 /tmpfs

Second

/ $ ls -ld /tmpfs
drwxr-xr-x    2 root     root            40 Feb 18 14:01 /tmpfs

[rhatdan]

I would guess that since the directory does not exists on the first run docker creates it and mounts the loose permissions on top. But the underlying directory gets created as 755. Second run the directory exists and --tmpfs sees this and gets the permissions of the underlying directory.

I guess the question is what is the correct behaviour.

[rhatdan]

cd /var/lib/docker/devicemapper/mnt/d38b20c7143d58cf10a7d8c673ae9daf8689394d91bb8d4caa24425ae50d9c7f/rootfs/
# ls -ld tmpfs
drwxr-xr-x. 2 root root 6 Feb 18 09:00 tmpfs

[rhatdan]

If you run this same test with /tmp or /var/tmp, I think it will work correctly.

[squeaky-pl]

Yes, I run the same testcase with /tmp instead of /tmpfs and it worked correctly.

According to the feature highlight this option is supposed to help applications that need to write some temporary data. I would understand it should work with any location and together with --user. If not then the documentation needs to be made explicit about it.

[rhatdan]

Well it will work with any directory, the problem here is the directory did not exist in the image before running your command. Since the directory does not exist, there is no way to know which permissions should be applied to the tmpfs, I guess we could allow the specification of the perms on the volume

docker run --tmpfs /tmpfs:777  

Or something like that.

Or do we fail the --tmpfs command and say the underlying directory has to exist for tmpfs to be mounted.

That would force you to build an image with /tmpfs in the image. I guess option two might be the better solution.

[squeaky-pl]

I also liked the second option more given it's explicitly noted in the docs about the existence of location in the image and that is has the same permissions. I think it should explode as well -- otherwise somebody might be tempted to use it like I did and it seemed to work.

[rhatdan]

Actually option #1 is available now.

docker run --tmpfs /tmpfs:mode=1777 -ti fedora sh

Except it requires more thinking on the users part.

[squeaky-pl]

Thank you. That's great to know, did you find a documentation on it or just found that out by reading the code?

I still think the first case needs to be made consistent.

[rhatdan]

I wrote the code, but forgot that we pass the mount options after the ":"

Part of my patch never got in though, so I might be opening a pull request for it later. I have to investigate the best way to handle the missing directory problem. In docker any missing directory that gets volume mounted on, gets 755 as its permissions.

[mauricios]

Is docker using the permissions mount options of tmpfs after start a container?
In my tests the permissions only works when creating the container and the underlying directory does not exist.

I think the expected and correct behavior should be that tmpfs should always honor the mount options permissions.

[ben-ng]

I'm running into the same issue right now.

[kolorafa]

same issue here

Trying to run a plexinc/pms-docker container but use tmpfs for /transcode directory to speedup transcoding process on slow HD, and don't wear out ssd, but non of the method above did mount the directory with permissions that allow plex to write after a restart.

To mitigate problem I need to:

• remake the container - as it does change uid to "plex" right after first "docker run ..." that with 755 permissions allow plex to write
• do a docker exec -it plex chmod 777 /transcode OR docker exec -it plex chown plex /transcode after every docker/server restart

What doesn't work:

--tmpfs /transcode:mode=1777
--tmpfs /transcode:mode=0777
--tmpfs /transcode:mode=777
--mount type=tmpfs,destination=/transcode,tmpfs-mode=1777
--mount type=tmpfs,destination=/transcode,tmpfs-mode=0777
--mount type=tmpfs,destination=/transcode,tmpfs-mode=777

always results right after run:
drwxr-xr-x 2 plex plex 40 Nov 6 21:49 .
after docker restart:
drwxr-xr-x 2 root root 40 Nov 6 21:50 .

Docker version 18.06.1-ce, build e68fc7a215

What does work:

--tmpfs /transcode:uid=1000,gid=1000
(docker/compose#3425)

uid/gid only affect directory after container restart

$ docker exec -it plex ls -lah /transcode; docker restart plex; docker exec -it plex ls -lah /transcode;
total 4.0K
drwxr-xr-x 2 plex plex 40 Nov 6 22:03 .
drwxr-xr-x 25 root root 4.0K Nov 6 22:03 ..
plex
total 4.0K
drwxr-xr-x 2 1001 1001 40 Nov 6 22:03 .
drwxr-xr-x 25 root root 4.0K Nov 6 22:03 ..

[Rubiobernal]

Ola