Populating a volume using a container does not work in Docker on Windows

[YossyTaka]

Description
I'm following the instruction on this Docker official page under "Populate a volume using a container" in an attempt to create a new volume populated with existing files in a newly launched container. However, file copy to the volume is not taking place at all and the volume remains completely empty.

Steps to reproduce the issue:

1. Create a Windows container image with a folder named C:\Data and place some files and folders under it.
2. Launch a new container using this image, with a new volume created at the same location by running docker run -it --name=test -v C:\Data dataimage/test1:version1
3. Now observe that the C:\Data inside the container is empty.

Describe the results you received:
The volume gets created but the original content of the folder is completely hidden and empty. If I run the above command without the -v option instead, then I can see the original files at the same location.

Describe the results you expected:
The command should create a new volume under C:Data inside the container and copy all existing files at that location into the volume.

Additional information you deem important (e.g. issue happens only occasionally):
The issue happens 100% of the time.

Output of docker version:

Client: Docker Engine - Community
 Version:           19.03.8
 API version:       1.40
 Go version:        go1.12.17
 Git commit:        afacb8b
 Built:             Wed Mar 11 01:23:10 2020
 OS/Arch:           windows/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.8
  API version:      1.40 (minimum version 1.24)
  Go version:       go1.12.17
  Git commit:       afacb8b
  Built:            Wed Mar 11 01:37:20 2020
  OS/Arch:          windows/amd64
  Experimental:     false

Output of docker info:

Client:
 Debug Mode: false

Server:
 Containers: 2
  Running: 1
  Paused: 0
  Stopped: 1
 Images: 3
 Server Version: 19.03.8
 Storage Driver: windowsfilter
  Windows:
 Logging Driver: json-file
 Plugins:
  Volume: local
  Network: ics internal l2bridge l2tunnel nat null overlay private transparent
  Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog
 Swarm: inactive
 Default Isolation: process
 Kernel Version: 10.0 17763 (17763.1.amd64fre.rs5_release.180914-1434)
 Operating System: Windows Server 2019 Datacenter Version 1809 (OS Build 17763.1098)
 OSType: windows
 Architecture: x86_64
 CPUs: 8
 Total Memory: 31.54GiB
 Name: EC2AMAZ-7SMIA34
 ID: RTKF:5UGA:G7JI:REYB:LC46:IIUE:GWAK:6S5Z:PW2T:5JNL:XWFJ:RBYD
 Docker Root Dir: C:\ProgramData\Docker
 Debug Mode: true
  File Descriptors: -1
  Goroutines: 47
  System Time: 2020-04-03T20:52:18.9110014Z
  EventsListeners: 2
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

Additional environment details (AWS, VirtualBox, physical, etc.):
Running on AWS G4 EC2 instance.

[guillaume86]

I was just going to post this and then I see this is the most recent issue, maybe it's a recent regression?

Here's the docker file I use to test this:

# escape=`
FROM mcr.microsoft.com/windows/nanoserver:1909
RUN mkdir C:\myvol
RUN echo "hello world" > C:\myvol\greeting
VOLUME C:\myvol
CMD type C:\myvol\greeting

I tried it in both process and hyperv isolation and none works. If you comment out the VOLUME instruction is does work of course. I also tried various path variants (using slashes, add ending backslash, etc) but the issue does not seem to be there.

[thaJeztah]

You seem to have the "host" and "container" paths reversed, and as a result are bind-mounting a host (C:\Data) directory into the container at /version1.

The syntax for the short-hand (-v) flag to mount a named volume is;

-v <vol-name>:<path in container>

For example;

-v myvolume:C:\Data

After that (assuming myvolume was empty), the named volume should now be propagated with the contents of the C:\Data directory from the container/image

[YossyTaka]

dataimage/test1:version1 is actually the name of the image I am using, it is not a file system path. There is no "host" path in my command line intentionally because I am not using bind mount. I am creating a new volume and having it attached at C:\Data inside of the container.

[guillaume86]

@thaJeztah the minimal repro is at #40771 (comment). Run it without volume mappings (let docker create the volume), the file will not be available.

[guillaume86]

Think I found the offending code:
https://github.com/moby/moby/blob/master/volume/mounts/linux_parser.go#L407-L409
vs
https://github.com/moby/moby/blob/master/volume/mounts/windows_parser.go#L447-L449

Edit: This might not be the issue, I tried it and it didn't fix the issue for me, but there's also a 50/50 chance I made a mistake when building/testing so take it with a grain of salt.

[jbaig]

I think this is actually related to the nanoserver image. It does not happen when using servercore images because of permissions.

Nanoserver uses the user ContainerUser which does not have permissions to named volumes stored in the default docker volume folder.

One option, although I don't know the security risks, is to add "Authenticated User" to the volume folder in the docker folder (usually C:\ProgramData\docker\volumes unless you set it up to be a different directory). Then when any named volume is created it will have that permission and allow access to ContainerUser.

As for the security concern, "Authenticated User" is a pretty minimal security level, just above Everyone except with authentication, especially considering docker container users are considered Authenticated Users; hence why this works.

ref: Persistent Storage in Containers

[guillaume86]

@jbaig I just tried with the servercore:1909 image and it doesn't work for me.
Do you have a example dockerfile?
Here's the one I tried:

# escape=`
FROM mcr.microsoft.com/windows/servercore:1909
RUN mkdir C:\myvol
RUN echo "hello world" > C:\myvol\greeting
VOLUME C:\myvol
CMD type C:\myvol\greeting

result:

PS C:\Users\guill\Downloads\dockerbug> docker build . -t bug_repro
Sending build context to Docker daemon  2.048kB
Step 1/5 : FROM mcr.microsoft.com/windows/servercore:1909
 ---> ae65f6800c31
Step 2/5 : RUN mkdir C:\myvol
 ---> Running in bc6b0782d77e
Removing intermediate container bc6b0782d77e
 ---> fc979fb76635
Step 3/5 : RUN echo "hello world" > C:\myvol\greeting
 ---> Running in 0ce9035240e2
Removing intermediate container 0ce9035240e2
 ---> 24ae171afb0b
Step 4/5 : VOLUME C:\myvol
 ---> Running in 50e9555fa90f
Removing intermediate container 50e9555fa90f
 ---> ca79418b9190
Step 5/5 : CMD type C:\myvol\greeting
 ---> Running in 49b3b235b5ad
Removing intermediate container 49b3b235b5ad
 ---> 3e232c49dbc9
Successfully built 3e232c49dbc9
Successfully tagged bug_repro:latest
PS C:\Users\guill\Downloads\dockerbug> docker run --rm bug_repro
The system cannot find the file specified.
PS C:\Users\guill\Downloads\dockerbug>

[jbaig]

@guillaume86 Sorry I misunderstood. In my case using nanoserver with shared volumes gave an access is denied (unless using Authenticated Users) vs servercore which was fine.

As for what you are saying, that relates to windows requiring the volume to be empty at the start: https://docs.docker.com/engine/reference/builder/#notes-about-specifying-volumes

The work around for that is to add the file(s) at startup and then it will live in the shared volume going forward.
For example with this docker file:

# escape=`
FROM mcr.microsoft.com/windows/servercore:1809
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'Continue'; $verbosePreference='Continue';"]
RUN mkdir C:\myvol
VOLUME C:\myvol
CMD echo 'hello world' | Out-File -FilePath C:\myvol\greeting; type C:\myvol\greeting

The end result will give the hello world and store the document in a shared volume.
One way I've found to handle this is to have the entrypoint command copy files if they didn't already exist, so that it would populate files the first time but then not do it again going forward.

[guillaume86]

@jbaig ha good find in the documentation, a warning in the section about populating new volumes would be nice to avoid future confusion.