path/filepath: EvalSymlinks of container mapped directory returns an error

[tescherm]

What version of Go are you using (go version)?

go version go1.9.2 windows/amd64 and go version go1.10beta2 windows/amd64.

Does this issue reproduce with the latest release?

Yes, this is reproducable with go 1.10beta2 and 1.9.2

What operating system and processor architecture are you using (go env)?

set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\ContainerAdministrator\AppData\Local\go-build
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\gopath
set GORACE=
set GOROOT=C:\go
set GOTMPDIR=
set GOTOOLDIR=C:\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\ContainerAdministrator\AppData\Local\Temp\go-build022882258
=/tmp/go-build -gno-record-gcc-switches

What did you do?

Running filepath.EvalSymlinks against a bind mounted directory within a windows container (for example, \\?\ContainerMappedDirectories\D48AFB19-F38F-4DAE-A698-4D718D506B15) returns the following error:

PS C:\Users\Administrator> docker run --rm -v c:\:c:\host golang:1.10-rc-nanoserver-sac2016 go run c:\host\eval_symlinks.go
panic: GetFileAttributesEx \ContainerMappedDirectories: The system cannot find the file specified.

goroutine 1 [running]:
main.main()
        c:/host/eval_symlinks.go:11 +0xe6
exit status 2

eval_symlinks.go above is the following program:

package main

import (
	"fmt"
	"path/filepath"
)

func main() {
	path, err := filepath.EvalSymlinks("c:\\host")
	if err != nil {
		panic(err)
	}

	fmt.Printf("eval path: %s\n", path)
}

Note that the symlink is valid. For example, I can cd to it:

PS C:\gopath> cmd /c dir c:\
 Volume in drive C has no label.
 Volume Serial Number is 8E1D-692C

 Directory of c:\

01/22/2018  01:34 PM    <DIR>          go
01/08/2018  10:34 AM    <DIR>          gopath
01/22/2018  01:34 PM    <SYMLINKD>     host [\\?\ContainerMappedDirectories\007C5A5F-50A3-4019-BCF1-1BA20F4745
11/20/2016  03:32 AM             1,894 License.txt
01/22/2018  01:34 PM    <DIR>          Program Files
07/16/2016  04:09 AM    <DIR>          Program Files (x86)
01/22/2018  01:34 PM    <DIR>          Users
01/22/2018  01:34 PM    <DIR>          Windows
               1 File(s)          1,894 bytes
               7 Dir(s)  21,256,376,320 bytes free
PS C:\gopath> cd \\?\ContainerMappedDirectories\007C5A5F-50A3-4019-BCF1-1BA20F4745F4
PS Microsoft.PowerShell.Core\FileSystem::\\?\ContainerMappedDirectories\007C5A5F-50A3-4019-BCF1-1BA20F4745F4>

golang 1.9.2 returns the same error:

PS C:\Users\Administrator> docker run --rm -v c:\:c:\host golang:1.9.2-nanoserver-sac2016 go run c:\host\eval_symlinks.go
panic: GetFileAttributesEx \ContainerMappedDirectories: The system cannot find the file specified.

goroutine 1 [running]:
main.main()
        c:/host/eval_symlinks.go:11 +0xfe
exit status 2

I see that EvalSymlinks on Windows was reimplemented recently (66c03d3), maybe there was a nuance wrt the \\?\ path style that was missed?

What did you expect to see?

I expected filepath.EvalSymlinks to return the original directory (C:\host) or the directory that was linked to (\\?\ContainerMappedDirectories\007C5A5F-50A3-4019-BCF1-1BA20F4745).

What did you see instead?

The error GetFileAttributesEx \ContainerMappedDirectories: The system cannot find the file specified.

[tklauser]

/cc @alexbrainman

[alexbrainman]

Thank you for the report. I will investigate this when I have time.

Alex

[tescherm]

Thanks @alexbrainman let me know if I can provide any additional info.

[tescherm]

I did a little more investigating. This following program also fails:

package main

import "os"
import "fmt"

func main() {
    path, err := os.Readlink("C:\\host")
    if err != nil {
        panic(fmt.Errorf("os.Readlink failed: %v", err))
    }

    println("readlink path", path)

    fi, err := os.Open(path)
    if err != nil {
        panic(fmt.Errorf("os.Open failed: %v", err))
    }

    defer fi.Close()

    println("file info", fi.Name())
}

readlink path \ContainerMappedDirectories\FF9714DB-37DA-4741-AC60-876DD8CFDFF2
panic: os.Open failed: open \ContainerMappedDirectories\FF9714DB-37DA-4741-AC60-876DD8CFDFF2: The system cannot find the path specified.

However if I prepend \\? to the path returned by os.Readlink, os.Open succeeds:

package main

import "os"
import "fmt"

func main() {
    path, err := os.Readlink("C:\\host")
    if err != nil {
        panic(fmt.Errorf("os.Readlink failed: %v", err))
    }

    println("readlink path", path)

    fi, err := os.Open("\\\\?" + path)
    if err != nil {
        panic(fmt.Errorf("os.Open failed: %v", err))
    }

    defer fi.Close()

    println("file info", fi.Name())
}

readlink path \ContainerMappedDirectories\846DD71E-ABA5-4F3B-93EB-0276148F0F98
file info \\?\ContainerMappedDirectories\846DD71E-ABA5-4F3B-93EB-0276148F0F98

Is is possible that os.Readlink on Windows doesn't handle the \\?\ path style?

[alexbrainman]

let me know if I can provide any additional info.

I just need to be able to reproduce your problem, and I will need Docker installed for that. What is the easiest way to install Docker if I have Windows 10?

You can also try and fix this yourself. If you try, you need to be aware about forthcoming change https://go-review.googlesource.com/#/c/go/+/86556/ that might affect your problem. Also Windows paths starting with \?\ are not handled very well in many Go standard library parts. For example path/filepath does not handle these paths at all. Also see issue #22230.

Is is possible that os.Readlink on Windows doesn't handle the \?\ path style?

os.Readlink is broken on Windows. os.Readlink current implementation does not work in many situations, and I do not know of a way to implement it properly. But lets not discuss os.Readlink here. Create new issue for os.Readlink, if you think it is important.

Alex

[tescherm]

@alexbrainman you can install Docker for Windows 10 by following the instructions here. You will want to make sure that you switch to Windows Containers (the second step).

I'll see if I can spend some time looking at this as well.

[alexbrainman]

@alexbrainman you can install Docker for Windows 10 by following the instructions here. You will want to make sure that you switch to Windows Containers (the second step).

When I run "Docker for Windows" app, I get this message

but I do not want to proceed, because I do use VirtualBox. I do not want to loose ability to run VirtualBox. What should I do?

Thank you.

Alex

[as]

It is specifically the hardware virtualization extensions that will not work; Hyper-V acquires them before Windows boots and Virtualbox can't use them anymore. This can be restored by disabling the hyper-v service in services.msc.

I happen to have Docker Enterprise on a W10 system and can attempt to reproduce it if that workaround isn't possible

[alexbrainman]

It is specifically the hardware virtualization extensions that will not work; Hyper-V acquires them before Windows boots and Virtualbox can't use them anymore.

I discovered that much myself.

I happen to have Docker Enterprise on a W10 system and can attempt to reproduce it if that workaround isn't possible

I will try https://cloud.google.com/compute/docs/containers/#docker_on_windows when I have time.

Alex

[tescherm]

@alexbrainman the link above should work on Windows Server 2016, without the need to disable Virtualbox.

[alexbrainman]

@alexbrainman the link above should work on Windows Server 2016, without the need to disable Virtualbox.

If you are referring to https://docs.microsoft.com/en-us/virtualization/windowscontainers/quick-start/quick-start-windows-10 then it does not help me, because I only have Windows 10 computer, not Windows Server 2016,

If you are referring to https://cloud.google.com/compute/docs/containers/#docker_on_windows then yes, it should work. But I would need to create computer from scratch and install all the tools I need. So I will need some free time.

Alex

[kolyshkin]

@tescherm can you please re-check that the issue was fixed in Go 1.11 (e83601b)?

[tescherm]

@kolyshkin I can confirm that this is still an issue. Running docker run --rm -v c:\:c:\host golang:1.11-nanoserver-sac2016 go run c:\host\eval_symlinks.go with the program above yields:

[alexbrainman]

@tescherm I am not surprised that os.Readlink is broken on windows. But maybe path/filepath.EvalSymlinks is OK. Please, try this program instead

package main

import (
	"fmt"
	"path/filepath"
)

func main() {
	path, err := filepath.EvalSymlinks("c:\\host")
	if err != nil {
		panic(err)
	}

	fmt.Printf("eval path: %s\n", path)
}

If it still fails, someone would have to investigate where it fails. I still do not have Windows Docker installed on my computer - I use VirtualBox, and, apparently, you cannot have both. So I would need to configure new PC somewhere (probably in the cloud), and have Go development environment installed on it before I can even start investigating this. I have very little free time nowadays, but I will do that, if nothing else.

Thank you.

Alex

[tescherm]

@alexbrainman I tried it out, unfortunately the program also fails on 1.11:

PS C:\> docker run --rm -v c:\:c:\host golang:1.10-nanoserver-sac2016 go run c:\host\eval_symlinks.go
panic: GetFileAttributesEx \ContainerMappedDirectories: The system cannot find the file specified.

goroutine 1 [running]:
main.main()
        c:/host/eval_symlinks.go:11 +0xf0
exit status 2

PS C:\> docker run --rm -v c:\:c:\host golang:1.11-nanoserver-sac2016 go run c:\host\eval_symlinks.go
panic: FindFirstFile \ContainerMappedDirectories: The system cannot find the file specified.

goroutine 1 [running]:
main.main()
        c:/host/eval_symlinks.go:11 +0xf0
exit status 2

[alexbrainman]

@tescherm thanks for checking. I would have to debug this myself when I have time.

Alex

[alexbrainman]

@tescherm

I managed to setup appropriate environment for your test. I build this program

package main

import (
	"fmt"
	"path/filepath"
)

func main() {
	path, err := filepath.EvalSymlinks("c:\\host")
	if err != nil {
		panic(err)
	}

	fmt.Printf("eval path: %s\n", path)
}

and copied resulting exe into c:\test.exe, and then run this command

docker run --rm -v c:\:c:\host microsoft/nanoserver c:\host\test.exe

I can reproduce your error now.

Looking at the code, we call Windows API GetFinalPathNameByHandle with c:\host and VOLUME_NAME_DOS inside the container, and that call fails with ERROR_FILE_NOT_FOUND. I tried all VOLUME_NAME_DOS alternatives, and only VOLUME_NAME_NT works - it converts c:\host into \\Device\\HarddiskVolume1\\.

If we adjust this code to return \\Device\\HarddiskVolume1\\ in your situation, would that help you? How? Please explain in details, what it would give you.

path/filepath.EvalSymlinks is started as code to follow symlinks on Unix. I am not sure what we have here is covered by that description.

Thank you.

Alex

[tescherm]

@alexbrainman great, yes, I believe that returning \\Device\\HarddiskVolume1\\ in the example above would help. The goal would be that this path resolves with other fs system calls. For example:

package main

import "os"
import "fmt"

func main() {
    path, err := filepath.EvalSymlinks("c:\\host")
    if err != nil {
        panic(err)
    }

    fmt.Printf("eval path: %s\n", path)

    // open resolves correctly
    fi, err := os.Open(path)
    if err != nil {
        panic(fmt.Errorf("os.Open failed: %v", err))
    }

    defer fi.Close()

    println("file info", fi.Name())
}

Thanks again for drilling in.

[alexbrainman]

I believe that returning \\Device\\HarddiskVolume1\\ in the example above would help. The goal would be that this path resolves with other fs system calls.

If we return \Device\HarddiskVolume1\, that name would refer to
directory inside you container. For example, if your current drive is
C:, then \Device\HarddiskVolume1\ would mean
C:\Device\HarddiskVolume1\. I don't think that is what you want. You
want C:\ outside of container.

The \Device\HarddiskVolume1\ returned by calling
GetFinalPathNameByHandle with VOLUME_NAME_NT, is not proper Windows
path. Search for "NT Namespaces" in
https://docs.microsoft.com/en-us/windows/desktop/fileio/naming-a-file
to understand what it is. I doubt that Windows CreateFile API (and other APIs that Go uses) can access "NT Namespaces" paths.

And, in #23512 (comment) , you did not explain why you need to to call filepath.EvalSymlinks? Why cannot you call os.Open("c:\host") instead?

Alex

[ianlancetaylor]

Note that filepath.EvalSymlinks has been rewritten completely for 1.12 in https://golang.org/cl/121676 for #23444. It may not affect this case, though.

[alexbrainman]

It may not affect this case, though.

I agree. https://golang.org/cl/121676 is unrelated to this issue. It is still not clear to me what filepath.EvalSymlink should do to path inside container that points to volume outside container. Maybe it should do nothing.

Alex