path/filepath: EvalSymlinks of container mapped directory returns an error

[tescherm]

What version of Go are you using (go version)?

go version go1.9.2 windows/amd64 and go version go1.10beta2 windows/amd64.

Does this issue reproduce with the latest release?

Yes, this is reproducable with go 1.10beta2 and 1.9.2

What operating system and processor architecture are you using (go env)?

set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\ContainerAdministrator\AppData\Local\go-build
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\gopath
set GORACE=
set GOROOT=C:\go
set GOTMPDIR=
set GOTOOLDIR=C:\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\ContainerAdministrator\AppData\Local\Temp\go-build022882258
=/tmp/go-build -gno-record-gcc-switches

What did you do?

Running filepath.EvalSymlinks against a bind mounted directory within a windows container (for example, \\?\ContainerMappedDirectories\D48AFB19-F38F-4DAE-A698-4D718D506B15) returns the following error:

PS C:\Users\Administrator> docker run --rm -v c:\:c:\host golang:1.10-rc-nanoserver-sac2016 go run c:\host\eval_symlinks.go
panic: GetFileAttributesEx \ContainerMappedDirectories: The system cannot find the file specified.

goroutine 1 [running]:
main.main()
        c:/host/eval_symlinks.go:11 +0xe6
exit status 2

eval_symlinks.go above is the following program:

package main

import (
	"fmt"
	"path/filepath"
)

func main() {
	path, err := filepath.EvalSymlinks("c:\\host")
	if err != nil {
		panic(err)
	}

	fmt.Printf("eval path: %s\n", path)
}

Note that the symlink is valid. For example, I can cd to it:

PS C:\gopath> cmd /c dir c:\
 Volume in drive C has no label.
 Volume Serial Number is 8E1D-692C

 Directory of c:\

01/22/2018  01:34 PM    <DIR>          go
01/08/2018  10:34 AM    <DIR>          gopath
01/22/2018  01:34 PM    <SYMLINKD>     host [\\?\ContainerMappedDirectories\007C5A5F-50A3-4019-BCF1-1BA20F4745
11/20/2016  03:32 AM             1,894 License.txt
01/22/2018  01:34 PM    <DIR>          Program Files
07/16/2016  04:09 AM    <DIR>          Program Files (x86)
01/22/2018  01:34 PM    <DIR>          Users
01/22/2018  01:34 PM    <DIR>          Windows
               1 File(s)          1,894 bytes
               7 Dir(s)  21,256,376,320 bytes free
PS C:\gopath> cd \\?\ContainerMappedDirectories\007C5A5F-50A3-4019-BCF1-1BA20F4745F4
PS Microsoft.PowerShell.Core\FileSystem::\\?\ContainerMappedDirectories\007C5A5F-50A3-4019-BCF1-1BA20F4745F4>

golang 1.9.2 returns the same error:

PS C:\Users\Administrator> docker run --rm -v c:\:c:\host golang:1.9.2-nanoserver-sac2016 go run c:\host\eval_symlinks.go
panic: GetFileAttributesEx \ContainerMappedDirectories: The system cannot find the file specified.

goroutine 1 [running]:
main.main()
        c:/host/eval_symlinks.go:11 +0xfe
exit status 2

I see that EvalSymlinks on Windows was reimplemented recently (66c03d3), maybe there was a nuance wrt the \\?\ path style that was missed?

What did you expect to see?

I expected filepath.EvalSymlinks to return the original directory (C:\host) or the directory that was linked to (\\?\ContainerMappedDirectories\007C5A5F-50A3-4019-BCF1-1BA20F4745).

What did you see instead?

The error GetFileAttributesEx \ContainerMappedDirectories: The system cannot find the file specified.

[tklauser]

/cc @alexbrainman

[alexbrainman]

Thank you for the report. I will investigate this when I have time.

Alex

[tescherm]

Thanks @alexbrainman let me know if I can provide any additional info.

[tescherm]

I did a little more investigating. This following program also fails:

package main

import "os"
import "fmt"

func main() {
    path, err := os.Readlink("C:\\host")
    if err != nil {
        panic(fmt.Errorf("os.Readlink failed: %v", err))
    }

    println("readlink path", path)

    fi, err := os.Open(path)
    if err != nil {
        panic(fmt.Errorf("os.Open failed: %v", err))
    }

    defer fi.Close()

    println("file info", fi.Name())
}

readlink path \ContainerMappedDirectories\FF9714DB-37DA-4741-AC60-876DD8CFDFF2
panic: os.Open failed: open \ContainerMappedDirectories\FF9714DB-37DA-4741-AC60-876DD8CFDFF2: The system cannot find the path specified.

However if I prepend \\? to the path returned by os.Readlink, os.Open succeeds:

package main

import "os"
import "fmt"

func main() {
    path, err := os.Readlink("C:\\host")
    if err != nil {
        panic(fmt.Errorf("os.Readlink failed: %v", err))
    }

    println("readlink path", path)

    fi, err := os.Open("\\\\?" + path)
    if err != nil {
        panic(fmt.Errorf("os.Open failed: %v", err))
    }

    defer fi.Close()

    println("file info", fi.Name())
}

readlink path \ContainerMappedDirectories\846DD71E-ABA5-4F3B-93EB-0276148F0F98
file info \\?\ContainerMappedDirectories\846DD71E-ABA5-4F3B-93EB-0276148F0F98

Is is possible that os.Readlink on Windows doesn't handle the \\?\ path style?

[alexbrainman]

let me know if I can provide any additional info.

I just need to be able to reproduce your problem, and I will need Docker installed for that. What is the easiest way to install Docker if I have Windows 10?

You can also try and fix this yourself. If you try, you need to be aware about forthcoming change https://go-review.googlesource.com/#/c/go/+/86556/ that might affect your problem. Also Windows paths starting with \?\ are not handled very well in many Go standard library parts. For example path/filepath does not handle these paths at all. Also see issue #22230.

Is is possible that os.Readlink on Windows doesn't handle the \?\ path style?

os.Readlink is broken on Windows. os.Readlink current implementation does not work in many situations, and I do not know of a way to implement it properly. But lets not discuss os.Readlink here. Create new issue for os.Readlink, if you think it is important.

Alex

[tescherm]

@alexbrainman you can install Docker for Windows 10 by following the instructions here. You will want to make sure that you switch to Windows Containers (the second step).

I'll see if I can spend some time looking at this as well.

[alexbrainman]

@alexbrainman you can install Docker for Windows 10 by following the instructions here. You will want to make sure that you switch to Windows Containers (the second step).

When I run "Docker for Windows" app, I get this message

but I do not want to proceed, because I do use VirtualBox. I do not want to loose ability to run VirtualBox. What should I do?

Thank you.

Alex

[as]

It is specifically the hardware virtualization extensions that will not work; Hyper-V acquires them before Windows boots and Virtualbox can't use them anymore. This can be restored by disabling the hyper-v service in services.msc.

I happen to have Docker Enterprise on a W10 system and can attempt to reproduce it if that workaround isn't possible

[alexbrainman]

It is specifically the hardware virtualization extensions that will not work; Hyper-V acquires them before Windows boots and Virtualbox can't use them anymore.

I discovered that much myself.

I happen to have Docker Enterprise on a W10 system and can attempt to reproduce it if that workaround isn't possible

I will try https://cloud.google.com/compute/docs/containers/#docker_on_windows when I have time.

Alex

[tescherm]

@alexbrainman the link above should work on Windows Server 2016, without the need to disable Virtualbox.

[alexbrainman]

@alexbrainman the link above should work on Windows Server 2016, without the need to disable Virtualbox.

If you are referring to https://docs.microsoft.com/en-us/virtualization/windowscontainers/quick-start/quick-start-windows-10 then it does not help me, because I only have Windows 10 computer, not Windows Server 2016,

If you are referring to https://cloud.google.com/compute/docs/containers/#docker_on_windows then yes, it should work. But I would need to create computer from scratch and install all the tools I need. So I will need some free time.

Alex