Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: 'go build' fails when current directory and output directory are symlinks on Windows #27515

Closed
ankeesler opened this issue Sep 5, 2018 · 11 comments

Comments

Projects
None yet
5 participants
@ankeesler
Copy link

commented Sep 5, 2018

What version of Go are you using (go version)?

go version go1.11 windows/amd64

Does this issue reproduce with the latest release?

Yes. This is a regression from go1.10.3.

What operating system and processor architecture are you using (go env)?

Windows Server 1709 (OS Version 10.0.16299) and Windows Server 1803 (OS Version 10.0.17134).

set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Administrator\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Users\Administrator\go
set GOPROXY=
set GORACE=
set GOROOT=C:\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g`<no output>` -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\Administrator\AppData\Local\Temp\go-build075376310=/tmp/go-build -gno-record-gcc-switches

What did you do?

On Windows, create this directory structure.

C:\some-directory\
  hello\
  src\
  src2 [-> symlinked to C:\some-directory\src]
    hello [-> symlinked to C:\some-directory\hello]
    hello.go

From the directory C:\some-directory\src2\, run go build -o c:\some-directory\src2\hello\hello.exe .\hello.go.

What did you expect to see?

The command succeeds and builds hello.exe in the directory C:\some-directory\hello\.

What did you see instead?

go build command-line-arguments: mkdir C:\some-directory\src2\hello\: Cannot create a file when that file already exists.

ankeesler added a commit to cloudfoundry/garden-windows-ci that referenced this issue Sep 5, 2018

Update go build tasks to work around go1.11 symlink issue
See golang/go#27515 for context

[#160287101]

Signed-off-by: Andrew Keesler <akeesler@pivotal.io>
@andybons

This comment has been minimized.

Copy link
Member

commented Sep 5, 2018

@andybons andybons added this to the Unplanned milestone Sep 5, 2018

arjun024 added a commit to cloudfoundry/winc-release that referenced this issue Sep 6, 2018

Revert "Golang: Bump version to 1.11"
This reverts commit 020a875.

We are holding off on upgrading to Go 1.11 until the following symlink
issue is fixed: golang/go#27515.

Signed-off-by: Andrew Keesler <akeesler@pivotal.io>
@alexbrainman

This comment has been minimized.

Copy link
Member

commented Sep 9, 2018

@ankeesler I can reproduce this here, thank you very much.. It is broken on the tip too.

Alex

@andybons andybons added the NeedsFix label Sep 10, 2018

@andybons andybons modified the milestones: Unplanned, Go1.11.1 Sep 10, 2018

@ankeesler

This comment has been minimized.

Copy link
Author

commented Sep 10, 2018

Thanks @andybons + @alexbrainman. Let me know if you need anything else from me, happy to help.

sunjayBhatia added a commit to cloudfoundry/diego-release that referenced this issue Sep 19, 2018

Workaround go1.11 windows symlink issue
Defensively checks if $BOSH_INSTALL_TARGET is a link, if so it installs
binaries in the link's target

See golang/go#27515 for more context

This can be reverted once we have a go version with the fix

[#160103190](https://www.pivotaltracker.com/story/show/160103190)
@rsc

This comment has been minimized.

Copy link
Contributor

commented Sep 20, 2018

I'm sorry this broke but we just fundamentally do not support magic tricks with symlinks. The answer is almost certainly to stop trying to trick the go command using symlinks.

@ankeesler

This comment has been minimized.

Copy link
Author

commented Sep 20, 2018

@rsc thanks for taking a look here. We were hoping that since this works on Linux and it has worked in previous versions on Windows, then it would continue to function the same way going forward. Windows symlinks can certainly by annoying at times (most times). All this being said, is there any chance that this will be fixed in a future go version?

We are working with an interface that expects the directory hierarchy outlined in my initial comment. Since this works on Linux, it will likely be difficult, if not impossible, for us to change the hierarchy. This interface is used in our automated deployment tooling for Kubernetes.

@alexbrainman

This comment has been minimized.

Copy link
Member

commented Sep 22, 2018

I have used https://play.golang.org/p/xYnM-XHKDve to find the commit that broke the test:

e83601b4356f92f2f4d05f302d5654754ff05a6d is the first bad commit
commit e83601b4356f92f2f4d05f302d5654754ff05a6d
Author: Alex Brainman <alex.brainman@gmail.com>
Date:   Sun Jan 7 12:12:25 2018 +1100

    os: use WIN32_FIND_DATA.Reserved0 to identify symlinks

    os.Stat implementation uses instructions described at
    https://blogs.msdn.microsoft.com/oldnewthing/20100212-00/?p=14963/
    to distinguish symlinks. In particular, it calls
    GetFileAttributesEx or FindFirstFile and checks
    either WIN32_FILE_ATTRIBUTE_DATA.dwFileAttributes
    or WIN32_FIND_DATA.dwFileAttributes to see if
    FILE_ATTRIBUTES_REPARSE_POINT flag is set.
    And that seems to worked fine so far.

    But now we discovered that OneDrive root folder
    is determined as directory:

    c:\>dir C:\Users\Alex | grep OneDrive
    30/11/2017  07:25 PM    <DIR>          OneDrive
    c:\>

    while Go identified it as symlink.

    But we did not follow Microsoft's advice to the letter - we never
    checked WIN32_FIND_DATA.Reserved0. And adding that extra check
    makes Go treat OneDrive as symlink. So use FindFirstFile and
    WIN32_FIND_DATA.Reserved0 to determine symlinks.

    Fixes #22579

    Change-Id: I0cb88929eb8b47b1d24efaf1907ad5a0e20de83f
    Reviewed-on: https://go-review.googlesource.com/86556
    Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
    Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
    TryBot-Result: Gobot Gobot <gobot@golang.org>

:040000 040000 6e4ea25b8e56d55d3119d258431eb49b66ac19fc a6eac27eee057861c2fa8254
dc627fa181fd6ae5 M      src

It looks like cmd/go fails when it calls os.MkdirAll, which calls os.Stat of C:\Users\Alex\AppData\Local\Temp\TestAlex496017355\src2\hello\ - note the backslash at the end of the path, and C:\Users\Alex\AppData\Local\Temp\TestAlex496017355\src2 is the symlink. os.Stat of such directory succeeds on linux, and returns directory, but fails on windows.

So this looks like a dup of #27225 or very similar.

If someone wants to have these 2 issues fixed, please, review https://golang.org/cl/134195 .

Alex

yulianedyalkova added a commit to cloudfoundry/garden-runc-release that referenced this issue Sep 28, 2018

Workaround go1.11 windows symlink issue
Temporary workaround for golang/go#27515

Co-authored-by: Julia Nedialkova <julianedialkova@hotmail.com>

davewalter added a commit to cloudfoundry/smb-volume-release that referenced this issue Oct 1, 2018

Workaround go1.11 windows symlink issue
- defensively checks if $BOSH_INSTALL_TARGET is a link
- if so it installs binaries in the link's target

See golang/go#27515 for more context

This can be reverted once we have a go version with the fix

[#160669222](https://www.pivotaltracker.com/story/show/160669222)

davewalter added a commit to cloudfoundry/local-volume-release that referenced this issue Oct 1, 2018

Workaround go1.11 windows symlink issue
- defensively checks if $BOSH_INSTALL_TARGET is a link
- if so it installs binaries in the link's target

See golang/go#27515 for more context

This can be reverted once we have a go version with the fix

[#160669222](https://www.pivotaltracker.com/story/show/160669222)

micahyoung added a commit to cloudfoundry/windows2016fs-online-release that referenced this issue Oct 3, 2018

Workaround go1.11 windows symlink issue
Defensively checks if $BOSH_INSTALL_TARGET is a link, if so it installs
binaries in the link's target

See golang/go#27515 for more context

This can be reverted once we have a go version with the fix

Signed-off-by: Amin Jamali <ajamali@pivotal.io>
Signed-off-by: Yael Harel <yharel@pivotal.io>

micahyoung added a commit to cloudfoundry/winc-release that referenced this issue Oct 3, 2018

Workaround go1.11 windows symlink issue
Defensively checks if $BOSH_INSTALL_TARGET is a link, if so it installs
binaries in the link's target

See golang/go#27515 for more context

This can be reverted once we have a go version with the fix

Signed-off-by: Amin Jamali <ajamali@pivotal.io>
Signed-off-by: Micah Young <myoung@pivotal.io>
@gopherbot

This comment has been minimized.

Copy link

commented Oct 20, 2018

Change https://golang.org/cl/143578 mentions this issue: os: use CreateFile for Stat of symlinks

@alexbrainman

This comment has been minimized.

Copy link
Member

commented Oct 20, 2018

@ankeesler please try https://go-review.googlesource.com/c/go/+/143578

Does it fixes your problem?

Thank you.

Alex

@ankeesler

This comment has been minimized.

Copy link
Author

commented Oct 22, 2018

@alexbrainman thank you for your work on this. We will try this out and let you know.

@ankeesler

This comment has been minimized.

Copy link
Author

commented Oct 22, 2018

@alexbrainman we verified that this change does indeed fix the issue. Thank you.

@alexbrainman

This comment has been minimized.

Copy link
Member

commented Oct 24, 2018

@alexbrainman we verified that this change does indeed fix the issue

Thank you for checking. I will have it reviewed and submitted.

Alex

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.