cmd/go: go fmt fails in symlinked directory

[sguillia]

What version of Go are you using (go version)?

$ go version
go version go1.12 linux/amd64

Does this issue reproduce with the latest release?

If latest release is go 1.12 then yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/rof/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/rof/"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build784063998=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I have one directory which is a clone of my git repo. Let's call it project.
I have a symbolic link to that directory. Let's call it link.

This works fine

$ cd src/github.com/.../project
$ go fmt

This does not work fine

$ cd link
$ go fmt
stat ../src/github.com/.../main.go: no such file or directory
exit status 2

What did you expect to see?

I expected go fmt to work with no error.

What did you see instead?

I saw a no such file or directory error, but the file exists.

Additional information

$ strace go fmt |& grep \\.\\./src
waitid(P_PID, 10294, stat ../src/github.com/.../main.go: no such file or directory

$ strace go fmt |& grep \\.\\./src
futex(0xe474e8, FUTEX_WAIT_PRIVATE, 0, NULLstat ../src/github.com/.../main.go: no such file or directory

go fmt in the link directory fails 100% of the time.
50% of time, futex fails.
50% of time, waitid fails.

This runs in the docker environment of my cloud CI, Codeship. I am reporting the bug here since everything else works fine.

Thank you

[ianlancetaylor]

Note that you are misreading the strace output. The strace is blocking on the waitid or futex call, and then the program prints an error message. For Go programs you must always use strace -f, and that will likely give you clearer output.

[ianlancetaylor]

In general symbolic links are problematic for the go tool. It needs to know the import path of a directory, and using a symbolic link breaks that. I don't know if this particular case can be handled better.

[sguillia]

Sorry about this. The correct strace output is:

$ strace -f go fmt |& grep \.\./src | grep main.go | grep ENOENT
[pid  5595] newfstatat(AT_FDCWD, "../src/github.com/.../main.go", 0xc0000c65e8, 0) = -1 ENOENT (No such file or directory)

Note that the problem does not occut with all symlinks (some of them just work). I did not manage to identify exactly why some do and why others don't.

[bcmills]

There have been a number of symlink-related fixes since the go1.12 release. Does this still reproduce using go1.12.6? What about a go command built from head?

(You can obtain runnable wrappers for those using go get golang.org/dl/go1.12.6 and go get golang.org/dl/gotip respectively.)

[sguillia]

I installed go1.12.16 and it still reproduces

$ ~/bin/go1.12.6 fmt
stat ../src/github.com/.../main.go: no such file or directory
exit status 2

I have to investigate a bit further why it works with some symlinks, please leave open, I will try to find more info soon.

[sguillia]

So I managed to reproduce on my machine...

The problem happens only when source code is in a directory called src and GOPATH points to its parent.

It happened that my sources were in an src directory in my home.
It also happened that GOPATH was set by default to my home.
It also happened that I work from a symlink of my sources.

Steps to reproduce

$ env | grep GO
GOROOT=/usr/local/go
GOPATH=/home/rof/

$ cd ~

$ mkdir -p src/alpha/bravo

$ cp /tmp/main.go ./src/alpha/bravo/

$ ln -s /home/rof/src/alpha/bravo ./link

$ cd link

$ go fmt
stat ../src/alpha/bravo/main.go: no such file or directory
exit status 2

$ ls ../src/alpha/bravo/main.go
ls: cannot access '../src/alpha/bravo/main.go': No such file or directory

$ ls ..
bravo

$ unset GOPATH

$ go fmt

Once I am in link, .. does not point to my home anymore. I think Go takes a wrong assumption here. It should not resolve the symlink in that way (or it should not resolve it at all). It should not behave differently if the source dir is called src.

Or maybe this is the intended behavior of setting GOPATH to home but this is not intuitive (it was set by default by my CI and it is hard to debug).

Thank you

[bcmills]

Thanks. What about a go command built from head, or go1.13beta1?

[sguillia]

File reproduce.sh

#!/bin/bash
export GOPATH=/tmp/test

gotip version

mkdir -p /tmp/test || true

cd /tmp/test

mkdir -p src/alpha/bravo

echo "package main" > ./src/alpha/bravo/main.go

ln -s /tmp/test/src/alpha/bravo ./link 2>&- || true

cd link

gotip fmt

Output

/tmp/test/reproduce.sh 
go version devel +67f181b Thu Jun 27 22:31:35 2019 +0000 linux/amd64
stat ../src/alpha/bravo/main.go: no such file or directory
exit status 2