Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: fails when main module is in a 9P network share on Windows Subsystem for Linux #37461

Open
JulienTant opened this issue Feb 26, 2020 · 21 comments

Comments

@JulienTant
Copy link

@JulienTant JulienTant commented Feb 26, 2020

What version of Go are you using (go version)?

$ go version
go version go1.14 windows/amd64

Does this issue reproduce with the latest release?

It is the last release

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GO111MODULE=on
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\a\AppData\Local\go-build
set GOENV=C:\Users\a\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS= -mod=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\a\go
set GOPRIVATE=
set GOPROXY=direct
set GOROOT=C:\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=Z:\home\a\go\graph\go.mod
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\a\AppData\Local\Temp\go-build115528494=/tmp/go-build -gno-record-gcc-switches

What did you do?

I'm using Windows Subsystem for Linux 2 and I access my files using a network driver. I tried to use go mod init in a new project and got this error :

a@Hackintosh-Windows MINGW64 /z/home/a/go/graph
$ go mod init graph
go: creating new go.mod: module graph
go: updating go.mod: Lock Z:\home\a\go\graph\go.mod: Fonction incorrecte.

(Fonction incorrecte means incorrect/invalid function)

Then i tried to create it by end with the content :

module graph

go 1.14

Running go list -m (or any command requiring this file actually) gives me this error : go: RLock Z:\home\a\go\graph\go.mod: Fonction incorrecte.

What did you expect to see?

my go.mod file created.

What did you see instead?

An error about Lock/RLock.

More information:

After reinstalling go 1.13.8, i had everything working back.

@ALTree ALTree changed the title Unable to use network drive on windows since go 1.14 cmd/go: unable to use network drive on Windows Subsystem for Linux since go 1.14 Feb 26, 2020
@ALTree
Copy link
Member

@ALTree ALTree commented Feb 26, 2020

Thanks for the report.

The Windows Subsystem for Linux is not a supported platform. The fact that it accidentally worked on Go1.13 does not necessarily mean that it will keep working going forward. We don't have a builder (since it's not a supported platform), so breakage may happen.

I'm closing this issue, but please feel free to comment if you can reproduce this on plain Windows or on Linux; and we'll re-open.

@ALTree ALTree closed this Feb 26, 2020
@artsafin
Copy link

@artsafin artsafin commented Jun 2, 2020

Can the support of WSL2 be considered or included into roadmap?

WSL2 is becoming a popular platform as it soothes many pains of development under Windows.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jun 2, 2020

@artsafin See https://golang.org/wiki/PortingPolicy .

That said, I thought the goal of WSL was to permit running GNU/Linux programs on Windows. It ought to work to run GOOS=linux go build to get a GNU/Linux binary and then run that binary under WSL.

@Wintereise
Copy link

@Wintereise Wintereise commented Jun 9, 2020

That said, I thought the goal of WSL was to permit running GNU/Linux programs on Windows. It ought to work to run GOOS=linux go build to get a GNU/Linux binary and then run that binary under WSL.

The problem here is when your code lives within the WSL2 VM, but you're working on it via the \wsl$ network share in your IDE / toolchain. A lot of us have adopted this since it allows us to code in Windows, but do all testing / verification in Linux.

The go binary fails to lock the needed .mod or .sum files when called with the wsl$ network path and subsequently fails. I suspect this will probably happen with all network shares in Windows.

@JulienTant
Copy link
Author

@JulienTant JulienTant commented Jun 13, 2020

It's just that it's not happening with go 1.13, it would be nice to be able to disable the rlock feature

@ld100
Copy link

@ld100 ld100 commented Jun 22, 2020

Another one here with the same problem. This issue makes working in Goland a chore... Each time I add a library to the project I have to copy the project to the Windows drive and download modules there...

The issue is not about WSL2 itself, but about Go being able to work on Windows with files on the share via P9 protocol.

@artsafin
Copy link

@artsafin artsafin commented Jun 25, 2020

It's possible to workaround the issue by launching IDE inside WSL connecting it to X server installed on Windows host.

However native support of working on network shares would be much appreciated.

@jaytaph
Copy link

@jaytaph jaytaph commented Oct 9, 2020

Many more people are in need of this. Having an option to disable the file locking would make it work

kamijin-fanta added a commit to kamijin-fanta/go that referenced this issue Dec 4, 2020
Some file systems do not support file locking.
Ignore lock errors in these environments.
I used the already implemented IsNotSupported function.

If the file system does not support locking on Windows,
ERROR_INVALID_FUNCTION is returned.
Edited isNotSupported to ignore this.

References golang#37461
@gopherbot
Copy link

@gopherbot gopherbot commented Dec 4, 2020

Change https://golang.org/cl/275392 mentions this issue: cmd/go/internal/lockedfile: ignore errors if locks are not supported

@bcmills
Copy link
Member

@bcmills bcmills commented Dec 4, 2020

We don't have a builder (since it's not a supported platform), so breakage may happen.

As it just so happens, we now have a WSL builder thanks to @mengzhuo (#17365 (comment)), so I think we can reopen this issue.

@bcmills bcmills reopened this Dec 4, 2020
@bcmills bcmills changed the title cmd/go: unable to use network drive on Windows Subsystem for Linux since go 1.14 cmd/go: fails when main module is in a 9P network share on Windows Subsystem for Linux Dec 4, 2020
@bcmills bcmills added this to the Unplanned milestone Dec 4, 2020
@bcmills
Copy link
Member

@bcmills bcmills commented Dec 4, 2020

(That said, the cmd/go tests are currently passing on the WSL builder because the tests are not using a network share for the working directory.)

@bcmills
Copy link
Member

@bcmills bcmills commented Dec 4, 2020

@ld100

The issue is not about WSL2 itself, but about Go being able to work on Windows with files on the share via P9 protocol.

Note that the 9P protocol does support file-locking — cmd/go's internal/lockedfile package has a Plan 9 implementation and it seems to work reliably. (The Go project's Plan 9 builders are failing frequently at the moment, but the cause of the failures does not appear to be related to file-locking.)

So it seems that the root problem here is that the Windows filesystem APIs are incomplete for the 9P filesystem, for which @kamijin-fanta has helpfully filed microsoft/WSL#5762.

@bcmills
Copy link
Member

@bcmills bcmills commented Dec 4, 2020

Having an option to disable the file locking would make it work

The go command locks files in order to prevent data corruption in both the Go module cache (#26794) and the main module's go.mod and go.sum files.

We had originally only used file locks in the module cache, instead preferring atomic renames for the go.mod and go.sum updates. However, the rename approach resulted in spurious changes to file permissions (#34634) and spurious errors on Windows due to certain idiosyncrasies of the Windows file APIs (#32188, #36568). So in Go 1.14 we switched to using file locks for the go.mod and go.sum updates as well.

@jaytaph
Copy link

@jaytaph jaytaph commented Dec 4, 2020

(That said, the cmd/go tests are currently passing on the WSL builder because the tests are not using a network share for the working directory.)

Is there a way to get involved in setting this up? I do have the time for it, but I don't know how to contribute to this.

@bcmills
Copy link
Member

@bcmills bcmills commented Dec 4, 2020

Honestly, this seems like a niche enough use-case that I would be satisfied with a test that only fails locally. (This particular part of cmd/go seems pretty stable and is unlikely to regress.)

That said, since this is an upstream bug (an implementation deficiency in the WSL2 filesystem) and there is what seems like a straightforward workaround in the interim (use a working directory and GOMODCACHE outside of the 9P share), I'm not sure that it's worth adding a lot of complexity to cmd/go itself to deal with this case.

@jaytaph
Copy link

@jaytaph jaytaph commented Dec 4, 2020

I also do think that adding more complexity than needed is not a good thing. However, PR #42994 seems a good way to deal with this issue (and others) when locking is not available (for some reason). Once (IF!) wsl2 implements plan9 locking correctly, no need for any changes would be needed, and we would be safe for other types of file-systems that will be thrown in the wsl (or other system) pool that we do not know of.

@bcmills
Copy link
Member

@bcmills bcmills commented Dec 4, 2020

@jaytaph

Once (IF!) wsl2 implements plan9 locking correctly, no need for any changes would be needed,

We already should not require any changes once the Windows 9P implementation supports LockFileEx correctly.

and we would be safe for other types of file-systems that will be thrown in the wsl (or other system) pool that we do not know of.

Given that ignoring file-locking errors can lead to silent corruption of the affected files, “safe” is definitely not the word I would use. 😒

@kamijin-fanta
Copy link

@kamijin-fanta kamijin-fanta commented Dec 4, 2020

Please tell me, where can I find the test code for plan9? I thought it wasn't working properly in plan9.

@bcmills
Copy link
Member

@bcmills bcmills commented Dec 4, 2020

@kamijin-fanta, the tests in lockedfile_test.go and transform_test.go should run on Plan 9 the same as on every other platform.

There is an additional integration test in mod_concurrent.txt that I believe also runs on Plan 9.

@bcmills
Copy link
Member

@bcmills bcmills commented Dec 4, 2020

(The last Plan 9 file-locking bug I'm aware of was #38719, which we fixed by using lockedfile.Read in even more places. 😅)

@kamijin-fanta
Copy link

@kamijin-fanta kamijin-fanta commented Dec 7, 2020

Thanks to you, I understand that Plan 9 implements a lock that doesn't use filelock.Lock here.
https://github.com/golang/go/blob/go1.15.6/src/cmd/go/internal/lockedfile/lockedfile_plan9.go

If the file system does not support locking, can it be implemented using lock files? Create a lock file in the TMP directory, etc. to control the lock. I'm thinking of implementing apt like /var/lib/apt/lists/lock. But this is an ad hoc fix for only WSL.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
10 participants
You can’t perform that action at this time.