Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: set up WSL Windows Subsystem For Linux (Linux-on-Windows) builder #17365

Closed
quentinmit opened this issue Oct 6, 2016 · 46 comments
Closed

Comments

@quentinmit
Copy link
Contributor

@quentinmit quentinmit commented Oct 6, 2016

Even if it's not a first-class port, it's useful to know if/how it's broken.

@quentinmit quentinmit added this to the Go1.9Early milestone Oct 6, 2016
@quentinmit quentinmit self-assigned this Oct 6, 2016
@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Oct 6, 2016

I'd prefer a Windows 10 builder (or "Windows Server 2016") first, which we don't even have.

But it's not offered on GCE yet: https://cloud.google.com/compute/docs/instances/windows/#windows_server (only 2008 and 2012).

So unless we want to run builders on Azure or our Mac VMWare cluster, this is blocked for now.

@quentinmit
Copy link
Contributor Author

@quentinmit quentinmit commented Oct 18, 2016

@bradfitz
Windows Server 2016 is now available on GCE

@quentinmit
Copy link
Contributor Author

@quentinmit quentinmit commented Oct 18, 2016

However, more relevantly for this bug, Windows Server 2016 does not support Linux on Windows yet.

@quentinmit
Copy link
Contributor Author

@quentinmit quentinmit commented Oct 20, 2016

I ran ./run.bash -no-rebuild and the output is attached here. Obviously, we have a ways to go. @aclements thinks many of the failures may be due to our signal handling, where we apparently modify siginfo_t and expect that to control where we return from the signal handler.
go-run.txt

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Oct 20, 2016

"We" don't have a ways to go. We target Linux.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Oct 20, 2016

Yes, what we do is modify the stack to look like a call to runtime.sigpanic, and then set the stack pointer and PC in the ucontext_t structure passed to the signal handler. When we return from the signal handler, we expect that to execute the sigreturn system call which should then re-enter the user program with the stack pointer and PC set to the values in the ucontext_t (on some systems we execute sigreturn directly). On tip this is done in sigctxt.preparePanic, which has CPU-specific versions in signal_GOARCH.go and for modifying the ucontext_t struct relies on support in signal_GOOS_GOARCH.go.

Looking at your log that is at least partially working. I see that the signal handler is indeed returning to sigpanic as it should. What seems to be failing is that g.sigcode0 does not have the expected value in sigpanic. That field is set from the si_code field of the siginfo_t passed to the signal handler. If you want to pursue this further, change sigpanic to print out the g.sigcode0 field.

@quentinmit
Copy link
Contributor Author

@quentinmit quentinmit commented Oct 21, 2016

@bradfitz: Agreed, this is mostly Microsoft's problem, not ours. I think we should have the goal of Go running on Linux-on-Windows, and we should invest (some) time in debugging problems with that configuration. If the problem is caused by our code, we should change our code. If the problem is a bug in the Windows kernel, we should file a bug upstream and not work around it in Go. Likely most/all of the test failures are kernel bugs, but we should at least do the debugging to find out what the kernel bugs are so we can file issues upstream.

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Oct 22, 2016

If the problem is caused by our code, we should change our code.

I think by definition is can not be a problem with our code, ever.

Microsoft has committed to emulating Linux, which isn't a spec. We're targeting Linux, not a spec. If our Linux builders are passing, it's not our fault.

So the onus is on Microsoft to forever play catch-up and react to Linux differences.

But I agree we can help report bugs to Microsoft, and it would be nice to see it work.

@peterGo
Copy link
Contributor

@peterGo peterGo commented Feb 21, 2017

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Feb 21, 2017

@peterGo, I don't care about their marketing department. They implement the Linux system calls. There's nothing specific to Ubuntu here other than marketing.

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Apr 30, 2017

Update: we have Windows Server 2016 builders now. Maybe this is easier with that?

Or maybe we need to wait for the Nano server builders? #15287

/cc @johnsonj

@quentinmit
Copy link
Contributor Author

@quentinmit quentinmit commented May 1, 2017

@bradfitz My understanding is that Linux-on-Windows is not yet available for Windows Server 2016, because it's "not intended for server use".

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented May 1, 2017

Hah. Interesting.

@johnsonj
Copy link
Contributor

@johnsonj johnsonj commented May 1, 2017

src

I wonder if there's a workaround to enable the feature? I'm not sure if we can run desktop SKUs on GCP.

@johnsonj
Copy link
Contributor

@johnsonj johnsonj commented May 13, 2017

And now that's going to change: WSL on Windows Server

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented May 13, 2017

Hah... I asked @bitcrazed about this at OSCON on Wednesday about an hour before this was announced. Props to Rich for managing to not spill the beans.

@mengzhuo
Copy link
Contributor

@mengzhuo mengzhuo commented Nov 17, 2020

@johnsonj Hi, any news?

@johnsonj johnsonj removed their assignment Nov 17, 2020
@johnsonj
Copy link
Contributor

@johnsonj johnsonj commented Nov 17, 2020

Thanks for the ping @mengzhuo !

I'm unassigning as I am not actively working on this.

@mengzhuo
Copy link
Contributor

@mengzhuo mengzhuo commented Nov 18, 2020

@bradfitz @bcmills Could you guys assigned this issue to me? I've got a spare Win10 NUC for this builder.

@bcmills
Copy link
Member

@bcmills bcmills commented Nov 18, 2020

@mengzhuo, see http://golang.org/wiki/DashboardBuilders#how-to-set-up-a-builder for setup instructions; you'll need to coordinate with @golang/release to get a builder key.

@mengzhuo
Copy link
Contributor

@mengzhuo mengzhuo commented Nov 20, 2020

@bcmills I've send the request on golang-dev.

@cagedmantis
Copy link
Contributor

@cagedmantis cagedmantis commented Nov 20, 2020

@mengzhuo I've responded to you via email with the host and private builder key. A list of next steps are included in the email as well.

@gopherbot
Copy link

@gopherbot gopherbot commented Nov 22, 2020

Change https://golang.org/cl/272266 mentions this issue: dashboard: add new linux-amd64-wsl builder

@gopherbot
Copy link

@gopherbot gopherbot commented Nov 23, 2020

Change https://golang.org/cl/272267 mentions this issue: cmd/buildlet: add linux-amd64-wsl to stage0

@mengzhuo
Copy link
Contributor

@mengzhuo mengzhuo commented Nov 23, 2020

@cagedmantis Hi, I've uploaded 2 CLs for adding this builder.

The builder is now runing the build service and ready to go :)

gopherbot pushed a commit to golang/build that referenced this issue Nov 25, 2020
Updates golang/go#17365

Change-Id: I90b9c1c8f82ae9164deacea36ff5dafd008eef03
Reviewed-on: https://go-review.googlesource.com/c/build/+/272266
Trust: Meng Zhuo <mzh@golangcn.org>
Run-TryBot: Meng Zhuo <mzh@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
@mengzhuo
Copy link
Contributor

@mengzhuo mengzhuo commented Nov 25, 2020

I've submitted builder to repo, however I found the binary from googlesource still end up like the old one. Is there any step I'm missing?

Nov 25 08:57:07 PN50 stage0[167743]: 2020/11/25 08:57:07 Dialing coordinator farmer.golang.org:443 ...
Nov 25 08:57:17 PN50 stage0[167743]: 2020/11/25 08:57:17 dialing proxy "proxy:8123" ...
Nov 25 08:57:17 PN50 stage0[167743]: 2020/11/25 08:57:17 Dialed coordinator farmer.golang.org:443.
Nov 25 08:57:17 PN50 stage0[167743]: 2020/11/25 08:57:17 Doing TLS handshake with coordinator (verifying hostname "farmer.golang.org")...
Nov 25 08:57:17 PN50 stage0[167743]: 2020/11/25 08:57:17 Registering reverse mode with coordinator...
Nov 25 08:57:18 PN50 stage0[167743]: 2020/11/25 08:57:18 Connected to coordinator; reverse dialing active
Nov 25 08:57:18 PN50 stage0[167743]: 2020/11/25 08:57:18 Dialing coordinator farmer.golang.org:443 ...
Nov 25 08:57:28 PN50 stage0[167743]: 2020/11/25 08:57:28 buildlet reverse mode exiting.
Nov 25 08:57:28 PN50 systemd[1]: go-builder.service: Succeeded.

@mengzhuo
Copy link
Contributor

@mengzhuo mengzhuo commented Dec 2, 2020

Ping

@bcmills
Copy link
Member

@bcmills bcmills commented Dec 2, 2020

@golang/release, is this just waiting on a cmd/coordinator push?

@dmitshur dmitshur changed the title build: set up WSL Windows Subsystem For Linux (Linux-on-Windows) builder x/build: set up WSL Windows Subsystem For Linux (Linux-on-Windows) builder Dec 2, 2020
@cagedmantis
Copy link
Contributor

@cagedmantis cagedmantis commented Dec 2, 2020

We can plan for a deployment of the coordinator. The buildlet should connect (and maintain a connection to the coordinator without a deployment though).

@cagedmantis
Copy link
Contributor

@cagedmantis cagedmantis commented Dec 2, 2020

@mengzhuo The coordinator has been deployed. That builder should be up and running.

@mengzhuo
Copy link
Contributor

@mengzhuo mengzhuo commented Dec 3, 2020

@mengzhuo The coordinator has been deployed. That builder should be up and running.

Thanks! It's up and testing CL now!

@oiooj
Copy link
Member

@oiooj oiooj commented Dec 22, 2020

I have talked with @mengzhuo add another new WSL builder in host-linux-amd64-wsl pool to improve availability.

@gopherbot
Copy link

@gopherbot gopherbot commented Dec 22, 2020

Change https://golang.org/cl/278874 mentions this issue: dashboard: add a new wsl builder to improve service availability

gopherbot pushed a commit to golang/build that referenced this issue Jan 8, 2021
For golang/go#17365

Change-Id: Icf776550e3f17d86bf0f74d5598912fe1a18cb50
Reviewed-on: https://go-review.googlesource.com/c/build/+/278874
Run-TryBot: Baokun Lee <bk@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Meng Zhuo <mzh@golangcn.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Trust: Bryan C. Mills <bcmills@google.com>
@oiooj
Copy link
Member

@oiooj oiooj commented Jan 14, 2021

The two builders for WSL are up now.

@oiooj oiooj closed this Jan 14, 2021
gopherbot pushed a commit to golang/build that referenced this issue Apr 23, 2021
Update golang/go#17365

Change-Id: Iae9cce9b356790cb643f2f0a7ba637f90f185ebd
Reviewed-on: https://go-review.googlesource.com/c/build/+/272267
Trust: Meng Zhuo <mzh@golangcn.org>
Run-TryBot: Meng Zhuo <mzh@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet