x/build: trybots should test subrepos against previous Go releases #17626

bradfitz · 2016-10-27T00:05:26Z

Subrepos trybots should test against Go tip, Go 1.N, Go 1.N-1.

dsnet · 2016-10-27T01:38:57Z

What's the general policy for how many versions back we maintain support on a given subrepo?

bradfitz · 2016-10-27T01:39:46Z

In general, the two most recent. (which means between 12-18 months for us)

siennathesane · 2016-12-01T03:54:44Z

I'm new to the language contribution process, but I'd like to help with this issue since it seems like something simple (relative). How can I help with this issue?

bradfitz · 2016-12-01T03:57:26Z

This is a hard one to contribute to because the Go build system is large and has many moving parts, and it's difficult to run without access to our GCP project, and/or paying for it.

The code is in https://github.com/golang/build (mostly in cmd/coordinator) and you can run it in localhost machine. Kinda. But it might require some hackery for local testing. That code bitrots often and needs occasional love. I suspect it's partially broken.

siennathesane · 2016-12-02T05:39:30Z

Alright, I'll see if I can make some progress, and update here if I need help. Do I need a builder hash at all?

bradfitz · 2016-12-02T05:47:53Z

Nope.

siennathesane · 2016-12-11T23:58:57Z

I'm getting this issue when trying to run it locally:

failed to get a token source: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.

Do those creds need to be on the Go build system or will any credentials work?

bradfitz · 2016-12-12T18:47:50Z

You're starting to discover why this is difficult. You shouldn't need any credentials in localhost mode. But we also don't develop the build system in localhost mode, so you probably need to fix it up to work.

siennathesane · 2016-12-12T19:08:40Z

Fix the system to fix the system, can do! I'll create a sub-issue to allow it to work in localhost mode and tackle that first.

bradfitz · 2017-06-16T05:00:19Z

@andybons, after the dev.golang.org migration (#20691), this would be a good second one.

gopherbot · 2017-07-28T19:53:11Z

Change https://golang.org/cl/51590 mentions this issue: maintner/maintnerd, cmd/coordinator: use maintnerd to find TryBot work

Instead of polling Gerrit every 60 seconds, poll maintnerd every second. This should improve TryBot-requested to TryBot-running latency by 30 seconds on average. (Ideally it'd long poll for changes and not even have ~500ms delay, but this is an improvement.) Also, in the process this does most of the work for golang/go#17626. And this cleans up the repo head tracking complication in the coordinator and just asks maintner for it instead. Running benchmarks in the coordinator has been disabled since 2017-06-23 but this CL disables it further, removing some code it'd need to work. But considering that benchmarking would need code repairs to get working again anyway, I consider this acceptable in the short term. I left TODO notes. Updates golang/go#17626 Updates golang/go#19871 Change-Id: Idab43f65a9cca861bffb37748b036a5c9baee864 Reviewed-on: https://go-review.googlesource.com/51590 Reviewed-by: Andrew Bonventre <andybons@golang.org>

heschi · 2018-10-16T20:39:50Z

This would be really nice for x/tools/go/packages, which has very different behavior on 1.10.

gopherbot · 2018-10-19T23:23:28Z

Change https://golang.org/cl/143538 mentions this issue: cmd/coordinator: run subrepo trybots against recent Go release branches

This is the coordinator half of the change to make the coordinator test subrepos against two prior releases of Go. The next change that's required is for the maintnerd API server to return the name of the past two releases. For now this change is a no-op. But once maintnerd starts returning more data, then this change will start causing causing two more linux-amd64 builders per subrepo trybot. Updates golang/go#17626 Change-Id: I1cedbc2e4eee51fb003c8bcc8e072fd10717a833 Reviewed-on: https://go-review.googlesource.com/c/143538 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>

gopherbot · 2018-10-30T21:52:29Z

Change https://golang.org/cl/146137 mentions this issue: maintner: add RPC endpoint for listing Go releases

This change adds an RPC endpoint to maintnerd API server to list the supported Go releases. This is needed and will be used by cmd/coordinator to run subrepo trybots on previous Go release (in addition to current). It's implemented on top of the Gerrit project ref data that the maintner corpus already tracks. A release is considered to be exist for each git tag named "goX", "goX.Y", or "goX.Y.Z". This functionality is also implemented in some other places using data that is further away from the source of truth. Such places can start to use this endpoint instead. Add a subcommand to maintq to invoke the new list Go releases endpoint. Its current output (against a local development maintnerd server): $ go run .../maintner/maintq -server=localhost:6344 list-releases major:1 minor:11 patch:1 tag_name:"go1.11.1" tag_commit:"26957168c4c0cdcc7ca4f0b19d0eb19474d224ac" branch_name:"release-branch.go1.11" branch_commit:"97781d2ed116d2cd9cb870d0b84fc0ec598c9abc" major:1 minor:10 patch:4 tag_name:"go1.10.4" tag_commit:"2191fce26a7fd1cd5b4975e7bd44ab44b1d9dd78" branch_name:"release-branch.go1.10" branch_commit:"e97b7d68f107ff60152f5bd5701e0286f221ee93" Updates golang/go#17626 Change-Id: Ia9ea7f49d421ce0c7a9e85c423aba31572cea52b Reviewed-on: https://go-review.googlesource.com/c/146137 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

gopherbot · 2018-11-02T18:07:13Z

Change https://golang.org/cl/147137 mentions this issue: failing_test: test of new trybot feature

gopherbot · 2018-11-02T19:23:33Z

Change https://golang.org/cl/147198 mentions this issue: maintner/maintnerd/maintapi: test subrepos against previous Go releases

gopherbot · 2018-11-02T19:53:18Z

Change https://golang.org/cl/147200 mentions this issue: maintner/maintnerd: change definition of Go release to require release branch

…e branch Previously, the list-Go-releases RPC endpoint was more flexible and considered a Go release to exist as long as there's a git tag for it, regardless of whether a corresponding release branch existed. Such a situation is very unlikely to come up, and we don't want to cause all users of this API to filter out releases without a release branch. So just re-define a Go release to require a release branch, and make that the API promise. In practice, this should not have any effect because all go tags have corresponding release branches. Updates golang/go#17626 Change-Id: Ia7a8354000483c969e123f0f3605fd360846c40b Reviewed-on: https://go-review.googlesource.com/c/147200 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

dmitshur · 2018-11-03T19:09:56Z

Reopening for deploy and verification.

dmitshur · 2018-11-03T19:33:55Z

Deployed.

According to https://go-review.googlesource.com/c/tools/+/147137#message-af8badf0a004b89d6130ff01bdf798052e2bebc8, it's not working yet. There are still 18 trybots for a subrepo run, that only include the latest Go tip version. /cc @bradfitz

dmitshur · 2018-11-03T19:37:09Z

Whoops. I deployed cmd/coordinator, but for this to take effect I should be deploying maintnerd.

So, this isn't actually deployed yet. Working on that now.

bradfitz · 2018-11-03T19:50:32Z

So it seems to work, but I missed something in the UI. It's using the builder name and not the new Name() string accessor here:

Notice that (and the grid below) is missing the Go version in parens.

bradfitz · 2018-11-03T19:51:18Z

But I do get the nice names in the final email:

bradfitz · 2018-11-03T19:52:23Z

Also, (go branch release-branch.go1.10) is pretty verbose. I clearly didn't think that through.

We could make that accessor clean up the string a bit and just say (Go 1.10).

dmitshur · 2018-11-03T19:53:18Z

Indeed! From https://go-review.googlesource.com/c/tools/+/147137#message-d1a29aad64e636ed4ffcbaf1f2c3f324dc5d5d54:

7 of 20 TryBots failed:
...
Failed on linux-amd64 (go branch release-branch.go1.10): https://storage.googleapis.com/go-build-log/1ae73979/linux-amd64_c18e47da.log
Failed on linux-amd64 (go branch release-branch.go1.11): https://storage.googleapis.com/go-build-log/e8a95aeb/linux-amd64_1798cb41.log
Failed on linux-amd64: https://storage.googleapis.com/go-build-log/f2cd0fa7/linux-amd64_fbab8151.log
...

That's proof it's working as intended! 🎉

I'll consider this issue resolved. Let's open a new one for the UI improvements (edit: opened #28580).

bradfitz · 2018-11-03T19:53:49Z

I'm fine reusing this bug, but new one is also fine.

gopherbot · 2018-11-05T17:53:06Z

Change https://golang.org/cl/147497 mentions this issue: cmd/coordinator: fix another UI issue with trybots against old Go releases

…eases Updates golang/go#28580 Updates golang/go#17626 Change-Id: Ifa88e4ad23e823e5d1a25c46b76afb11b4e7907b Reviewed-on: https://go-review.googlesource.com/c/147497 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>

bradfitz added the Builders x/build issues (builders, bots, dashboards) label Oct 27, 2016

bradfitz added this to the Unreleased milestone Oct 27, 2016

bradfitz self-assigned this Oct 27, 2016

siennathesane mentioned this issue Dec 12, 2016

x/build: fix coordinator's localhost dev mode; should not require GCE #18291

Closed

dmitshur mentioned this issue Feb 21, 2017

build: breaking change in go-github #19208

Closed

bradfitz changed the title ~~x/build: test subrepos against previous Go releases~~ x/build: trybots should test subrepos against previous Go releases Apr 24, 2017

bradfitz added the help wanted label Apr 24, 2017

bradfitz mentioned this issue May 19, 2017

x/talks: build broken for !go1.9 after State of Go May 2017 #20424

Closed

bradfitz assigned andybons Jun 16, 2017

bradfitz mentioned this issue Jul 31, 2018

x/proto: create repo and setup Gerrit #26723

Closed

dmitshur removed the help wanted label Nov 2, 2018

gopherbot closed this as completed in golang/build@906dcfd Nov 3, 2018

dmitshur reopened this Nov 3, 2018

bradfitz assigned dmitshur and unassigned andybons Nov 3, 2018

dmitshur closed this as completed Nov 3, 2018

dmitshur mentioned this issue Nov 3, 2018

x/build: trybots testing subrepos against previous Go releases UI improvements #28580

Closed

2 tasks

dmitshur mentioned this issue Nov 6, 2018

x/tools/internal/lsp: TestDiagnostics/GOPATH is failing on Go 1.10 #28581

Closed

bradfitz mentioned this issue Nov 11, 2018

x/build/cmd/coordinator: some CLs exp subrepo left strange active trybots running #28714

Closed

This was referenced Jan 2, 2019

x/build/cmd/coordinator: subrepo trybots testing against security branch #29510

Closed

x/build/cmd/coordinator: don't try trybots for oauth2, even for 1.10 and 1.11 branches #29637

Closed

golang locked and limited conversation to collaborators Nov 5, 2019

gopherbot added the FrozenDueToAge label Nov 5, 2019

rsc unassigned bradfitz and dmitshur Jun 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

x/build: trybots should test subrepos against previous Go releases #17626

x/build: trybots should test subrepos against previous Go releases #17626

bradfitz commented Oct 27, 2016 •

edited

Loading

dsnet commented Oct 27, 2016

bradfitz commented Oct 27, 2016

siennathesane commented Dec 1, 2016

bradfitz commented Dec 1, 2016 •

edited

Loading

siennathesane commented Dec 2, 2016

bradfitz commented Dec 2, 2016

siennathesane commented Dec 11, 2016

bradfitz commented Dec 12, 2016

siennathesane commented Dec 12, 2016

bradfitz commented Jun 16, 2017

gopherbot commented Jul 28, 2017

heschi commented Oct 16, 2018

gopherbot commented Oct 19, 2018

gopherbot commented Oct 30, 2018

gopherbot commented Nov 2, 2018

gopherbot commented Nov 2, 2018

gopherbot commented Nov 2, 2018

dmitshur commented Nov 3, 2018

dmitshur commented Nov 3, 2018

dmitshur commented Nov 3, 2018

bradfitz commented Nov 3, 2018

bradfitz commented Nov 3, 2018

bradfitz commented Nov 3, 2018

dmitshur commented Nov 3, 2018 •

edited

Loading

bradfitz commented Nov 3, 2018

gopherbot commented Nov 5, 2018

x/build: trybots should test subrepos against previous Go releases #17626

x/build: trybots should test subrepos against previous Go releases #17626

Comments

bradfitz commented Oct 27, 2016 • edited Loading

dsnet commented Oct 27, 2016

bradfitz commented Oct 27, 2016

siennathesane commented Dec 1, 2016

bradfitz commented Dec 1, 2016 • edited Loading

siennathesane commented Dec 2, 2016

bradfitz commented Dec 2, 2016

siennathesane commented Dec 11, 2016

bradfitz commented Dec 12, 2016

siennathesane commented Dec 12, 2016

bradfitz commented Jun 16, 2017

gopherbot commented Jul 28, 2017

heschi commented Oct 16, 2018

gopherbot commented Oct 19, 2018

gopherbot commented Oct 30, 2018

gopherbot commented Nov 2, 2018

gopherbot commented Nov 2, 2018

gopherbot commented Nov 2, 2018

dmitshur commented Nov 3, 2018

dmitshur commented Nov 3, 2018

dmitshur commented Nov 3, 2018

bradfitz commented Nov 3, 2018

bradfitz commented Nov 3, 2018

bradfitz commented Nov 3, 2018

dmitshur commented Nov 3, 2018 • edited Loading

bradfitz commented Nov 3, 2018

gopherbot commented Nov 5, 2018

bradfitz commented Oct 27, 2016 •

edited

Loading

bradfitz commented Dec 1, 2016 •

edited

Loading

dmitshur commented Nov 3, 2018 •

edited

Loading