Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

all: update standard-library dependencies at the start and end of each development cycle #36905

Open
bcmills opened this issue Jan 30, 2020 · 64 comments
Labels
early-in-cycle NeedsFix recurring release-blocker
Milestone

Comments

@bcmills
Copy link
Member

@bcmills bcmills commented Jan 30, 2020

The x dependencies vendored into the standard library (via src/go.mod and src/cmd/go.mod) should be updated for each code freeze, so that we can apply any needed fixes to those dependencies without also pulling in unnecessary changes or requiring significant backporting work (see, for example, #36851).

This task, like #11811, #12042, and the API audit (#36167, #32813), should occur regularly in each development cycle. Probably we should update the dependencies at the beginning of each code freeze, and again when we reopen the tree at the end of each code freeze.

CC @golang/osp-team

@bcmills bcmills added this to the Go1.15 milestone Jan 30, 2020
@bcmills bcmills added NeedsFix early-in-cycle labels Jan 30, 2020
@gopherbot
Copy link

@gopherbot gopherbot commented Feb 3, 2020

Change https://golang.org/cl/217517 mentions this issue: all: update module dependencies

@dmitshur
Copy link
Contributor

@dmitshur dmitshur commented Feb 19, 2020

This issue is currently labeled as early-in-cycle for Go 1.15. That time is now, so friendly ping.

gopherbot pushed a commit that referenced this issue Feb 19, 2020
Updates #36905
Updates #36907

Change-Id: I293dcef67800d5c81ff3a254bbd49309c5880710
Reviewed-on: https://go-review.googlesource.com/c/go/+/217517
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
@bcmills bcmills removed the early-in-cycle label Feb 19, 2020
@bcmills
Copy link
Member Author

@bcmills bcmills commented Feb 19, 2020

Completed the early-in-cycle update for the 1.15 cycle. Removing the label so we remember to revisit at the close of the cycle.

@gopherbot
Copy link

@gopherbot gopherbot commented May 1, 2020

Change https://golang.org/cl/231657 mentions this issue: all: update module dependencies

gopherbot pushed a commit that referenced this issue May 4, 2020
The Go 1.15 code freeze has just started. This is the time to update
all golang.org/x/... module versions that contribute packages to the
std and cmd modules in the standard library to latest master versions.

Those versions have already gone through code review, and now they
will undergo additional testing during the freeze period.
If there are new issues in these dependencies discovered, we have
the freeze period to deal with that. By the end of the freeze period,
we will have confidence that the Go 1.15 release and the dependency
versions it has selected are robust.

If one of the Go 1.15.x minor releases requires changing code in one of
the vendored packages, we'll be able to do so on top of the versions
that are selected here, and not be forced to use versions that came
from different time periods, or try to jump across multiple untested
versions in a minor release.

The dependency versions that are selected in this commit are:

	github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3
	github.com/ianlancetaylor/demangle v0.0.0-20200414190113-039b1ae3a340
	golang.org/x/arch v0.0.0-20200312215426-ff8b605520f4
	golang.org/x/crypto v0.0.0-20200429183012-4b2356b1ed79
	golang.org/x/mod v0.2.1-0.20200429172858-859b3ef565e2
	golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5
	golang.org/x/sys v0.0.0-20200501145240-bc7a7d42d5c3
	golang.org/x/text v0.3.3-0.20200430171850-afb9336c4530
	golang.org/x/tools v0.0.0-20200504152539-33427f1b0364
	golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543

github.com/ianlancetaylor/demangle is considered in scope and updated.
github.com/google/pprof is out of scope and was not updated.

For #36905.

Change-Id: Icb6996eb0df11f16edd9a42e04434012c0336354
Reviewed-on: https://go-review.googlesource.com/c/go/+/231657
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
@dmitshur
Copy link
Contributor

@dmitshur dmitshur commented May 4, 2020

Completed the start-of-freeze update for the 1.15 cycle in CL 231657. Moving to 1.16 milestone and adding early-in-cycle.

@dmitshur dmitshur removed this from the Go1.15 milestone May 4, 2020
@dmitshur dmitshur added this to the Go1.16 milestone May 4, 2020
@dmitshur dmitshur added the early-in-cycle label May 4, 2020
@dmitshur dmitshur removed their assignment May 4, 2020
@gopherbot
Copy link

@gopherbot gopherbot commented May 14, 2020

Change https://golang.org/cl/234002 mentions this issue: cmd: update golang.org/x/mod to v0.3.0 (same commit)

gopherbot pushed a commit that referenced this issue May 18, 2020
v0.3.0 is a tag on 859b3ef565e2, the version that was already being
used. This change is a no-op, except for letting us use a release
version instead of a pseudo-version.

For #36905

Change-Id: I70b8ce2a3f1451f5602c469501362d7a6a673b12
Reviewed-on: https://go-review.googlesource.com/c/go/+/234002
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
xujianhai666 pushed a commit to xujianhai666/go-1 that referenced this issue May 21, 2020
The Go 1.15 code freeze has just started. This is the time to update
all golang.org/x/... module versions that contribute packages to the
std and cmd modules in the standard library to latest master versions.

Those versions have already gone through code review, and now they
will undergo additional testing during the freeze period.
If there are new issues in these dependencies discovered, we have
the freeze period to deal with that. By the end of the freeze period,
we will have confidence that the Go 1.15 release and the dependency
versions it has selected are robust.

If one of the Go 1.15.x minor releases requires changing code in one of
the vendored packages, we'll be able to do so on top of the versions
that are selected here, and not be forced to use versions that came
from different time periods, or try to jump across multiple untested
versions in a minor release.

The dependency versions that are selected in this commit are:

	github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3
	github.com/ianlancetaylor/demangle v0.0.0-20200414190113-039b1ae3a340
	golang.org/x/arch v0.0.0-20200312215426-ff8b605520f4
	golang.org/x/crypto v0.0.0-20200429183012-4b2356b1ed79
	golang.org/x/mod v0.2.1-0.20200429172858-859b3ef565e2
	golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5
	golang.org/x/sys v0.0.0-20200501145240-bc7a7d42d5c3
	golang.org/x/text v0.3.3-0.20200430171850-afb9336c4530
	golang.org/x/tools v0.0.0-20200504152539-33427f1b0364
	golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543

github.com/ianlancetaylor/demangle is considered in scope and updated.
github.com/google/pprof is out of scope and was not updated.

For golang#36905.

Change-Id: Icb6996eb0df11f16edd9a42e04434012c0336354
Reviewed-on: https://go-review.googlesource.com/c/go/+/231657
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
@andybons andybons added the recurring label Jul 30, 2020
@dmitshur
Copy link
Contributor

@dmitshur dmitshur commented Aug 12, 2020

This issue is currently labeled as early-in-cycle for Go 1.16.
That time is now, so this is a friendly ping so the issue is looked at again.

@dmitshur
Copy link
Contributor

@dmitshur dmitshur commented Sep 17, 2020

I'm creating a short program that performs this task. Using a program makes it easier to precisely define our update policy, and adjust it over time as needed. It also makes this task less error prone and easier to review. Finally, this can help with #41409 and #36852.

The policy I'm implementing right now is informed by the strategy we used in past manual CLs (e.g., CL 231657), with small adjustments. The policy is for all modules in the GOROOT/src directory (currently, std and cmd), update all direct (not indirect) modules with the import path prefix "golang.org/x/" to their latest tip pseudo-version (which can later become tagged as a release version).

The small adjustment is that the github.com/ianlancetaylor/demangle module won't be automatically updated as part of this process. My rationale is that it's different enough that it probably warrants being updated on its own schedule, when there is an explicit need. /cc @ianlancetaylor

Indirect golang.org/x dependencies are not updated, although they could be. I'm going with the more conservative choice for now. I'm open to feedback on this, and can see pros/cons to both strategies.

I've also considered but left out other modules in the GOROOT tree (e.g., GOROOT/misc) for now.

@dmitshur
Copy link
Contributor

@dmitshur dmitshur commented Sep 17, 2020

Indirect golang.org/x dependencies are not updated, although they could be. I'm going with the more conservative choice for now. I'm open to feedback on this, and can see pros/cons to both strategies.

Looks like we did update indirect dependencies last time. If we don't, it leaves much fewer dependencies to update, because the go.mod files of golang.org/x modules aren't being updated very often.

Will think more about it and likely go with what we did last time as the next step here.

@gopherbot
Copy link

@gopherbot gopherbot commented Sep 18, 2020

Change https://golang.org/cl/255859 mentions this issue: misc/update: add program to update golang.org/x dependencies

@gopherbot
Copy link

@gopherbot gopherbot commented Sep 18, 2020

Change https://golang.org/cl/255860 mentions this issue: all: update vendored dependencies during Go 1.15 development

@dmitshur dmitshur changed the title all: update standard-library dependencies at the start and end of each code freeze all: update standard-library dependencies at the start and end of each development cycle Sep 18, 2020
@gopherbot
Copy link

@gopherbot gopherbot commented Sep 21, 2020

Change https://golang.org/cl/256357 mentions this issue: cmd/updatestd: add program to maintain standard library modules

@dmitshur
Copy link
Contributor

@dmitshur dmitshur commented Nov 9, 2021

CL 361294 updated github.com/google/pprof.
CLs 362734, 362735, 362736, and 361094 updated all golang.org/x dependencies other than x/mod. That is all that's left for the purpose of this issue (other individual updates may also happen, as long as they're bringing in things in scope for the 1.18 freeze).

@dmitshur
Copy link
Contributor

@dmitshur dmitshur commented Nov 16, 2021

CL 364114 took care of x/mod, the remaining bit for this issue.

Moving to 1.19 early in cycle.

@dmitshur dmitshur removed their assignment Nov 16, 2021
@dmitshur dmitshur removed this from the Go1.18 milestone Nov 16, 2021
@dmitshur dmitshur added this to the Go1.19 milestone Nov 16, 2021
@dmitshur dmitshur added the early-in-cycle label Nov 16, 2021
@gopherbot
Copy link

@gopherbot gopherbot commented Mar 16, 2022

This issue is currently labeled as early-in-cycle for Go 1.19.
That time is now, so a friendly reminder to look at it again.

@heschi heschi self-assigned this Mar 16, 2022
@gopherbot
Copy link

@gopherbot gopherbot commented Mar 16, 2022

Change https://go.dev/cl/393370 mentions this issue: cmd: update vendored pprof

@gopherbot
Copy link

@gopherbot gopherbot commented Mar 16, 2022

Change https://go.dev/cl/393373 mentions this issue: all: update vendored golang.org/x/tools

gopherbot pushed a commit that referenced this issue Mar 17, 2022
Update the vendored golang.org/x/tools to pick up the fix for #51717.

This also picks up some changes to support Fuzz tests in the tests
analyzer, but they are currently still guarded by an internal flag.

Fixes #51717
Updates #36905

Change-Id: Ibcd5006624dd9cd9797c811093985e8775c57d51
Reviewed-on: https://go-review.googlesource.com/c/go/+/393373
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
@gopherbot
Copy link

@gopherbot gopherbot commented Mar 17, 2022

Change https://go.dev/cl/393674 mentions this issue: cmd/updatestd: ignore _ directories

gopherbot pushed a commit to golang/build that referenced this issue Mar 17, 2022
This seems like an oversight and it's causing it to update the _asm
submodule in ed25519.

Updates golang/go#36905.

Change-Id: I1f580c2f55028c1012432c704df6ac0133e4f4d6
Reviewed-on: https://go-review.googlesource.com/c/build/+/393674
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
gopherbot pushed a commit that referenced this issue Mar 17, 2022
Generated with x/build/cmd/updatestd.

Updates #36905.

Change-Id: I5d12dfc3b49c1476ce4b8d4cbeb35bd3d3443d26
Reviewed-on: https://go-review.googlesource.com/c/go/+/393369
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
Auto-Submit: Heschi Kreinick <heschi@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
@gopherbot
Copy link

@gopherbot gopherbot commented Mar 17, 2022

Change https://go.dev/cl/393369 mentions this issue: all: update vendored x dependencies for Go 1.19 development cycle

@heschi heschi removed their assignment Mar 18, 2022
gopherbot pushed a commit that referenced this issue Mar 18, 2022
  go get github.com/google/pprof@latest
  go mod vendor

Plus a tiny change to the pprof command to match an upstream interface
change.

Updates #36905.

Change-Id: I4c7bbe8c317a6eeb217fce971b208f96ab0727fa
Reviewed-on: https://go-review.googlesource.com/c/go/+/393370
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
Auto-Submit: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
@dmitshur dmitshur removed the early-in-cycle label May 16, 2022
@dmitshur dmitshur self-assigned this May 16, 2022
@gopherbot
Copy link

@gopherbot gopherbot commented May 16, 2022

Change https://go.dev/cl/406355 mentions this issue: all: update vendored golang.org/x/{arch,sys,term} for Go 1.19 release

@gopherbot
Copy link

@gopherbot gopherbot commented May 16, 2022

Change https://go.dev/cl/406356 mentions this issue: all: update vendored golang.org/x dependencies for Go 1.19 release

gopherbot pushed a commit that referenced this issue May 16, 2022
The Go 1.19 code freeze has recently started. This is a time to update
all golang.org/x/... module versions that contribute packages to the
std and cmd modules in the standard library to latest master versions.

This CL updates only the lower-level modules arch, sys, term for better
bisection. The next CL will update further ones.

For #36905.

Change-Id: I15f6f8b015f8e425571f4f072d6942c806c6ec3e
Reviewed-on: https://go-review.googlesource.com/c/go/+/406355
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
gopherbot pushed a commit that referenced this issue May 16, 2022
The Go 1.19 code freeze has recently started. This is a time to update
all golang.org/x/... module versions that contribute packages to the
std and cmd modules in the standard library to latest master versions.

This CL updates the rest of the modules with x/build/cmd/updatestd.

For #36905.

Change-Id: I4751ca477365b036a8e5ad6a9256293b44ddcd2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/406356
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
@gopherbot
Copy link

@gopherbot gopherbot commented May 17, 2022

Change https://go.dev/cl/406359 mentions this issue: cmd/pprof: update vendored github.com/google/pprof

gopherbot pushed a commit that referenced this issue May 17, 2022
Pull in the latest published version of github.com/google/pprof
as part of go.dev/issue/36905.

Done with:

	go get github.com/google/pprof@upgrade
	go mod tidy
	go mod vendor

For #36905.

Change-Id: I3c8279fce2f20cb940a4e46b2b850703e1fc7964
Reviewed-on: https://go-review.googlesource.com/c/go/+/406359
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
@dmitshur
Copy link
Contributor

@dmitshur dmitshur commented May 17, 2022

CL 406355 and CL 406356 updated golang.org/x dependencies.
CL 406359 updated github.com/google/pprof.
(Other individual updates may also happen, as long as they're bringing in things in scope for the 1.19 freeze, e.g., to fix release-blocking issues, but that's outside of scope if this issue.)

Moving to 1.20 early in cycle.

@dmitshur dmitshur removed their assignment May 17, 2022
@dmitshur dmitshur removed this from the Go1.19 milestone May 17, 2022
@dmitshur dmitshur added this to the Go1.20 milestone May 17, 2022
@dmitshur dmitshur added the early-in-cycle label May 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
early-in-cycle NeedsFix recurring release-blocker
Projects
None yet
Development

No branches or pull requests

8 participants