Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools: automate major version upgrades for modules #32014

Open
thepudds opened this issue May 13, 2019 · 4 comments

Comments

Projects
None yet
4 participants
@thepudds
Copy link

commented May 13, 2019

This is a spin out of #31543 ("cmd/go: creating v2+ modules has lower success rate than it could").

Background

Semantic import versioning places the major version in module paths and import paths for v2+ modules, such as:

  • module github.com/some/mod/v2 in the author's go.mod.
  • require github.com/some/mod/v2 v2.0.0 in the consumer's go.mod.
  • import "github.com/some/mod/v2/some/pkg" in the consumer's .go files, and in the author's .go files when the module's packages import other packages within the same v2+ module.

This approach has value, but empirically it currently seems it can be a challenge to do correctly the first time (e.g., modules with v2 major version semver tags that are missing the required /v2 in their own module statements, or modules that accidentally do not update all import statements and accidentally depend on the v1 version of themselves, etc.)

It also creates additional work for authors and consumers if a module's major version increments, such as from v1 to v2, or v3 to v4, etc.

Suggestion

Tooling should be able to help here in a substantial way.

github.com/marwan-at-work/mod is great for people who know about it and who are willing to trust it.

However, a tool from the broader community won't have the penetration and impact of something from golang.org, or at least it would likely take much longer to get a similar type of penetration following a typical trajectory for a community tool.

Therefore, the suggestion here is to create a golang.org/x utility that can edit go.mod and .go files to simplify the workflow for authors and consumers of v2+ modules. It might be possible to port marwan-at-work/mod itself, especially with the creation in #31761 of the x/mod repo that exposes APIs for module mechanics (such as an API for go.mod parsing).

Three sample use cases:

  1. If someone is adopting modules for the first time as the author of a v2+ set of packages, ideally the utility would:
    • set the /vN in the module path in the module statement in the go.mod
    • update any import paths in the module's .go files if needed
  2. If the author is later changing the major version for a v2+ module, ideally the utility would:
    • set the /vN in the module path in the module statement in the go.mod
    • update any import paths in the module's .go files if needed
  3. If a consumer wants to use a particular major version of a v2+ module, ideally the utility would:
    • set the require statement properly in the consumer's go.mod
    • update any import paths in the consumer's .go files if needed

I think marwan-at-work/mod can currently do all of those things.

Perhaps one day similar functionality could live in cmd/go (e.g., the closed #27248), but it seems more practical to start with something in golang.org/x.

Non-goals

The suggestion here is that this golang.org/x utility would not do anything with VCS tagging, nor do anything for creating /vN subdirectories for anyone following the "Major Subdirectory" approach. Also, it would probably be reasonable for a first version to error out if used by a consumer that has a replace for the module of interest (or, that could be handled more gracefully).

CC @bcmills @jayconrod @marwan-at-work

@jayconrod jayconrod changed the title golang.org/x utility to automate major version upgrades for modules (perhaps port marwan-at-work/mod) golang.org/x/tools: automate major version upgrades for modules May 14, 2019

@jayconrod jayconrod changed the title golang.org/x/tools: automate major version upgrades for modules x/tools: automate major version upgrades for modules May 14, 2019

@gopherbot gopherbot added this to the Unreleased milestone May 14, 2019

@jayconrod jayconrod modified the milestones: Unreleased, Unplanned May 14, 2019

@jayconrod

This comment has been minimized.

Copy link
Contributor

commented May 14, 2019

I think this would be really valuable to have. Semantic import versioning is a common frustration people have with modules, and anything we could do to make it a better user experience would be helpful.

I'd hesitate to add this functionality to cmd/go. It has a lot of responsibilities, but it's focused on build and dependency management, and it would be good not to expand the scope too far beyond that. That said, there is some precedence in go fix.

We should also consider how this fits into any long-term plans for automatic refactoring. For example, is this something gopls could do? Also, should we provide a way to migrate automatically across incompatible API changes?

cc @ianthehat @stamblerre

@bcmills

This comment has been minimized.

Copy link
Member

commented May 14, 2019

I definitely think it would be useful to provide a tool that can add (or update) the /vN suffix in a module's module directive and update its internal import statements to match.

I'm less convinced on updating consumers; I think an author-side tool like goforward is probably a better fit for that in the general case, although there may be some benefit to reducing indirection on the consumer side. I think the consumer-updating tool should probably be separate: if we're using forwarding declarations anyway, then the v1-to-v2 upgrade looks a lot more like example-based refactoring than mere path-rewriting. (And that provides a way to migrate across incompatible changes, too.)

@jayconrod

This comment has been minimized.

Copy link
Contributor

commented May 15, 2019

I think this is important on the module consumer side, too, but I admit I don't have a sense of how frequent and how inconvenient these upgrades are.

I spoke with @ianthehat this morning. There's some ongoing work adding automated refactoring through gopls. Some of the points below are being actively worked on, others are just ideas.

  • Renaming symbols and other traditional refactoring.
  • Applying suggested changes from the analysis framework.
  • Explaining entries in go.sum.
  • Applying suggested upgrades when editing go.mod.

The last idea was upgrading a particular module, not replacing it with a new major version. But maybe we could work this in, too.

@thepudds

This comment has been minimized.

Copy link
Author

commented May 29, 2019

I think this is important on the module consumer side, too, but I admit I don't have a sense of how frequent and how inconvenient these upgrades are.

I think it is (arguably) even more important to support the consumer side, or at least I wouldn't suggest forgoing the consumer side.

It is not just how frequently the upgrades occur -- the adoption of modules by a v2+ dependency means a consumer needs to go through this process, even if there is no actual major version upgrade involved (or if the "major version" upgrade was solely due to the adoption of modules).

As has been said before, one would hope there are usually more consumers than authors for most open source modules.

I think an author has more moving parts and it is harder for an author to get right, but the consumer pain here is more frequent, I think.

People are currently doing this with things like sed, and frequently are not pleased to be doing so.

For me, the best answer is "the tooling handles it for you".

go fix or goforward could be alternative solutions in the future, but I think there is a lot to be said for a standalone golang.org/x utility targeted at this problem. gopls could be nice, but also seems the plate there is pretty full both on the gopls and editor/IDE sides.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.