Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: add 'go release' #26420

Open
rsc opened this issue Jul 17, 2018 · 28 comments
Open

cmd/go: add 'go release' #26420

rsc opened this issue Jul 17, 2018 · 28 comments

Comments

@rsc
Copy link
Contributor

@rsc rsc commented Jul 17, 2018

Collecting things we would want a 'go release' command to do:

  • API-level compatibility checks
  • warning about dependence on prerelease or pseudo-versions.
  • checking that root/go.mod and root/v2/go.mod don't both say module foo/v2.
@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jul 17, 2018

  • run the tests and make sure they pass
  • add a tag to the source code repo if possible and appropriate
@komuw
Copy link
Contributor

@komuw komuw commented Jul 18, 2018

  • detects api changes and bumps version for you when incompatibility happens.
    similar to http://elm-lang.org/

elm-package diff NoRedInk/json-elm-schema 1.5.0 3.0.0

Comparing NoRedInk/json-elm-schema 1.5.0 to 3.0.0...
This is a MAJOR change.

------ Changes to module JsonSchema - MAJOR ------

    Added:
        additionalItems : JsonSchema.Schema -> JsonSchema.TupleSchemaProperty
        examples : (a -> Json.Encode.Value) -> List a -> JsonSchema.BaseSchemaProperty extras
        maxProperties : Int -> JsonSchema.ObjectSchemaProperty
        minProperties : Int -> JsonSchema.ObjectSchemaProperty
        tuple : List JsonSchema.TupleSchemaProperty -> JsonSchema.Schema
        tupleItems : List JsonSchema.Schema -> JsonSchema.TupleSchemaProperty

    Changed:
      - boolean : List (JsonSchema.BaseSchemaProperty {
                                                      }) -> JsonSchema.Schema
      + boolean : List JsonSchema.BooleanSchemaProperty -> JsonSchema.Schema

      - maxItems : Int -> JsonSchema.ArraySchemaProperty
      + maxItems : Int -> { a |
                            maxItems : Maybe.Maybe Int
                          } -> { a | maxItems : Maybe.Maybe Int }
@crawshaw
Copy link
Contributor

@crawshaw crawshaw commented Jul 21, 2018

for building automated release notes (to encourage releases to have notes):

  • collect any new mentions of Deprecated: in the documentation
  • collect RELNOTES from commit messages
  • collect fixed bugs from commit messages
@mvdan
Copy link
Member

@mvdan mvdan commented Jul 21, 2018

collect RELNOTES from commit messages

Isn't that a bit specific to the Go project itself? I haven't seen it in any other projects. Unless the idea is to start encouraging this practice, which might work out.

@bcmills
Copy link
Member

@bcmills bcmills commented Jul 30, 2018

  • warning about using versions excluded by other dependencies
  • warning about using versions replaced by other dependencies (unless the replacements match)
@thepudds
Copy link

@thepudds thepudds commented Aug 15, 2018

  • Consider a flag or mode to run tests ignoring local replace/exclude directives in the current module in order to test in the configuration users will see if they import as a dependency.

Useful if the current module is intended to be imported as a dependency but uses local replace/exclude directives that will be ignored by the ultimate top-level module.

(This is the use case described in #24666, but that was closed with a 'Fixes #24666' comment in a CL that caused the issue to be closed automatically, but the CL did not implement the described behavior. Not sure if that 'Fixes #24666' comment was accidentally or deliberately left in the broader CL).

@thepudds
Copy link

@thepudds thepudds commented Aug 15, 2018

  • Consider automatically running the equivalent of:
    • go mod verify
    • go mod tidy
  • Consider validating the consistency of the current vendor directory, if any (edit: this was later added as #27348)

I don't know the degree to which the intent is to automatically do things covered by other commands, but wanted to at least record these as options to consider.

@thepudds
Copy link

@thepudds thepudds commented Aug 19, 2018

  • Consider a flag or mode of go release that tests as if you had just done go get -u and have the latest available versions of your dependencies, but where your on-disk go.mod requirements are not actually updated. In other words, without actually changing the versions used in your require directives, simulate what would happen if someone downloaded your module, did go get -u, and then tested.

The overall design of the system is such that a module author might not ready to move to the latest version of dependencies for the module's go.mod, but a module's consumers might move to a more recent set of dependencies of that module.

Coupled with the fact that the module author would already naturally be testing against versions used in the actually released require directives, the flag or mode suggested here for go release would mean the high end (latest available) and low end (versions listed in the released go.mod) would both naturally be tested by the module author before release (or at least, this would make it more easy to do so).

@bcmills
Copy link
Member

@bcmills bcmills commented Sep 25, 2018

  • warning if a replace directive points to a local path in the same repository whose contents have changed since the required version of that module.

(Related to #27542.)

@dmitshur
Copy link
Contributor

@dmitshur dmitshur commented Oct 12, 2018

/cc @bradleyfalzon who created apicompat, a tool to check for the introduction of backwards incompatible changes, which is relevant here.

@Sajmani
Copy link
Contributor

@Sajmani Sajmani commented Oct 12, 2018

Recommend which kind of semver change to make, patch bump or minor bump.

Major bumps (incompatible changes) are more involved, as they require a new import path & work to allow the new major version to coexist with previous ones (e.g., by wrapping & adding aliases, or extracting package-level state into a shared internal package).

@Sajmani
Copy link
Contributor

@Sajmani Sajmani commented Oct 12, 2018

Detect when creating go.mod in a directory under an existing go.mod (is this an error?)

CC @hyangah

@bcmills
Copy link
Member

@bcmills bcmills commented Oct 24, 2018

I think @jba is planning to do some preliminary work, but this needs more design. Pushing to 1.13.

@bcmills bcmills modified the milestones: Go1.12, Go1.13 Oct 24, 2018
@jba
Copy link
Contributor

@jba jba commented Oct 24, 2018

I'm working only on the part that recommends which part of the semver to bump.

@thepudds
Copy link

@thepudds thepudds commented Oct 25, 2018

  • Consider a warning or some mode to check if there are different version requirements for a shared v0 dependency.

For example, if module A requires v0.1.0 of dependency D, and module B requires v0.2.0 of dependency D, then the build will use v0.2.0 of D. However, that might subtly or explicitly break A, given v0 is a "compatibility free zone", and hence this scenario might be worth some flavor of detection or warning.

Related to #28395 and possibly #28396.

@jba
Copy link
Contributor

@jba jba commented Nov 11, 2018

The spec/design doc for API compatibility checks is ready. Implementation is in review.

@komuw
Copy link
Contributor

@komuw komuw commented Jan 8, 2019

If package:

  • has yet to adopt modules and
  • is already tagged v2.0.0 or higher before adopting modules

then:
recommend that the major version should be incremented (when first adopting modules) as this seems to be the best practice[1][2]

  1. #25967 (comment)
  2. https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher
@bcmills
Copy link
Member

@bcmills bcmills commented Jan 10, 2019

  • hard warning if any dependency contains a vulnerability (#24031)
  • softer warning if any dependency contains a known bug or incompatibility (#26829)
@perillo
Copy link
Contributor

@perillo perillo commented Jan 25, 2019

Run tests for the new version of the API/code using the previous version of the test suite.

Requires support from the go test tool, and only black box tests must be executed.

This should catch API incompatibilities due to a change in the implementation.

@thepudds
Copy link

@thepudds thepudds commented Mar 11, 2019

  • Consider a warning if you have a filesystem-based replace directive that reaches outside of the module being released.

In some cases, someone might deliberately want to release with a replace foo => /home/user/foo or similar, but that is probably more often going to be a mistake than purposeful, and hence perhaps worth at least a sanity check or warning by go release. If you want to edit multiple modules at the same time during development, an absolute or relative filesystem-based replace can be a common solution (in some cases via rogpeppe/gohack), but often that is a temporary change to a go.mod file that should not be released. My guess is this would not be caught in all cases by #24666.

@bcmills
Copy link
Member

@bcmills bcmills commented Apr 17, 2019

• hard warning if the module has a transitive dependency on its own path at a version that falls semantically after the proposed tag (#31491)

@thepudds
Copy link

@thepudds thepudds commented Apr 18, 2019

  • Suggestion: consider starting go release as a command in golang.org/x/tools or as golang.org/x/gorelease or similar. This might:

    • help with the looming 1.13 freeze deadline.
    • allow for more iteration as 1.13 matures through beta, rc, 1.13.1, etc., including as community experience grows with modules and as it becomes clearer which theoretical capabilities of go release might be more valuable based on the state of 1.13.
    • provide something of a safety net if something is slightly wrong, given it is easier to update something in golang.org/x than to create a new Go release.
@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Apr 19, 2019

@thepudds That's the plan right now. We're planning to develop this as golang.org/x/tools/cmd/gorelease after the 1.13 freeze starts (too much to do before then). We're hoping to have a prototype available this summer so we can collect feedback. We'll planning to eventually merge it into the Go command as go release in the 1.14 cycle.

@gopherbot
Copy link

@gopherbot gopherbot commented Aug 20, 2019

Change https://golang.org/cl/190904 mentions this issue: x/tools/cmd/gorelease: detect problems before releasing new module versions

@gopherbot
Copy link

@gopherbot gopherbot commented Sep 25, 2019

Change https://golang.org/cl/197299 mentions this issue: x/exp/cmd/gorelease: detect problems before releasing new module versions

@arnottcr
Copy link
Contributor

@arnottcr arnottcr commented Oct 7, 2019

Experience reports from the cl/197299 patch set 4:

Breaking out submodules is not detected. IIRC, this is supported and a process was written up somewhere (include the new submodule as an indirect dependency of the root/old module to prevent import conflicts). It is also currently being done to the cloud.google.com/go packages.

Calculating the exact change set for a PR/change is nontrivial. Currently gorelease -base v0.0.0-20191006222123-1165772a78a0 is the best option I have found, where the provided pseudo-version points to the old version, but generating psudo-versions is hard. Other attempts to resolve this include making testing tags, but they can be accidentally pushed. I would want a way to pass a vcs ref (e.g. HEAD, HEAD~1, origin/master, etc.) as base or via a distinct flag, say -ref, instead.

After migrating to the new semantic import version path, e.g. /v2/, gorelease does not provide any change information from the v0/v1 line. I get why this happens, but it prevents using gorelease to generate changelogs. Sometimes it is possible to jump back in git history to before the commit that did the v1 -> v2 bump, but others there were changes after that commit and it gets tricky.

@gopherbot
Copy link

@gopherbot gopherbot commented Jan 23, 2020

Change https://golang.org/cl/216078 mentions this issue: cmd/gorelease: infer base version if unspecified

gopherbot pushed a commit to golang/exp that referenced this issue Feb 13, 2020
gorelease is an experimental tool that helps module authors avoid
common problems before releasing a new version of a module.

gorelease is intended to eventually be merged into the go command as
"go release".

Updates golang/go#26420

Change-Id: I23fbab171b2cb9f4598fa525ecef5dcf006dc7c4
Reviewed-on: https://go-review.googlesource.com/c/exp/+/197299
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
gopherbot pushed a commit to golang/exp that referenced this issue Feb 28, 2020
If the -base flag is not set on the command line, it is now inferred
from the -version flag and available versions. If the -version flag is
given, the base version will be the highest available release version
lower than -version. Otherwise, the base version will be the highest
release version. Pre-release versions are not considered.

If there are no appropriate versions, and the release version is
unspecified or appears to be the first release (e.g., v0.1.0, v2.0.0),
the inferred base version will be "none" (meaning no comparison will
be made). Otherwise, an error will be shown.

Updates golang/go#26420

Change-Id: Iee45a4183f3e4a219c02a69b5d16a3cc5478644c
Reviewed-on: https://go-review.googlesource.com/c/exp/+/216078
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet