proposal: a version numbering scheme and release process for Go projects

Preface

Go projects do not have version numbers in the way it is commonly understood by our counterparts in other languages communities. This is because there is no formalised notion of releasing a Go project. There is no process of taking an arbitrary vcs commit hash and assigning it a version number that is both meaningful for humans and machines.

Additionally, Operating System distributors such as Debian and Ubuntu strongly prefer to package released versions of a project, and are currently reduced to doing things like this.

To put it another way,

Go projects are commonly tracked in vcs repositories and derive their import prefix from their vcs location. Version control systems assign revision identifiers or hashes to various copies of the source they track over time. These vcs hashes are ideal for identifying a particular copy, or revision, of a Go project. However vcs hashes are less useful at answering other types of question like:

"Am I using the latest copy of this project ?"
"Are there any bug fixes for version 1.5 available ?"
"Which version of the postgres driver are compatible with this library ?"

The aim of this proposal is to establish a single recommended procedure for releasing Go projects, in the same way that gofmt defines a single recommended way to format Go source code.

Proposal

This proposal seeks to establish a recommended version numbering scheme and minimal release process for Go projects.

A Go project is a collection of one or more Go packages whose source is tracked in a vcs repository.

A repository root, as defined by cmd/go, identifies both the unique import path prefix for the Go project, and the vcs repository that holds the project's source.

This proposal describes a release process for Go projects by tagging the repository which hold the project's code.

This process is intended to be light weight and will facilitate the creation of tools that automate the creation and consumption of released versions of Go projects.

Version numbering scheme

This proposal recommends that Go projects adopt the semantic versioning 2.0 standard for their version numbering scheme.

This recommendation is informed by the broad support for semantic versioning across our contemporaries like node.js (npm), rust (cargo), javascript (bower), and ruby (rubygems). Adherence to a commonly accepted ideal of what constitutes a major, minor, or patch release will allow Go programmers to benefit from the experiences of these other communities' dependency management ecosystems.

Tag format

Furthermore, this proposal recommends that Go projects adopt a process of releasing their software by applying a tag to their vcs repositories. The format of this tag is defined as

v<semver>

That is, the character v, U+0075, followed directly by a string which is compliant with the SemVer 2.0 standard. Tags which do not fit this format should be ignored for the purpose of determining which versions of a Go project are released.

Out of scope

The following items are out of scope, but would be addressed in a later proposals:

How Go projects can declare the version numbers or ranges for projects they depend on.
How go get may be changed to consume this version information.

Additionally, this proposal not seek to change the release process, or version numbering scheme for the Go (https://golang.org) distribution itself.

Thank you for your time.

It would be easier to judge this proposal if it included examples from a specific extant tool that would work better with a standard versioning scheme. As it stands, the proposal's preface justifies versions, but not consistent versions.

@crawshaw

To be explicit, this proposal ties requires versions be assigned at the repository level, it cannot be assigned at the package level. Agree with @crawshaw observation.

@crawshaw

To be explicit, this proposal ties requires versions be assigned at the repository level, it cannot be assigned at the package level. Agree with @crawshaw observation.

@kardianos this is correct. Go packages belong to projects which live inside vcs repositories. This is a proposal to release these projects by tagging those repositories with semver tags.

@crawshaw

It would be easier to judge this proposal if it included examples from a specific extant tool that would work better with a standard versioning scheme.

@crawshaw thank you for your comments. To give two concrete examples:

\1. For my project, gb I provide a helper plugin, gb-vendor, which assists users in maintaining the contents of the code they have vendored. gb-vendor also provides an update command, but at the moment the best we can do is update from the currently vendored revision to the latest on the branch, if users want something else they have to manually update to a different revision.

This is because, given an arbitrary branch and revision, it is not possible to say "give me the latest stable version". Sure, if you knew the latest "blessed" revision, you could do that, which is effectively what gb-vendor users have to do now, but then again, if they already knew the revision they wanted to switch to, they wouldn't ask questions like "give me the latest stable version".

\2. For my day job, both Debian and Ubuntu have a strong preference to only package released versions of software. Ubuntu want to make their distribution an excellent platform for Go developers by packaging up all the commonly used dependencies, as we do for other popular languages like Ruby and Python. At the moment Debian maintainers are forced to do things like this, which are of no use to anyone because it's a made up un released version number assigned by the packager, not the owner of the source code.

Now, that is not to say that all projects do not have usable versions, docker, kubernetes, and coreos, the three biggest projects in Go, use the form I proposed above and that benefits all the downstream consumers of those projects.

As it stands, the proposal's preface justifies versions, but not consistent versions.

If this proposal removed the use of semver, and made the version string opaque, that is to say v<x.y.z> where x, y and z were not specified, would you be more favourable to this propsal ?

As a Debian Developer, Go user, and Docker project maintainer, I'm definitely very +1 to explicit versioning. I'm not personally attached to any particular scheme (semver is nice), but having explicit points in time where a package maintainer has said "I feel good enough about my library/program at this particular point in time to attach some extra meaning to it" is really useful, even if they're simple monotonic increases like what systemd uses.

As a simplistic example, it's easy for users to say "I'm on version 34 and hitting bug XYZ" and the maintainer can respond with something simple for users to understand like "that was fixed in version 36".

Semver specifically is nice when used properly, but it does apply a light layer of "social contract" between package authors and consumers such that (for example) version 1.0.0 and version 1.0.1 are "compatible" (for varying values of compatible, I suppose).

The fake VCS-based version numbers used in Debian can be useful (0.0~git20150526.1.5478c06-1 for example), but IMO should only be necessary for snapshots between releases (such as 1.2.3~git20150526.1.5478c06-1).

So what does the implementation of this look like? Where would the convention be documented? Would it be enforced by the Go tool chain somewhere? Is this analogous to the vendor-spec proposal?

So what does the implementation of this look like?

I imagine the implementation would be a design document in the proposal repo, like the security policy.

Where would the convention be documented?

In the document above.

Would it be enforced by the Go tool chain somewhere?

Not at the moment. Releasing code is outside the purview of the go tool at the moment and go get has no capabilities to choose which tag to check out at the moment. However, there are many other tools, both in the Go sphere, and outside, that consume Go projects and will benefit from being able to identify released versions of Go projects.

Is this analogous to the vendor-spec proposal?

Not directly. The vendor spec is about recording what is vendored into the local copy. This proposal is about providing a way for Go project owners to indicate which revisions of their repository they want to be considered "released", in the same way that we develop Go on master, but tag a specific revision when it's ready to be released.

I imagine the implementation would be a design document in the proposal repo, like the security policy.

The security policy will be documented somewhere outside the proposal repo. Either the wiki or on the web site (probably the former). I would expect the same outcome both for this proposal and vendor-spec.

I understand how this is distinct from vendor-spec. My question about vendor-spec was asking if this were basically "version-spec". I think that vendor-spec might be influenced by this proposal, though.

The security policy will be documented somewhere outside the proposal repo. Either the wiki or on the web site (probably the former). I would expect the same outcome both for this proposal and vendor-spec.

I am sorry, this was my misunderstanding, I'm still getting used to the proposal process.

I agree that the website, maybe in one of the 'how to write Go code' pages would be a good place for this information, or possibly on the wiki. For me, the location is not as important as establishing a single recommended way to tag a Go project in their repos.

I understand how this is distinct from vendor-spec. My question about vendor-spec was asking if this were basically "version-spec".

I'm sorry for misunderstanding. I guess this is a version-spec if you want to think of it that way.

I should add that in this initial proposal I have err'd on the side of brevity to focus on the outcomes, not the procedure. I fully expect that should this proposal move forward, the next step would be a design document as you outlined in the proposal process.

I think that vendor-spec might be influenced by this proposal, though.

Possibly a bit, but I don't think either proposal should be dependant on one another. The former, this one, is about a format for a release tag on repositories, while the vendor-spec caters for tags, branches, and revisions equally, and it is hard to see this proposal designing a tag format that is impossible to express in the vendor-spec.

Thanks

Dave

I've stated this elsewhere, but if a version was specified to be consistent I would prefer a spec that allowed sub-paths to be separately versioned. "websocket-v2.2.3" or "context-v1.0.0" in addition to the repo wide option specified above.

I don't have any problem with the specifics of this proposal, I'm just a little confused by the scope. The go tool has opinions about Go packages and the layout of the repositories they exist in, but has no concept of a project. Introducing project-level versioning requires introducing the notion of a project.

The kinds of questions I might ask about projects:

Can a project span repositories?
Can a single repository contain multiple projects?
How many main packages can be in a project?
What names a project?

@crawshaw

@crawshaw Dave has stated elsewhere that the framing is one version per VCS repository. So the repository becomes the "project". This is a stronger notion than even what the "vendor" folder implies as it defines a single root.

That's fine, it's just that project needs to be defined before project versioning can be defined. That probably needs to happen as part of the proposal.

@Merovius

@Merovius thank you for your comments. I can see you have strong views on versioning, releasing, and methodologies like semver itself. Like others who have replied, we'll have to agree to disagree, and as SemVer is key to this proposal, I'm sorry but I cannot accept your suggestions into this proposal.

To repeat myself again: My suggestion was primarily to at least define what constitutes a "breaking change", or rather a "non-breaking change". I don't understand why this should be impossible or conflict with the target of using Semantic versioning, in fact I would think that it would promote the introduction of SemVer. I have strong opinions on the merits of SemVer, yes, but I agreed to disagree on those from the very beginning. But so far no one has even attempted to give this definition, acknowledge it's necessity or to give a reason to not give such a definition. Clarity on what constitutes a breakage seems like a hard requirement for introducing SemVer to me, because after all the spec references this term and only makes sense after this term is clear.

I am going to leave this discussion now, because I feel unwanted and ignored.

@Merovius

@Merovius I'm sorry you feel that your views are not being respected. If I have given that impression, please accept my apologies.

With respect to your specific concerns about what defines "breaking or non breaking change", I hold it is whatever Semver says, which is clearly stated at the top of the summary section of http://semver.org/spec/v2.0.0.html.

You argue that what constitutes a breaking change is much larger than what is commonly accepted, as pointed out by your well written article. As I said above, I agree completely with what you wrote, and see this as a point which tools can help as the rules for "breaking vs non breaking" are not ambiguous -- we already have this with the Go api tool, although i think you are arguing for some even more stringent, and that is also fine.

In summary, I agree with your points on breaking changes being almost any addition to an existing Go type (sorry if I have not reproduced the depth of your argument), but that seems to me to be tangental to this proposal. This proposal is to adopt some kind of release process -- if there is no release process then arguing about the nuances of the application of Semver to Go APIs serves no purpose.

Thanks

Dave

@Merovius

@Merovius, a few points.

While it may be true that "every API change potentially breaks the build", that's not a terribly useful stance. For example, adding a method to an existing type is one kind of change; deleting that type is qualitatively different. It is helpful to distinguish these two, even if both are potential breaking changes.
Go itself is using SemVer numbering (although we'd never heard of it when we started) to good effect. So if we need a formal spec about what constitutes a breaking change and what does not, then Go's own rules (https://golang.org/doc/go1compat) might be a decent start.
Although the proposal does mention SemVer explicitly, it seems to me best not to get too hung up by the specific version numbering scheme. The reason this proposal is stalled is lack of a clear idea, assuming one can create a semantically useful versioning scheme, what you do next.

For anyone who wants to back SemVer into their toolchains I wrote up a quick tutorial (with a package) for the Gopher Academy 2015 Advent

@davecheney

This proposal would ratify what many people and tools are already doing. Docker, Kubernetes, and CoreOS were mentioned. When I wrote about tagging repositories over two years ago, there were already third-party tools that worked with the v<SemVer> format.

The fsnotify library currently uses gopkg.in to turn tagged releases into new import paths for each major version. One feature gopkg.in provides is the distinction between released code and what's sitting on master. The url http://gopkg.in/fsnotify.v1 retrieves the latest v1.x.y tag. Code sitting on master while preparing a release is not made available until tagged. It also has experimental support for prerelease versions. Adding a v2.0.0-unstable tag would provide an opt-in way to retrieve an upcoming release.

While gopkg.in has been useful, I think it is quickly becoming unnecessary. Many people vendor code now, and GO15VENDOREXPERIMENT is further formalizing that recommendation. Everything gopkg.in provides can be replaced with tools such as Godep and gb _if_ they can retrieve the latest tag rather than master, or pull in prerelease versions or previous versions.

Gopkg.in is built on the idea of taking a common VCS practice of tagging releases and providing multiple import paths, derived from the recommendation in the Go FAQ. @davecheney has already listed the issues his colleagues at Canonical had with the multiple import path approach.

There are situations where creating an entirely new library with a new name is appropriate. But for bumping a major version to remove deprecated APIs, there are better alternatives to "create a new package with a new import path", and I believe the FAQ deserves an update.

Providing more visibility into "a release process for Go repositories" will allow third-party tools to start expecting v<SemVer> tags, using them to benefit their users.

Updating GoDoc.org to show documentation for any version may or may not be worth the effort involved. A more reasonable start would be to simply display the version number for the most current release. That will help make this recommendation to tag releases more visible. The distinction between tagged and untagged repositories could also be used to help ranking, by boosting released libraries higher than hobby projects.

What comes after that? I don't know the future, but I believe this is a good step, and one worth taking. Thanks @davecheney.

@nathany

@nathany If I understand what @rsc is say the issue is that adding SemVer as a recommendation without having part of the go toolchain using it is the issue. I'm not sure we'd find agreement on the right way to use SemVer in the toolchain right now. I think we really want SemVer so we can start to use it in tools and figure out directions it can be useful. So we can leverage versions like other communities.

If community things that aren't going to be built directly into the tools aren't appropriate for this forum do we need something similar to the PHP FIG for the community?

If community things that aren't going to be built directly into the tools aren't appropriate for this forum do we need something similar to the PHP FIG for the community?

That sounds well meaning, but entirely too much bureaucracy. How about people just tag their releases ?

Yes, my objection is that this proposal does not propose any actual changes to any part of Go. It is not actionable in its current form. I'm still not even sure it's a good idea, in large part because one way I evaluate whether something is a good idea is to understand its effects, and this proposal has no direct effects.

Build a system that uses semantic versions effectively. Demonstrate what great tools can be built. Give people incentives, and they will tag their releases no matter what the Go team says. Certainly the Go team saying "you should do it this way" has not in the past been successful without these things (think not-vendoring).

I feel that it is time to draw the discussion to a close. The proposal guidelines clearly state "a lack of agreement means [the proposal is] declined", as such it is appropriate that I withdraw this proposal.

I want to extend my sincere thanks to all who have contributed to this debate. I remain passionate about improving the tools Go programmers have to manage package dependencies in large Go projects and your feedback has been invaluable to me and my future efforts.

Thank you again

Dave

Fair enough Dave. Thanks for the time you spent on the proposal. I certainly agree that a solution to versioning would be great, and that SemVer seems like a very good basis for building such a solution. The devil is always in the details, of course.

For the Versioning Numbering Cheme I would, any day, choose Explicit Versioning insted of Semantic Versioning. Why? It's so simple the first four words from the title of the link explain it for most of the developers. And besides that, no more ambiguity caused by the "Major" and "Minor" fields of the version number. Here's the promissed link: Release.Breaking.Feature.Fix or why the Semantic Versioning should be replaced with Explicit Versioning as soon as possible

Navigation Menu

Repository navigation

proposal: a version numbering scheme and release process for Go projects #12302

Preface

Proposal

Version numbering scheme

Tag format

Out of scope

143 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

proposal: a version numbering scheme and release process for Go projects #12302

Description

Preface

Proposal

Version numbering scheme

Tag format

Out of scope

Activity

crawshaw commented on Aug 24, 2015

kardianos commented on Aug 25, 2015

davecheney commented on Aug 25, 2015

davecheney commented on Aug 25, 2015

tianon commented on Aug 25, 2015

adg commented on Aug 25, 2015

davecheney commented on Aug 25, 2015

adg commented on Aug 25, 2015

davecheney commented on Aug 25, 2015

kardianos commented on Aug 25, 2015

crawshaw commented on Aug 25, 2015

kardianos commented on Aug 25, 2015

crawshaw commented on Aug 25, 2015

143 remaining items

davecheney commented on Nov 28, 2015

Merovius commented on Nov 29, 2015

davecheney commented on Nov 29, 2015

rsc commented on Nov 30, 2015

mattfarina commented on Dec 2, 2015

nathany commented on Dec 2, 2015

mattfarina commented on Dec 2, 2015

davecheney commented on Dec 2, 2015

rsc commented on Dec 2, 2015

davecheney commented on Dec 3, 2015

rsc commented on Dec 3, 2015

sapioit commented on Jul 5, 2016

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions