proposal: a version numbering scheme and release process for Go projects #12302

Closed
davecheney opened this Issue Aug 24, 2015 · 161 comments

Comments

Projects
None yet
@davecheney
Contributor

davecheney commented Aug 24, 2015

Preface

Go projects do not have version numbers in the way it is commonly understood by our counterparts in other languages communities. This is because there is no formalised notion of releasing a Go project. There is no process of taking an arbitrary vcs commit hash and assigning it a version number that is both meaningful for humans and machines.

Additionally, Operating System distributors such as Debian and Ubuntu strongly prefer to package released versions of a project, and are currently reduced to doing things like this.

To put it another way,

Go projects are commonly tracked in vcs repositories and derive their import prefix from their vcs location. Version control systems assign revision identifiers or hashes to various copies of the source they track over time. These vcs hashes are ideal for identifying a particular copy, or revision, of a Go project. However vcs hashes are less useful at answering other types of question like:

  • "Am I using the latest copy of this project ?"
  • "Are there any bug fixes for version 1.5 available ?"
  • "Which version of the postgres driver are compatible with this library ?"

The aim of this proposal is to establish a single recommended procedure for releasing Go projects, in the same way that gofmt defines a single recommended way to format Go source code.

Proposal

This proposal seeks to establish a recommended version numbering scheme and minimal release process for Go projects.

A Go project is a collection of one or more Go packages whose source is tracked in a vcs repository.

A repository root, as defined by cmd/go, identifies both the unique import path prefix for the Go project, and the vcs repository that holds the project's source.

This proposal describes a release process for Go projects by tagging the repository which hold the project's code.

This process is intended to be light weight and will facilitate the creation of tools that automate the creation and consumption of released versions of Go projects.

Version numbering scheme

This proposal recommends that Go projects adopt the semantic versioning 2.0 standard for their version numbering scheme.

This recommendation is informed by the broad support for semantic versioning across our contemporaries like node.js (npm), rust (cargo), javascript (bower), and ruby (rubygems). Adherence to a commonly accepted ideal of what constitutes a major, minor, or patch release will allow Go programmers to benefit from the experiences of these other communities' dependency management ecosystems.

Tag format

Furthermore, this proposal recommends that Go projects adopt a process of releasing their software by applying a tag to their vcs repositories. The format of this tag is defined as

v<semver>

That is, the character v, U+0075, followed directly by a string which is compliant with the SemVer 2.0 standard. Tags which do not fit this format should be ignored for the purpose of determining which versions of a Go project are released.

Out of scope

The following items are out of scope, but would be addressed in a later proposals:

  • How Go projects can declare the version numbers or ranges for projects they depend on.
  • How go get may be changed to consume this version information.

Additionally, this proposal not seek to change the release process, or version numbering scheme for the Go (https://golang.org) distribution itself.

Thank you for your time.

@mikioh mikioh added the Proposal label Aug 24, 2015

@mikioh mikioh changed the title from Proposal: a version numbering scheme and release process for Go projects to proposal: a version numbering scheme and release process for Go projects Aug 24, 2015

@crawshaw

This comment has been minimized.

Show comment
Hide comment
@crawshaw

crawshaw Aug 24, 2015

Contributor

It would be easier to judge this proposal if it included examples from a specific extant tool that would work better with a standard versioning scheme. As it stands, the proposal's preface justifies versions, but not consistent versions.

Contributor

crawshaw commented Aug 24, 2015

It would be easier to judge this proposal if it included examples from a specific extant tool that would work better with a standard versioning scheme. As it stands, the proposal's preface justifies versions, but not consistent versions.

@kardianos

This comment has been minimized.

Show comment
Hide comment
@kardianos

kardianos Aug 25, 2015

Contributor

To be explicit, this proposal ties requires versions be assigned at the repository level, it cannot be assigned at the package level. Agree with @crawshaw observation.

Contributor

kardianos commented Aug 25, 2015

To be explicit, this proposal ties requires versions be assigned at the repository level, it cannot be assigned at the package level. Agree with @crawshaw observation.

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Aug 25, 2015

Contributor

To be explicit, this proposal ties requires versions be assigned at the repository level, it cannot be assigned at the package level. Agree with @crawshaw observation.

@kardianos this is correct. Go packages belong to projects which live inside vcs repositories. This is a proposal to release these projects by tagging those repositories with semver tags.

Contributor

davecheney commented Aug 25, 2015

To be explicit, this proposal ties requires versions be assigned at the repository level, it cannot be assigned at the package level. Agree with @crawshaw observation.

@kardianos this is correct. Go packages belong to projects which live inside vcs repositories. This is a proposal to release these projects by tagging those repositories with semver tags.

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Aug 25, 2015

Contributor

It would be easier to judge this proposal if it included examples from a specific extant tool that would work better with a standard versioning scheme.

@crawshaw thank you for your comments. To give two concrete examples:

\1. For my project, gb I provide a helper plugin, gb-vendor, which assists users in maintaining the contents of the code they have vendored. gb-vendor also provides an update command, but at the moment the best we can do is update from the currently vendored revision to the latest on the branch, if users want something else they have to manually update to a different revision.

This is because, given an arbitrary branch and revision, it is not possible to say "give me the latest stable version". Sure, if you knew the latest "blessed" revision, you could do that, which is effectively what gb-vendor users have to do now, but then again, if they already knew the revision they wanted to switch to, they wouldn't ask questions like "give me the latest stable version".

\2. For my day job, both Debian and Ubuntu have a strong preference to only package released versions of software. Ubuntu want to make their distribution an excellent platform for Go developers by packaging up all the commonly used dependencies, as we do for other popular languages like Ruby and Python. At the moment Debian maintainers are forced to do things like this, which are of no use to anyone because it's a made up un released version number assigned by the packager, not the owner of the source code.

Now, that is not to say that all projects do not have usable versions, docker, kubernetes, and coreos, the three biggest projects in Go, use the form I proposed above and that benefits all the downstream consumers of those projects.

As it stands, the proposal's preface justifies versions, but not consistent versions.

If this proposal removed the use of semver, and made the version string opaque, that is to say v<x.y.z> where x, y and z were not specified, would you be more favourable to this propsal ?

Contributor

davecheney commented Aug 25, 2015

It would be easier to judge this proposal if it included examples from a specific extant tool that would work better with a standard versioning scheme.

@crawshaw thank you for your comments. To give two concrete examples:

\1. For my project, gb I provide a helper plugin, gb-vendor, which assists users in maintaining the contents of the code they have vendored. gb-vendor also provides an update command, but at the moment the best we can do is update from the currently vendored revision to the latest on the branch, if users want something else they have to manually update to a different revision.

This is because, given an arbitrary branch and revision, it is not possible to say "give me the latest stable version". Sure, if you knew the latest "blessed" revision, you could do that, which is effectively what gb-vendor users have to do now, but then again, if they already knew the revision they wanted to switch to, they wouldn't ask questions like "give me the latest stable version".

\2. For my day job, both Debian and Ubuntu have a strong preference to only package released versions of software. Ubuntu want to make their distribution an excellent platform for Go developers by packaging up all the commonly used dependencies, as we do for other popular languages like Ruby and Python. At the moment Debian maintainers are forced to do things like this, which are of no use to anyone because it's a made up un released version number assigned by the packager, not the owner of the source code.

Now, that is not to say that all projects do not have usable versions, docker, kubernetes, and coreos, the three biggest projects in Go, use the form I proposed above and that benefits all the downstream consumers of those projects.

As it stands, the proposal's preface justifies versions, but not consistent versions.

If this proposal removed the use of semver, and made the version string opaque, that is to say v<x.y.z> where x, y and z were not specified, would you be more favourable to this propsal ?

@tianon

This comment has been minimized.

Show comment
Hide comment
@tianon

tianon Aug 25, 2015

As a Debian Developer, Go user, and Docker project maintainer, I'm definitely very +1 to explicit versioning. I'm not personally attached to any particular scheme (semver is nice), but having explicit points in time where a package maintainer has said "I feel good enough about my library/program at this particular point in time to attach some extra meaning to it" is really useful, even if they're simple monotonic increases like what systemd uses.

As a simplistic example, it's easy for users to say "I'm on version 34 and hitting bug XYZ" and the maintainer can respond with something simple for users to understand like "that was fixed in version 36".

Semver specifically is nice when used properly, but it does apply a light layer of "social contract" between package authors and consumers such that (for example) version 1.0.0 and version 1.0.1 are "compatible" (for varying values of compatible, I suppose).

The fake VCS-based version numbers used in Debian can be useful (0.0~git20150526.1.5478c06-1 for example), but IMO should only be necessary for snapshots between releases (such as 1.2.3~git20150526.1.5478c06-1).

tianon commented Aug 25, 2015

As a Debian Developer, Go user, and Docker project maintainer, I'm definitely very +1 to explicit versioning. I'm not personally attached to any particular scheme (semver is nice), but having explicit points in time where a package maintainer has said "I feel good enough about my library/program at this particular point in time to attach some extra meaning to it" is really useful, even if they're simple monotonic increases like what systemd uses.

As a simplistic example, it's easy for users to say "I'm on version 34 and hitting bug XYZ" and the maintainer can respond with something simple for users to understand like "that was fixed in version 36".

Semver specifically is nice when used properly, but it does apply a light layer of "social contract" between package authors and consumers such that (for example) version 1.0.0 and version 1.0.1 are "compatible" (for varying values of compatible, I suppose).

The fake VCS-based version numbers used in Debian can be useful (0.0~git20150526.1.5478c06-1 for example), but IMO should only be necessary for snapshots between releases (such as 1.2.3~git20150526.1.5478c06-1).

@adg

This comment has been minimized.

Show comment
Hide comment
@adg

adg Aug 25, 2015

Contributor

So what does the implementation of this look like? Where would the convention be documented? Would it be enforced by the Go tool chain somewhere? Is this analogous to the vendor-spec proposal?

Contributor

adg commented Aug 25, 2015

So what does the implementation of this look like? Where would the convention be documented? Would it be enforced by the Go tool chain somewhere? Is this analogous to the vendor-spec proposal?

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Aug 25, 2015

Contributor

So what does the implementation of this look like?

I imagine the implementation would be a design document in the proposal repo, like the security policy.

Where would the convention be documented?

In the document above.

Would it be enforced by the Go tool chain somewhere?

Not at the moment. Releasing code is outside the purview of the go tool at the moment and go get has no capabilities to choose which tag to check out at the moment. However, there are many other tools, both in the Go sphere, and outside, that consume Go projects and will benefit from being able to identify released versions of Go projects.

Is this analogous to the vendor-spec proposal?

Not directly. The vendor spec is about recording what is vendored into the local copy. This proposal is about providing a way for Go project owners to indicate which revisions of their repository they want to be considered "released", in the same way that we develop Go on master, but tag a specific revision when it's ready to be released.

Contributor

davecheney commented Aug 25, 2015

So what does the implementation of this look like?

I imagine the implementation would be a design document in the proposal repo, like the security policy.

Where would the convention be documented?

In the document above.

Would it be enforced by the Go tool chain somewhere?

Not at the moment. Releasing code is outside the purview of the go tool at the moment and go get has no capabilities to choose which tag to check out at the moment. However, there are many other tools, both in the Go sphere, and outside, that consume Go projects and will benefit from being able to identify released versions of Go projects.

Is this analogous to the vendor-spec proposal?

Not directly. The vendor spec is about recording what is vendored into the local copy. This proposal is about providing a way for Go project owners to indicate which revisions of their repository they want to be considered "released", in the same way that we develop Go on master, but tag a specific revision when it's ready to be released.

@adg

This comment has been minimized.

Show comment
Hide comment
@adg

adg Aug 25, 2015

Contributor

I imagine the implementation would be a design document in the proposal repo, like the security policy.

The security policy will be documented somewhere outside the proposal repo. Either the wiki or on the web site (probably the former). I would expect the same outcome both for this proposal and vendor-spec.

I understand how this is distinct from vendor-spec. My question about vendor-spec was asking if this were basically "version-spec". I think that vendor-spec might be influenced by this proposal, though.

Contributor

adg commented Aug 25, 2015

I imagine the implementation would be a design document in the proposal repo, like the security policy.

The security policy will be documented somewhere outside the proposal repo. Either the wiki or on the web site (probably the former). I would expect the same outcome both for this proposal and vendor-spec.

I understand how this is distinct from vendor-spec. My question about vendor-spec was asking if this were basically "version-spec". I think that vendor-spec might be influenced by this proposal, though.

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Aug 25, 2015

Contributor

The security policy will be documented somewhere outside the proposal repo. Either the wiki or on the web site (probably the former). I would expect the same outcome both for this proposal and vendor-spec.

I am sorry, this was my misunderstanding, I'm still getting used to the proposal process.

I agree that the website, maybe in one of the 'how to write Go code' pages would be a good place for this information, or possibly on the wiki. For me, the location is not as important as establishing a single recommended way to tag a Go project in their repos.

I understand how this is distinct from vendor-spec. My question about vendor-spec was asking if this were basically "version-spec".

I'm sorry for misunderstanding. I guess this is a version-spec if you want to think of it that way.

I should add that in this initial proposal I have err'd on the side of brevity to focus on the outcomes, not the procedure. I fully expect that should this proposal move forward, the next step would be a design document as you outlined in the proposal process.

I think that vendor-spec might be influenced by this proposal, though.

Possibly a bit, but I don't think either proposal should be dependant on one another. The former, this one, is about a format for a release tag on repositories, while the vendor-spec caters for tags, branches, and revisions equally, and it is hard to see this proposal designing a tag format that is impossible to express in the vendor-spec.

Thanks

Dave

Contributor

davecheney commented Aug 25, 2015

The security policy will be documented somewhere outside the proposal repo. Either the wiki or on the web site (probably the former). I would expect the same outcome both for this proposal and vendor-spec.

I am sorry, this was my misunderstanding, I'm still getting used to the proposal process.

I agree that the website, maybe in one of the 'how to write Go code' pages would be a good place for this information, or possibly on the wiki. For me, the location is not as important as establishing a single recommended way to tag a Go project in their repos.

I understand how this is distinct from vendor-spec. My question about vendor-spec was asking if this were basically "version-spec".

I'm sorry for misunderstanding. I guess this is a version-spec if you want to think of it that way.

I should add that in this initial proposal I have err'd on the side of brevity to focus on the outcomes, not the procedure. I fully expect that should this proposal move forward, the next step would be a design document as you outlined in the proposal process.

I think that vendor-spec might be influenced by this proposal, though.

Possibly a bit, but I don't think either proposal should be dependant on one another. The former, this one, is about a format for a release tag on repositories, while the vendor-spec caters for tags, branches, and revisions equally, and it is hard to see this proposal designing a tag format that is impossible to express in the vendor-spec.

Thanks

Dave

@kardianos

This comment has been minimized.

Show comment
Hide comment
@kardianos

kardianos Aug 25, 2015

Contributor

I've stated this elsewhere, but if a version was specified to be consistent I would prefer a spec that allowed sub-paths to be separately versioned. "websocket-v2.2.3" or "context-v1.0.0" in addition to the repo wide option specified above.

Contributor

kardianos commented Aug 25, 2015

I've stated this elsewhere, but if a version was specified to be consistent I would prefer a spec that allowed sub-paths to be separately versioned. "websocket-v2.2.3" or "context-v1.0.0" in addition to the repo wide option specified above.

@crawshaw

This comment has been minimized.

Show comment
Hide comment
@crawshaw

crawshaw Aug 25, 2015

Contributor

I don't have any problem with the specifics of this proposal, I'm just a little confused by the scope. The go tool has opinions about Go packages and the layout of the repositories they exist in, but has no concept of a project. Introducing project-level versioning requires introducing the notion of a project.

The kinds of questions I might ask about projects:

  • Can a project span repositories?
  • Can a single repository contain multiple projects?
  • How many main packages can be in a project?
  • What names a project?
Contributor

crawshaw commented Aug 25, 2015

I don't have any problem with the specifics of this proposal, I'm just a little confused by the scope. The go tool has opinions about Go packages and the layout of the repositories they exist in, but has no concept of a project. Introducing project-level versioning requires introducing the notion of a project.

The kinds of questions I might ask about projects:

  • Can a project span repositories?
  • Can a single repository contain multiple projects?
  • How many main packages can be in a project?
  • What names a project?
@kardianos

This comment has been minimized.

Show comment
Hide comment
@kardianos

kardianos Aug 25, 2015

Contributor

@crawshaw Dave has stated elsewhere that the framing is one version per VCS repository. So the repository becomes the "project". This is a stronger notion than even what the "vendor" folder implies as it defines a single root.

Contributor

kardianos commented Aug 25, 2015

@crawshaw Dave has stated elsewhere that the framing is one version per VCS repository. So the repository becomes the "project". This is a stronger notion than even what the "vendor" folder implies as it defines a single root.

@crawshaw

This comment has been minimized.

Show comment
Hide comment
@crawshaw

crawshaw Aug 25, 2015

Contributor

That's fine, it's just that project needs to be defined before project versioning can be defined. That probably needs to happen as part of the proposal.

Contributor

crawshaw commented Aug 25, 2015

That's fine, it's just that project needs to be defined before project versioning can be defined. That probably needs to happen as part of the proposal.

@gravis

This comment has been minimized.

Show comment
Hide comment
@gravis

gravis Aug 25, 2015

This looks like the early days of ruby on rails. Before having bundler, we had to put dependencies in /vendor/plugins. It lasted for some years, then rubygems became the standard. The dependencies were defined in the code, and a rake task was used to install the deps: rake gems:install (<= search for issues with that on google...)
It introduced a lot of issues, because the gem (ruby packages) were installed using the application itself. Of course, sometimes, the app couldn't load enough code to install the gems, and the user was stuck.
Then, bundler was designed, an independent tool, reading one config file (Gemfile). The locked versions are saved to Gemfile.lock, and installed most of the time in a dedicated "gemset" (a GOPATH).
I writing all of this, because it seems go is heading the same way, and will spend months/years to adopt what the other languages figured out in the last years ;)
What go could do better, is the namespace of packages, because the source of the package is obviously explicit. There's no need for a central registry, VCS tags are more than enough.

gravis commented Aug 25, 2015

This looks like the early days of ruby on rails. Before having bundler, we had to put dependencies in /vendor/plugins. It lasted for some years, then rubygems became the standard. The dependencies were defined in the code, and a rake task was used to install the deps: rake gems:install (<= search for issues with that on google...)
It introduced a lot of issues, because the gem (ruby packages) were installed using the application itself. Of course, sometimes, the app couldn't load enough code to install the gems, and the user was stuck.
Then, bundler was designed, an independent tool, reading one config file (Gemfile). The locked versions are saved to Gemfile.lock, and installed most of the time in a dedicated "gemset" (a GOPATH).
I writing all of this, because it seems go is heading the same way, and will spend months/years to adopt what the other languages figured out in the last years ;)
What go could do better, is the namespace of packages, because the source of the package is obviously explicit. There's no need for a central registry, VCS tags are more than enough.

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Aug 25, 2015

Contributor

@crawshaw

I don't have any problem with the specifics of this proposal, I'm just a little confused by the scope. The go tool has opinions about Go packages and the layout of the repositories they exist in, but has no concept of a project. Introducing project-level versioning requires introducing the notion of a project.

This is correct, the go tool does not have a concept of how the packages on your GOPATH came to be there, but go get does. Before fetching, go get must canonicalise the import path, find some prefix of the path you provided you gave it to the location of a remote repository, which it fetches in its entirety. It is this import prefix, or what the Go tool calls, the repository root, that is a project.

With that said, any changes to the go tool are outside the scope of this proposal.

I've updated the original proposal with a definition of a Go project.

The kinds of questions I might ask about projects:

  • Can a project span repositories?

No. Each repository is a versioned and released individually.

  • Can a single repository contain multiple projects?

No. The contents of a repository are released together as a whole, golang.org/x/tools is a great example of that.

  • How many main packages can be in a project?

Unlimited. There is an unfortunate coincidence with the gb definition of a project. If you think it would help I can reword this proposal to remove all references to Project, and replace it with Remote Repository.

  • What names a project?

Go projects are named by the import prefix that is defined by their repository root. Import prefix and repository root are defined by the documentation on the go tool.

Contributor

davecheney commented Aug 25, 2015

@crawshaw

I don't have any problem with the specifics of this proposal, I'm just a little confused by the scope. The go tool has opinions about Go packages and the layout of the repositories they exist in, but has no concept of a project. Introducing project-level versioning requires introducing the notion of a project.

This is correct, the go tool does not have a concept of how the packages on your GOPATH came to be there, but go get does. Before fetching, go get must canonicalise the import path, find some prefix of the path you provided you gave it to the location of a remote repository, which it fetches in its entirety. It is this import prefix, or what the Go tool calls, the repository root, that is a project.

With that said, any changes to the go tool are outside the scope of this proposal.

I've updated the original proposal with a definition of a Go project.

The kinds of questions I might ask about projects:

  • Can a project span repositories?

No. Each repository is a versioned and released individually.

  • Can a single repository contain multiple projects?

No. The contents of a repository are released together as a whole, golang.org/x/tools is a great example of that.

  • How many main packages can be in a project?

Unlimited. There is an unfortunate coincidence with the gb definition of a project. If you think it would help I can reword this proposal to remove all references to Project, and replace it with Remote Repository.

  • What names a project?

Go projects are named by the import prefix that is defined by their repository root. Import prefix and repository root are defined by the documentation on the go tool.

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Aug 25, 2015

Contributor

@gravis thanks for your comments and your perspective.

I agree that Go is lucky that we have always had a package namespace (although noting that Go packages are not hierarchical) so we can lean on this heavily.

For example, what defines a project, and who gets to name a project ? Luckily go get already defines the rules for what is a remote package and how it is fetched, so questions of ownership can be punted to a 3rd party.

Who owns the github.com/docker/docker project, and thus the github.com/docker/docker/... namespace ? It's who controls the docker repository on the docker account at github.

Who owns the golang.org/x/tools project ? It's whoever controls the dns name for golang.org and can host a web server to serve a meta redirect.

Contributor

davecheney commented Aug 25, 2015

@gravis thanks for your comments and your perspective.

I agree that Go is lucky that we have always had a package namespace (although noting that Go packages are not hierarchical) so we can lean on this heavily.

For example, what defines a project, and who gets to name a project ? Luckily go get already defines the rules for what is a remote package and how it is fetched, so questions of ownership can be punted to a 3rd party.

Who owns the github.com/docker/docker project, and thus the github.com/docker/docker/... namespace ? It's who controls the docker repository on the docker account at github.

Who owns the golang.org/x/tools project ? It's whoever controls the dns name for golang.org and can host a web server to serve a meta redirect.

@thockin

This comment has been minimized.

Show comment
Hide comment
@thockin

thockin Aug 26, 2015

My 2 cents: There's not much here to disagree with (and yet...). I think versioning is good, but I also happen to work on a project that versions in exactly this way.

With that said, any changes to the go tool are outside the scope of this proposal.

Without some hypothetical usages of this convention, it might be hard to justify. For example, could go get HYPOTHETICALLY take a version string and fetch that specific revision? Could godep or gb vendor HYPOTHETICALLY take a version to import and/or rewrite imports (gopkg.in style) to versioned?

Those are more exciting than an abstract convention which, to be clear, is painfully obvious to me (but clearly not everyone).

TL;DR

+1 but how do we USE it?

thockin commented Aug 26, 2015

My 2 cents: There's not much here to disagree with (and yet...). I think versioning is good, but I also happen to work on a project that versions in exactly this way.

With that said, any changes to the go tool are outside the scope of this proposal.

Without some hypothetical usages of this convention, it might be hard to justify. For example, could go get HYPOTHETICALLY take a version string and fetch that specific revision? Could godep or gb vendor HYPOTHETICALLY take a version to import and/or rewrite imports (gopkg.in style) to versioned?

Those are more exciting than an abstract convention which, to be clear, is painfully obvious to me (but clearly not everyone).

TL;DR

+1 but how do we USE it?

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Aug 26, 2015

Contributor

@thockin thanks for your comments. I understand your frustration that this does all appear to be hypothetical at the moment.

My rational for starting as small as possible is (and I think well founded) that a proposal that introduced semver and required everyone to tag their repos and had to make a change to go get and had to fit into the overfull 1.6 release cycle would be viewed with blank stares.

This is the smallest possible thing that I think can make the dependency management situation for Go programmers better. I think it is a reasonable thing that improves the life of Go developers and the ecosystem downstream from them even if the go tool does not grow to embrace it, as there are several dozen other competing Go dependency management tools in the market.

With that said, here are a few concrete areas that would benefit immediately

  • Debian and other operating systems can use this information today.
  • I plan to rev the manifest format that gb-vendor uses to include the tag information so I can offer my users a real upgrade command, not just one that grabs the latest head of the branch they are on.
  • godoc.org can show users the docs for the version of the package they are using, not just show the latest that was in the upstream repo.
Contributor

davecheney commented Aug 26, 2015

@thockin thanks for your comments. I understand your frustration that this does all appear to be hypothetical at the moment.

My rational for starting as small as possible is (and I think well founded) that a proposal that introduced semver and required everyone to tag their repos and had to make a change to go get and had to fit into the overfull 1.6 release cycle would be viewed with blank stares.

This is the smallest possible thing that I think can make the dependency management situation for Go programmers better. I think it is a reasonable thing that improves the life of Go developers and the ecosystem downstream from them even if the go tool does not grow to embrace it, as there are several dozen other competing Go dependency management tools in the market.

With that said, here are a few concrete areas that would benefit immediately

  • Debian and other operating systems can use this information today.
  • I plan to rev the manifest format that gb-vendor uses to include the tag information so I can offer my users a real upgrade command, not just one that grabs the latest head of the branch they are on.
  • godoc.org can show users the docs for the version of the package they are using, not just show the latest that was in the upstream repo.
@thockin

This comment has been minimized.

Show comment
Hide comment
@thockin

thockin Aug 26, 2015

Oh, sorry if I mis-communicated. I think this is obviously correct. I'm
just a concrete sort of fellow, so having a mental model of what this would
enable helps me see the purpose. I assume I am not alone.

No "frustration" here, other than wanting to help get a resounding "YES" to
something like this.

On Tue, Aug 25, 2015 at 11:36 PM, Dave Cheney notifications@github.com
wrote:

@thockin https://github.com/thockin thanks for your comments. I
understand your frustration that this does all appear to be hypothetical at
the moment.

My rational for starting as small as possible is (and I think well
founded) that a proposal that introduced semver and required everyone
to tag their repos and had to make a change to go get and had to fit
into the overfull 1.6 release cycle would be viewed with blank stares.

This is the smallest possible thing that I think can make the dependency
management situation for Go programmers better. I think it is a reasonable
thing that improves the life of Go developers and the ecosystem downstream
from them even if the go tool does not grow to embrace it, as there are
several dozen other competing Go dependency management tools in the market.

With that said, here are a few concrete areas that would benefit
immediately

  • Debian and other operating systems could use this information today.
  • I plan to rev the manifest format that gb-vendor uses to include the
    tag information so I can offer my users a real upgrade command, not
    just one that grabs the latest head of the branch they are on.
  • godoc.org can show users the docs for the version of the package
    they are using, not just show the latest that was in the upstream repo.


Reply to this email directly or view it on GitHub
#12302 (comment).

thockin commented Aug 26, 2015

Oh, sorry if I mis-communicated. I think this is obviously correct. I'm
just a concrete sort of fellow, so having a mental model of what this would
enable helps me see the purpose. I assume I am not alone.

No "frustration" here, other than wanting to help get a resounding "YES" to
something like this.

On Tue, Aug 25, 2015 at 11:36 PM, Dave Cheney notifications@github.com
wrote:

@thockin https://github.com/thockin thanks for your comments. I
understand your frustration that this does all appear to be hypothetical at
the moment.

My rational for starting as small as possible is (and I think well
founded) that a proposal that introduced semver and required everyone
to tag their repos and had to make a change to go get and had to fit
into the overfull 1.6 release cycle would be viewed with blank stares.

This is the smallest possible thing that I think can make the dependency
management situation for Go programmers better. I think it is a reasonable
thing that improves the life of Go developers and the ecosystem downstream
from them even if the go tool does not grow to embrace it, as there are
several dozen other competing Go dependency management tools in the market.

With that said, here are a few concrete areas that would benefit
immediately

  • Debian and other operating systems could use this information today.
  • I plan to rev the manifest format that gb-vendor uses to include the
    tag information so I can offer my users a real upgrade command, not
    just one that grabs the latest head of the branch they are on.
  • godoc.org can show users the docs for the version of the package
    they are using, not just show the latest that was in the upstream repo.


Reply to this email directly or view it on GitHub
#12302 (comment).

@mattfarina

This comment has been minimized.

Show comment
Hide comment
@mattfarina

mattfarina Aug 26, 2015

Member

&tldr; +1

One question, why the v at the start of the version number? Should that be required or optional?

I looked and npm (for Node.js) and composer (for PHP) the v is optional. In others such as Cargo (Rust) or Bower (JavaScript) I don't see any mention of v. It appears to not be used.

There are Go projects with and without the v prefix today. Can it be optional?

As one of the developers of Glide, another package manager, I can say that handling this well is already a request and need.

Going beyond issues, such a distros, you can see what version of the project/package API you're using. If I've used a package who bumped their API version from 1.2.3 to 2.0.0 I need to know this because the API contract changed. Versioning can help with the API contract and expectations.

Member

mattfarina commented Aug 26, 2015

&tldr; +1

One question, why the v at the start of the version number? Should that be required or optional?

I looked and npm (for Node.js) and composer (for PHP) the v is optional. In others such as Cargo (Rust) or Bower (JavaScript) I don't see any mention of v. It appears to not be used.

There are Go projects with and without the v prefix today. Can it be optional?

As one of the developers of Glide, another package manager, I can say that handling this well is already a request and need.

Going beyond issues, such a distros, you can see what version of the project/package API you're using. If I've used a package who bumped their API version from 1.2.3 to 2.0.0 I need to know this because the API contract changed. Versioning can help with the API contract and expectations.

@technosophos

This comment has been minimized.

Show comment
Hide comment
@technosophos

technosophos Aug 26, 2015

To @mattfarina's point, SemVer 2.0 removed the requirement for the superfluous v. I don't see a compelling reason to require it in this context.

To @mattfarina's point, SemVer 2.0 removed the requirement for the superfluous v. I don't see a compelling reason to require it in this context.

@kostya-sh

This comment has been minimized.

Show comment
Hide comment
@kostya-sh

kostya-sh Aug 26, 2015

Contributor

A few concerns:

  1. I beleive that incompatible versions of the same library should use different import paths. Otherwise they cannot co-exist as dependencies of a single project. Having them tagged with two different tags is not enough or even harmful.

    There a lot of examples of Java libraries where following this simple rule would have made life of library and application developers much easier. E.g. protobuf 2.4 vs 2.5, various versions of asm library, hadoop-1 vs hadoop-2, various versions of jetty, etc.

  2. Very often for applications (not libraries) it makes little sense to use semver. Something like build number + git hash works well enough. Think about version numbers of such applications like Chrome, Firefox, systemd, less, etc.

Contributor

kostya-sh commented Aug 26, 2015

A few concerns:

  1. I beleive that incompatible versions of the same library should use different import paths. Otherwise they cannot co-exist as dependencies of a single project. Having them tagged with two different tags is not enough or even harmful.

    There a lot of examples of Java libraries where following this simple rule would have made life of library and application developers much easier. E.g. protobuf 2.4 vs 2.5, various versions of asm library, hadoop-1 vs hadoop-2, various versions of jetty, etc.

  2. Very often for applications (not libraries) it makes little sense to use semver. Something like build number + git hash works well enough. Think about version numbers of such applications like Chrome, Firefox, systemd, less, etc.

@tianon

This comment has been minimized.

Show comment
Hide comment
@tianon

tianon Aug 26, 2015

@kostya-sh if the version number is in the import path, then every source file using the import must be updated every time the dependency is (which is a lot of churn and merge conflicts)

IMO if two versions of a package are different enough to be used concurrently, they ought to live under separate paths anyhow (or use a scheme like gopkg.in), especially since using both concurrently is probably an edge case for most.

tianon commented Aug 26, 2015

@kostya-sh if the version number is in the import path, then every source file using the import must be updated every time the dependency is (which is a lot of churn and merge conflicts)

IMO if two versions of a package are different enough to be used concurrently, they ought to live under separate paths anyhow (or use a scheme like gopkg.in), especially since using both concurrently is probably an edge case for most.

@thockin

This comment has been minimized.

Show comment
Hide comment
@thockin

thockin Aug 26, 2015

This is why gopkg.in only versions to the major number. If I import
libfoo.v1 and libfoo.v2 at the same time, I have to rename something. If
the tooling doesn't handle it well, users do their own thing and everyone
ends up with bespoke vendoring scripts. Puke.

On Wed, Aug 26, 2015 at 7:43 AM, Tianon Gravi notifications@github.com
wrote:

@kostya-sh https://github.com/kostya-sh if the version number is in the
import path, then every source file using the import must be updated every
time the dependency is (which is a lot of churn and merge conflicts)

IMO if two versions of a package are different enough to be used
concurrently, they ought to live under separate paths anyhow (or use a
scheme like gopkg.in), especially since using both concurrently is
probably an edge case for most.


Reply to this email directly or view it on GitHub
#12302 (comment).

thockin commented Aug 26, 2015

This is why gopkg.in only versions to the major number. If I import
libfoo.v1 and libfoo.v2 at the same time, I have to rename something. If
the tooling doesn't handle it well, users do their own thing and everyone
ends up with bespoke vendoring scripts. Puke.

On Wed, Aug 26, 2015 at 7:43 AM, Tianon Gravi notifications@github.com
wrote:

@kostya-sh https://github.com/kostya-sh if the version number is in the
import path, then every source file using the import must be updated every
time the dependency is (which is a lot of churn and merge conflicts)

IMO if two versions of a package are different enough to be used
concurrently, they ought to live under separate paths anyhow (or use a
scheme like gopkg.in), especially since using both concurrently is
probably an edge case for most.


Reply to this email directly or view it on GitHub
#12302 (comment).

@kostya-sh

This comment has been minimized.

Show comment
Hide comment
@kostya-sh

kostya-sh Aug 27, 2015

Contributor

@tianon, I didn't propose to include version number in the import path. What I said was that incompatible versions of the same package should be different packages (with different import paths). Though in practice this probably matters only for popular well established libraries used by many projects. E.g. Java protobuf 2.4 vs 2.5. Note that the major version is the same but they are still incompatible in a subtle way.

If the goal of this proposal is to create a document describing best practices for versioning then it should cover topics that I raised including backward compatibility. A document describing best practices for versioning, releasing, using vcs, maintain backward compatibility, etc would be very useful.

If the goal of this proposal is to make semver versions standard in the Go community then I think:

  • tools support should come first. gofmt mentioned in the proposal is the excellent example why
  • it is fairly easy to enforce the version format but it is much more difficult or even impossible to make people follow the rules behind it
Contributor

kostya-sh commented Aug 27, 2015

@tianon, I didn't propose to include version number in the import path. What I said was that incompatible versions of the same package should be different packages (with different import paths). Though in practice this probably matters only for popular well established libraries used by many projects. E.g. Java protobuf 2.4 vs 2.5. Note that the major version is the same but they are still incompatible in a subtle way.

If the goal of this proposal is to create a document describing best practices for versioning then it should cover topics that I raised including backward compatibility. A document describing best practices for versioning, releasing, using vcs, maintain backward compatibility, etc would be very useful.

If the goal of this proposal is to make semver versions standard in the Go community then I think:

  • tools support should come first. gofmt mentioned in the proposal is the excellent example why
  • it is fairly easy to enforce the version format but it is much more difficult or even impossible to make people follow the rules behind it
@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Aug 27, 2015

Contributor

@technosophos @mattfarina thank you for your feedback. To reply specifically to your comments about the v prefix. This was the most contentious part of the discussion on go-pm that preceeded this proposal.

The preference for a v prefix comes from observations that three of the biggest Go projects, Docker, Kubernetes, and CoreOs all use that variation, it is also widely used by our language contemporaries like Rust, Nodejs and Bower (javascript). With that said, I'm sure an equal number of prominent examples could be found that do not have a v prefix.

If this proposal were amended to make the v prefix optional, would that be acceptable ?

Contributor

davecheney commented Aug 27, 2015

@technosophos @mattfarina thank you for your feedback. To reply specifically to your comments about the v prefix. This was the most contentious part of the discussion on go-pm that preceeded this proposal.

The preference for a v prefix comes from observations that three of the biggest Go projects, Docker, Kubernetes, and CoreOs all use that variation, it is also widely used by our language contemporaries like Rust, Nodejs and Bower (javascript). With that said, I'm sure an equal number of prominent examples could be found that do not have a v prefix.

If this proposal were amended to make the v prefix optional, would that be acceptable ?

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Aug 27, 2015

Contributor

@kostya-sh thank you for your feedback. To address your points

I beleive that incompatible versions of the same library should use different import paths. Otherwise they cannot co-exist as dependencies of a single project. Having them tagged with two different tags is not enough or even harmful.

I'm sorry but I do not agree with you. There must be only one copy (not version) of any package in a Go binary; the linker requires it. To subvert this by renaming the package, even well intentioned, so as to permit go get to have some way of identifying different major versions of packages, is in my opinion a mistake.

To give some background:

Canonical were, I believe, one of the first to attempt to use this method with the mgo driver back in August 2012 when we had to make an incompatible change to the API. At the time we thought were were rather clever, but since that time I have come to believe this was a mistake, for the following reasons:

  1. Putting the major version in the API is not sufficient for a repeatable build, at best it only guides Go get to the major API version. It does not give Go get sufficient information to fetch a specific release version or a specific revision. Using these versioned import paths, the Juju developers still had to develop an additional tool to make sure everyone had the same revision of every dependency checked out.
  2. Encoding a version number in the import path has a high operational overhead, I recently saw a patch from a co-worker that touched 220 files to update a dependency that used this format via the gopkg.in proxy. What is worse, this changed from v5 to v6-unstable, so you just know that in a week or so another 220 file patch will be coming to remove the -unstable prefix.
  3. Encoding a version number in the import path has a high cost for new adopters of the language. You have to find every reference in your project to one versioned import path, and replace it with another, even in the face of conditional compilation, even in tests, and so forth. As someone who is passionate about teaching newcomers about Go, the conceptual burden of educating newcomers about this requirement would be far to high.
  4. Finally, even with the most rigorous development standards for their own code, Go developers are still at risk from the dependencies they rely on not getting it right. With versioned import paths it is very easy to see two arms of a dependency graph importing different versions of a database driver who's init() function registers the driver. Have a read of the documentation for sql.Register. This is not a hypothetical problem, and one that must be prevented at all costs.

There a lot of examples of Java libraries where following this simple rule would have made life of library and application developers much easier. E.g. protobuf 2.4 vs 2.5, various versions of asm library, hadoop-1 vs hadoop-2, various versions of jetty, etc.

I think java has made a serious mistake by developing more (OSGI) and more (Project Jigsaw) complicated methods of classpath resolution to continue to support multiple versions of a library in the same JVM runtime. This complexity is entirely self inflicted and not something I want to see in a Go dependency management solution.

Very often for applications (not libraries) it makes little sense to use semver. Something like build number + git hash works well enough. Think about version numbers of such applications like Chrome, Firefox, systemd, less, etc.

I agree that for end user applications, projects which produce a program or server binary, versioning makes less sense, as by definition no code consumes them.

However, there are many downstream consumers of all Go projects that do care about version numbers, operating system distributions are a major one. I don't think your examples provide a compelling exception to the rule to argue against the value of semver.

Contributor

davecheney commented Aug 27, 2015

@kostya-sh thank you for your feedback. To address your points

I beleive that incompatible versions of the same library should use different import paths. Otherwise they cannot co-exist as dependencies of a single project. Having them tagged with two different tags is not enough or even harmful.

I'm sorry but I do not agree with you. There must be only one copy (not version) of any package in a Go binary; the linker requires it. To subvert this by renaming the package, even well intentioned, so as to permit go get to have some way of identifying different major versions of packages, is in my opinion a mistake.

To give some background:

Canonical were, I believe, one of the first to attempt to use this method with the mgo driver back in August 2012 when we had to make an incompatible change to the API. At the time we thought were were rather clever, but since that time I have come to believe this was a mistake, for the following reasons:

  1. Putting the major version in the API is not sufficient for a repeatable build, at best it only guides Go get to the major API version. It does not give Go get sufficient information to fetch a specific release version or a specific revision. Using these versioned import paths, the Juju developers still had to develop an additional tool to make sure everyone had the same revision of every dependency checked out.
  2. Encoding a version number in the import path has a high operational overhead, I recently saw a patch from a co-worker that touched 220 files to update a dependency that used this format via the gopkg.in proxy. What is worse, this changed from v5 to v6-unstable, so you just know that in a week or so another 220 file patch will be coming to remove the -unstable prefix.
  3. Encoding a version number in the import path has a high cost for new adopters of the language. You have to find every reference in your project to one versioned import path, and replace it with another, even in the face of conditional compilation, even in tests, and so forth. As someone who is passionate about teaching newcomers about Go, the conceptual burden of educating newcomers about this requirement would be far to high.
  4. Finally, even with the most rigorous development standards for their own code, Go developers are still at risk from the dependencies they rely on not getting it right. With versioned import paths it is very easy to see two arms of a dependency graph importing different versions of a database driver who's init() function registers the driver. Have a read of the documentation for sql.Register. This is not a hypothetical problem, and one that must be prevented at all costs.

There a lot of examples of Java libraries where following this simple rule would have made life of library and application developers much easier. E.g. protobuf 2.4 vs 2.5, various versions of asm library, hadoop-1 vs hadoop-2, various versions of jetty, etc.

I think java has made a serious mistake by developing more (OSGI) and more (Project Jigsaw) complicated methods of classpath resolution to continue to support multiple versions of a library in the same JVM runtime. This complexity is entirely self inflicted and not something I want to see in a Go dependency management solution.

Very often for applications (not libraries) it makes little sense to use semver. Something like build number + git hash works well enough. Think about version numbers of such applications like Chrome, Firefox, systemd, less, etc.

I agree that for end user applications, projects which produce a program or server binary, versioning makes less sense, as by definition no code consumes them.

However, there are many downstream consumers of all Go projects that do care about version numbers, operating system distributions are a major one. I don't think your examples provide a compelling exception to the rule to argue against the value of semver.

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Aug 27, 2015

Contributor

@kostya-sh to address your points

If the goal of this proposal is to create a document describing best practices for versioning then it should cover topics that I raised including backward compatibility. A document describing best practices for versioning, releasing, using vcs, maintain backward compatibility, etc would be very useful.

These are good points, if this proposal moves forward to the design document stage it should address these points.

If the goal of this proposal is to make semver versions standard in the Go community then I think:
tools support should come first. gofmt mentioned in the proposal is the excellent example why
it is fairly easy to enforce the version format but it is much more difficult or even impossible to make people follow the rules behind it

I disagree, as I mentioned about to @thockin this is the smallest possible proposal I can think of to make forward progress. Yes, larger proposals would be more comprehensive, but would take significantly longer to achieve consensus and would then be further delayed by a requirement to be scheduled into the Go release cycle -- do you really want to wait up to a year to see this implemented in a released version of Go ?

This proposal is the smallest I can make. It pushes all the questions of what versioning means to an accepted standard, semver, which is widely used across languages who we consider to be contemporaries (or competitors) to Go, and lets us benefit from their experiences rather than reinventing what would likely be a very similar wheel.

If you agree with that position, then this proposal boils down to a format for vcs tags that I believe can achieve consensus easily and can be of use immediately for all the tool makers, godoc.org maintainers, and operating system distros, in the Go ecosystem.

Thanks

Dave

Contributor

davecheney commented Aug 27, 2015

@kostya-sh to address your points

If the goal of this proposal is to create a document describing best practices for versioning then it should cover topics that I raised including backward compatibility. A document describing best practices for versioning, releasing, using vcs, maintain backward compatibility, etc would be very useful.

These are good points, if this proposal moves forward to the design document stage it should address these points.

If the goal of this proposal is to make semver versions standard in the Go community then I think:
tools support should come first. gofmt mentioned in the proposal is the excellent example why
it is fairly easy to enforce the version format but it is much more difficult or even impossible to make people follow the rules behind it

I disagree, as I mentioned about to @thockin this is the smallest possible proposal I can think of to make forward progress. Yes, larger proposals would be more comprehensive, but would take significantly longer to achieve consensus and would then be further delayed by a requirement to be scheduled into the Go release cycle -- do you really want to wait up to a year to see this implemented in a released version of Go ?

This proposal is the smallest I can make. It pushes all the questions of what versioning means to an accepted standard, semver, which is widely used across languages who we consider to be contemporaries (or competitors) to Go, and lets us benefit from their experiences rather than reinventing what would likely be a very similar wheel.

If you agree with that position, then this proposal boils down to a format for vcs tags that I believe can achieve consensus easily and can be of use immediately for all the tool makers, godoc.org maintainers, and operating system distros, in the Go ecosystem.

Thanks

Dave

@kostya-sh

This comment has been minimized.

Show comment
Hide comment
@kostya-sh

kostya-sh Aug 27, 2015

Contributor

@davecheney, I guess I didn't express myself clear enough. I agree with you that any kind of automated way of adding versions to the import paths is not the best way to solve backward (in)compatibility problems. Something like libv5, libv6, libv100 is certainly not what I want to see in Go projects.

Maintaining backward compatibility and using different import path for incompatible versions should be responsibility of package authors. Good example: go/types from Go 1.5 is incompatible with the original version, but it is not a big problem because import paths are different, application developers are free to choose one or another depending on their requirements.

However if a package changes in a backward incompatible way between v1 and v2 and keeps using the same import path then burden of resolving dependency mess is on users of the package. I am not aware of Go examples, but there are plenty in Java world (just because it is more widely used), I mentioned few of them in my previous comments.

I just wanted to suggest that raising awareness about these problems among package developers would be a good thing.

I think java has made a serious mistake by developing more (OSGI) and more (Project Jigsaw) complicated methods of classpath resolution to continue to support multiple versions of a library in the same JVM runtime. This complexity is entirely self inflicted and not something I want to see in a Go dependency management solution.

My examples were referring to a single class loader (no OSGI). You will be surprised how many similarities Go and Java have in package/library management. I would say that within a single class loader they are equivalent:

  • Go package import path = Java package name
  • Go static compilation = Many java applications come with all dependencies included in the distribution
  • Import path rewrite (e.g. hiding 3-rd party packages under internal) = shading (usually via maven-shade-plugin)
  • Vendoring = putting jar files that a project depends on in the source control (though it is getting less and less common mainly because of maven)

I disagree, as I mentioned about to @thockin this is the smallest possible proposal I can think of to make forward progress. Yes, larger proposals would be more comprehensive, but would take significantly longer to achieve consensus and would then be further delayed by a requirement to be scheduled into the Go release cycle -- do you really want to wait up to a year to see this implemented in a released version of Go ?

I am just a bit skeptical that the proposal alone would be enough. I might be wrong about that though as Go community seems to be quite united.

If you agree with that position, then this proposal boils down to a format for vcs tags that I believe can achieve consensus easily and can be of use immediately for all the tool makers, godoc.org maintainers, and operating system distros, in the Go ecosystem.

Unfortunately dealing with backward (in)compatibility is not as simple as enforcing semver and vcs tag format.

Very interesting study of libraries in the maven central repository that contains majority of open source Java libraries: http://avandeursen.com/2014/10/09/semantic-versioning-in-maven-central-breaking-changes/ . Almost 24% of patch updates introduced binary (detectable by tools) incompatibility!

Please treat my comments not as arguments against the proposal but as feedback and food for thoughts.

Contributor

kostya-sh commented Aug 27, 2015

@davecheney, I guess I didn't express myself clear enough. I agree with you that any kind of automated way of adding versions to the import paths is not the best way to solve backward (in)compatibility problems. Something like libv5, libv6, libv100 is certainly not what I want to see in Go projects.

Maintaining backward compatibility and using different import path for incompatible versions should be responsibility of package authors. Good example: go/types from Go 1.5 is incompatible with the original version, but it is not a big problem because import paths are different, application developers are free to choose one or another depending on their requirements.

However if a package changes in a backward incompatible way between v1 and v2 and keeps using the same import path then burden of resolving dependency mess is on users of the package. I am not aware of Go examples, but there are plenty in Java world (just because it is more widely used), I mentioned few of them in my previous comments.

I just wanted to suggest that raising awareness about these problems among package developers would be a good thing.

I think java has made a serious mistake by developing more (OSGI) and more (Project Jigsaw) complicated methods of classpath resolution to continue to support multiple versions of a library in the same JVM runtime. This complexity is entirely self inflicted and not something I want to see in a Go dependency management solution.

My examples were referring to a single class loader (no OSGI). You will be surprised how many similarities Go and Java have in package/library management. I would say that within a single class loader they are equivalent:

  • Go package import path = Java package name
  • Go static compilation = Many java applications come with all dependencies included in the distribution
  • Import path rewrite (e.g. hiding 3-rd party packages under internal) = shading (usually via maven-shade-plugin)
  • Vendoring = putting jar files that a project depends on in the source control (though it is getting less and less common mainly because of maven)

I disagree, as I mentioned about to @thockin this is the smallest possible proposal I can think of to make forward progress. Yes, larger proposals would be more comprehensive, but would take significantly longer to achieve consensus and would then be further delayed by a requirement to be scheduled into the Go release cycle -- do you really want to wait up to a year to see this implemented in a released version of Go ?

I am just a bit skeptical that the proposal alone would be enough. I might be wrong about that though as Go community seems to be quite united.

If you agree with that position, then this proposal boils down to a format for vcs tags that I believe can achieve consensus easily and can be of use immediately for all the tool makers, godoc.org maintainers, and operating system distros, in the Go ecosystem.

Unfortunately dealing with backward (in)compatibility is not as simple as enforcing semver and vcs tag format.

Very interesting study of libraries in the maven central repository that contains majority of open source Java libraries: http://avandeursen.com/2014/10/09/semantic-versioning-in-maven-central-breaking-changes/ . Almost 24% of patch updates introduced binary (detectable by tools) incompatibility!

Please treat my comments not as arguments against the proposal but as feedback and food for thoughts.

@kostya-sh

This comment has been minimized.

Show comment
Hide comment
@kostya-sh

kostya-sh Aug 27, 2015

Contributor

Also how would this proposal allow to version golang.org/x packages?

Contributor

kostya-sh commented Aug 27, 2015

Also how would this proposal allow to version golang.org/x packages?

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Aug 27, 2015

Contributor

@kostya-sh sorry for not addressing every one of your points

However if a package changes in a backward incompatible way between v1 and v2 and keeps using the same import path then burden of resolving dependency mess is on users of the package.

Yes, and this will always be true, and this will always be true if we have a release process and version numbers, or continue to muddle along using vcs revisions as a proxy.

I just wanted to suggest that raising awareness about these problems among package developers would be a good thing.

I agree, and think that semver is a concrete and well understood standard that works well. Do you think that Go should develop its own version numbering scheme ? What benefits do you think that would have over semver ?

My examples were referring to a single class loader (no OSGI). You will be surprised how many similarities Go and Java have in package/library management. I would say that within a single class loader they are equivalent:

Yes, the linking model for java, with its class path and the Go toolchain are quite similar and I draw a lot of inspiration from my work with Java and Maven in the past.

I am just a bit skeptical that the proposal alone would be enough. I might be wrong about that though as Go community seems to be quite united.

It's absolutely not enough. This proposal is just the start, more will follow if this one is accepted, but I belive that this is the place to start.

If you agree with that position, then this proposal boils down to a format for vcs tags that I believe can achieve consensus easily and can be of use immediately for all the tool makers, godoc.org maintainers, and operating system distros, in the Go ecosystem.
Unfortunately dealing with backward (in)compatibility is not as simple as enforcing semver and vcs tag format.

Could you please expand on this point and provide some specifics.

Very interesting study of libraries in the maven central repository that contains majority of open source Java libraries: http://avandeursen.com/2014/10/09/semantic-versioning-in-maven-central-breaking-changes/ . Almost 24% of patch updates introduced binary (detectable by tools) incompatibility!

To cut to the chase, semver is not a magic bullet, it cannot force developers to do the right thing, only incentivise them to do the right thing. I believe that this property would hold true no matter what version number methodology was proposed, semver or something of our own concoction.

I think there is a lot of benefit from working from a position of assuming Go programmers want to do the right thing, not engineer a straight jacket process which inhibits them from doing the wrong thing.

As you mentioned above social pressure work well in the Go community, I point to the ubiquitous success of gofmt as an example. Code that is no go formatted is considered my many to be a first order indicator of a lack of quality.

Similarly, adherence to a commonly accepted ideal of what constitutes semver's major, minor and patch release definitions will come through the same social pressures, and we all benefit from a shared vocabulary across our counterparts in node, ruby and rust.

Please treat my comments not as arguments against the proposal but as feedback and food for thoughts.

Thank you for your continued feedback, I appreciate your experience and it is very valuable to me.

Contributor

davecheney commented Aug 27, 2015

@kostya-sh sorry for not addressing every one of your points

However if a package changes in a backward incompatible way between v1 and v2 and keeps using the same import path then burden of resolving dependency mess is on users of the package.

Yes, and this will always be true, and this will always be true if we have a release process and version numbers, or continue to muddle along using vcs revisions as a proxy.

I just wanted to suggest that raising awareness about these problems among package developers would be a good thing.

I agree, and think that semver is a concrete and well understood standard that works well. Do you think that Go should develop its own version numbering scheme ? What benefits do you think that would have over semver ?

My examples were referring to a single class loader (no OSGI). You will be surprised how many similarities Go and Java have in package/library management. I would say that within a single class loader they are equivalent:

Yes, the linking model for java, with its class path and the Go toolchain are quite similar and I draw a lot of inspiration from my work with Java and Maven in the past.

I am just a bit skeptical that the proposal alone would be enough. I might be wrong about that though as Go community seems to be quite united.

It's absolutely not enough. This proposal is just the start, more will follow if this one is accepted, but I belive that this is the place to start.

If you agree with that position, then this proposal boils down to a format for vcs tags that I believe can achieve consensus easily and can be of use immediately for all the tool makers, godoc.org maintainers, and operating system distros, in the Go ecosystem.
Unfortunately dealing with backward (in)compatibility is not as simple as enforcing semver and vcs tag format.

Could you please expand on this point and provide some specifics.

Very interesting study of libraries in the maven central repository that contains majority of open source Java libraries: http://avandeursen.com/2014/10/09/semantic-versioning-in-maven-central-breaking-changes/ . Almost 24% of patch updates introduced binary (detectable by tools) incompatibility!

To cut to the chase, semver is not a magic bullet, it cannot force developers to do the right thing, only incentivise them to do the right thing. I believe that this property would hold true no matter what version number methodology was proposed, semver or something of our own concoction.

I think there is a lot of benefit from working from a position of assuming Go programmers want to do the right thing, not engineer a straight jacket process which inhibits them from doing the wrong thing.

As you mentioned above social pressure work well in the Go community, I point to the ubiquitous success of gofmt as an example. Code that is no go formatted is considered my many to be a first order indicator of a lack of quality.

Similarly, adherence to a commonly accepted ideal of what constitutes semver's major, minor and patch release definitions will come through the same social pressures, and we all benefit from a shared vocabulary across our counterparts in node, ruby and rust.

Please treat my comments not as arguments against the proposal but as feedback and food for thoughts.

Thank you for your continued feedback, I appreciate your experience and it is very valuable to me.

@kardianos

This comment has been minimized.

Show comment
Hide comment
@kardianos

kardianos Aug 27, 2015

Contributor

@kostya-sh This proposal would not allow effective versioning of the golang.org/x packages because those repositories contain packages that will version at vastly different rates. This proposal explicitly defines the unit of version to equal the repository. For instance golang.org/x/crypto/ssh will version differently then golang.org/crypto/md4, or golang.org/x/net/context will version much more slowly then golang.org/x/net/websocket.

In previously conversations regarding this proposal it was commented that the golang.org/x repos were really too large/mixed and should be more granular ideally.

Contributor

kardianos commented Aug 27, 2015

@kostya-sh This proposal would not allow effective versioning of the golang.org/x packages because those repositories contain packages that will version at vastly different rates. This proposal explicitly defines the unit of version to equal the repository. For instance golang.org/x/crypto/ssh will version differently then golang.org/crypto/md4, or golang.org/x/net/context will version much more slowly then golang.org/x/net/websocket.

In previously conversations regarding this proposal it was commented that the golang.org/x repos were really too large/mixed and should be more granular ideally.

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Aug 27, 2015

Contributor

Also how would this proposal allow to version golang.org/x packages?

@kostya-sh Yes, should this proposal be accepted, I believe these projects should adopt the version number format and provide consumers with released versions via tags.

Contributor

davecheney commented Aug 27, 2015

Also how would this proposal allow to version golang.org/x packages?

@kostya-sh Yes, should this proposal be accepted, I believe these projects should adopt the version number format and provide consumers with released versions via tags.

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Aug 27, 2015

Contributor

@kardianos I understand that you want to version Go repositories on a per package basis. I recognise the rational for that, and the possibilities that would unlock.

I also recognise that this would be much harder to explain to newcomers, much more nuanced for toolmakers, and ultimately a far more complex solution than I think is necessary.

If developers are unhappy with the granularity of a single version number series per repository, they should split the unrelated pieces out into independent repositories.

If this should be done for your examples of the crypto and net projects is outside the scope of this proposal.

Thank you

Dave

Contributor

davecheney commented Aug 27, 2015

@kardianos I understand that you want to version Go repositories on a per package basis. I recognise the rational for that, and the possibilities that would unlock.

I also recognise that this would be much harder to explain to newcomers, much more nuanced for toolmakers, and ultimately a far more complex solution than I think is necessary.

If developers are unhappy with the granularity of a single version number series per repository, they should split the unrelated pieces out into independent repositories.

If this should be done for your examples of the crypto and net projects is outside the scope of this proposal.

Thank you

Dave

@philips

This comment has been minimized.

Show comment
Hide comment
@philips

philips Aug 27, 2015

+1 on this proposal. It would be very helpful for a community convention to be established. Today CoreOS projects like etcd, rkt, and flannel, depend on a number of Go libraries. And it would be great if there was an overall community convention to make regular library releases so instead of selecting the version based on what master was whenever our team imported the library with godep we could start relying on a git tag instead.

philips commented Aug 27, 2015

+1 on this proposal. It would be very helpful for a community convention to be established. Today CoreOS projects like etcd, rkt, and flannel, depend on a number of Go libraries. And it would be great if there was an overall community convention to make regular library releases so instead of selecting the version based on what master was whenever our team imported the library with godep we could start relying on a git tag instead.

@kostya-sh

This comment has been minimized.

Show comment
Hide comment
@kostya-sh

kostya-sh Aug 27, 2015

Contributor

I agree, and think that semver is a concrete and well understood standard that works well. Do you think that Go should develop its own version numbering scheme ? What benefits do you think that would have over semver ?

I think that it doesn't matter much what version numbering scheme is used. However when there is a possibility that two incompatible versions of the same package are used as dependencies of a single project (in semver terms libfoo 1.x and libfoo 2.y) then having libfooone x and libfootwo y is easier to deal with. So semver way of dealing with breaking changes by incrementing major version doesn't work in all cases.

I understand that this is not applicable to all packages and cannot be enforced thus this is just a recommendation.

I point to the ubiquitous success of gofmt as an example. Code that is no go formatted is considered my many to be a first order indicator of a lack of quality.

I think you are mixing the cause and effect here. I believe that the only reason of a single formatting style for Go code is the existence of gofmt. The tool came first and no formal document describing formatting rules was necessary. Compare this with PEP-8 and python.

Why no one tags their repositories now (is it true btw?)? Could it be because there are no tools that support tags?

Contributor

kostya-sh commented Aug 27, 2015

I agree, and think that semver is a concrete and well understood standard that works well. Do you think that Go should develop its own version numbering scheme ? What benefits do you think that would have over semver ?

I think that it doesn't matter much what version numbering scheme is used. However when there is a possibility that two incompatible versions of the same package are used as dependencies of a single project (in semver terms libfoo 1.x and libfoo 2.y) then having libfooone x and libfootwo y is easier to deal with. So semver way of dealing with breaking changes by incrementing major version doesn't work in all cases.

I understand that this is not applicable to all packages and cannot be enforced thus this is just a recommendation.

I point to the ubiquitous success of gofmt as an example. Code that is no go formatted is considered my many to be a first order indicator of a lack of quality.

I think you are mixing the cause and effect here. I believe that the only reason of a single formatting style for Go code is the existence of gofmt. The tool came first and no formal document describing formatting rules was necessary. Compare this with PEP-8 and python.

Why no one tags their repositories now (is it true btw?)? Could it be because there are no tools that support tags?

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Aug 27, 2015

Contributor

Why no one tags their repositories now (is it true btw?)? Could it be because there are no tools that support tags?

This is almost certainly the case. Right now, few Go projects have a release process, and those that do use a tag format of their choosing. Because of this, no tools exist to mechanically consume release tags (although many tools exist that leave the problem of choosing the tag, branch or revision entirely to the user), hence there is no impetus to do releases.

I think we're both saying the same thing, we both want to break this cycle. You want a tool to be written first, and I want agreement on the format first, then the tool. I think my approach has a higher chance of achieving consensus as it requires no code change (yet), and will enable the entire Go ecosystem to benefit as soon as it is adopted, not when the tool, and I'm assuming you mean the go tool inside the Go distribution itself, ships.

Contributor

davecheney commented Aug 27, 2015

Why no one tags their repositories now (is it true btw?)? Could it be because there are no tools that support tags?

This is almost certainly the case. Right now, few Go projects have a release process, and those that do use a tag format of their choosing. Because of this, no tools exist to mechanically consume release tags (although many tools exist that leave the problem of choosing the tag, branch or revision entirely to the user), hence there is no impetus to do releases.

I think we're both saying the same thing, we both want to break this cycle. You want a tool to be written first, and I want agreement on the format first, then the tool. I think my approach has a higher chance of achieving consensus as it requires no code change (yet), and will enable the entire Go ecosystem to benefit as soon as it is adopted, not when the tool, and I'm assuming you mean the go tool inside the Go distribution itself, ships.

@peterbourgon

This comment has been minimized.

Show comment
Hide comment
@peterbourgon

peterbourgon Aug 27, 2015

Member

A strong +1 to the proposal. It would be wonderful to provide consumers of Go kit with version semantics, and the most widely-adopted way of doing that now (gopkg.in) is unsuitable for a variety of reasons.

I would also like to see a larger proposal, including some suggestions (if not spec implementations) for tool integration. But I defer to @davecheney's preference for a smaller scope. SemVer 2.0 is the obvious choice for schema, and versioning whole repositories strikes a good balance between versatility and simplicity.

@kostya-sh

However when there is a possibility that two incompatible versions of the same package are used as dependencies of a single project (in semver terms libfoo 1.x and libfoo 2.y) . . .

This is a distraction, because—as I understand it—it categorically can't be allowed to occur. Two versions of the same package can't be imported into the same dependency graph, because of issues like the double-driver-registration problem enumerated by @davecheney above.

Member

peterbourgon commented Aug 27, 2015

A strong +1 to the proposal. It would be wonderful to provide consumers of Go kit with version semantics, and the most widely-adopted way of doing that now (gopkg.in) is unsuitable for a variety of reasons.

I would also like to see a larger proposal, including some suggestions (if not spec implementations) for tool integration. But I defer to @davecheney's preference for a smaller scope. SemVer 2.0 is the obvious choice for schema, and versioning whole repositories strikes a good balance between versatility and simplicity.

@kostya-sh

However when there is a possibility that two incompatible versions of the same package are used as dependencies of a single project (in semver terms libfoo 1.x and libfoo 2.y) . . .

This is a distraction, because—as I understand it—it categorically can't be allowed to occur. Two versions of the same package can't be imported into the same dependency graph, because of issues like the double-driver-registration problem enumerated by @davecheney above.

@rsc

This comment has been minimized.

Show comment
Hide comment
@rsc

rsc Nov 23, 2015

Contributor
Contributor

rsc commented Nov 23, 2015

@perillo

This comment has been minimized.

Show comment
Hide comment
@perillo

perillo Nov 23, 2015

On Mon, Nov 23, 2015 at 4:19 PM, Russ Cox notifications@github.com wrote:

Everyone continues to talk about how to identify versions. We can keep
talking about that, of course, but it's not the real problem.

The real problem is OK, suppose there are perfect version identifiers with
semantically guaranteed meanings. Then what? That's the big question that
needs to be answered here.

As an example:

  1. Since go by default builds static app, store the imported packages
    (including standard ones) and their version in the app, using a custom
    object header.
  2. Provide a service to catalog severe bugs of go packages (note that I'm
    using package not project as it is usually done).

With this information I (sysadmin) can check when and why I need to update
a go application on my system, using a tool.

perillo commented Nov 23, 2015

On Mon, Nov 23, 2015 at 4:19 PM, Russ Cox notifications@github.com wrote:

Everyone continues to talk about how to identify versions. We can keep
talking about that, of course, but it's not the real problem.

The real problem is OK, suppose there are perfect version identifiers with
semantically guaranteed meanings. Then what? That's the big question that
needs to be answered here.

As an example:

  1. Since go by default builds static app, store the imported packages
    (including standard ones) and their version in the app, using a custom
    object header.
  2. Provide a service to catalog severe bugs of go packages (note that I'm
    using package not project as it is usually done).

With this information I (sysadmin) can check when and why I need to update
a go application on my system, using a tool.

@kardianos

This comment has been minimized.

Show comment
Hide comment
@kardianos

kardianos Nov 23, 2015

Contributor

I asked a similar question to @rsc last week which boils down to: (1) what is your current vendor process? (2) What is currently problematic in your current process? (3) If everyone released and used semver, what could you envision your vendor process could be?
https://forum.golangbridge.org/t/what-is-your-process-for-dealing-with-vendor-packages/1541

I will be adding support for remote packages and potentially semver to govendor. However I don't really know what semver support will look like. I would love input on this.

@perillo Without offering an opinion on the idea, I think that would be an independent proposal. Even if no semvers were used, a machine could very quickly check a range of revision hashes. Something like the vendor-spec could help you there record versions or revisions.

Contributor

kardianos commented Nov 23, 2015

I asked a similar question to @rsc last week which boils down to: (1) what is your current vendor process? (2) What is currently problematic in your current process? (3) If everyone released and used semver, what could you envision your vendor process could be?
https://forum.golangbridge.org/t/what-is-your-process-for-dealing-with-vendor-packages/1541

I will be adding support for remote packages and potentially semver to govendor. However I don't really know what semver support will look like. I would love input on this.

@perillo Without offering an opinion on the idea, I think that would be an independent proposal. Even if no semvers were used, a machine could very quickly check a range of revision hashes. Something like the vendor-spec could help you there record versions or revisions.

@mattfarina

This comment has been minimized.

Show comment
Hide comment
@mattfarina

mattfarina Nov 23, 2015

Member

Then what? That's the big question that needs to be answered here.

@rsc That's a great question. I think the answer is we figure it out as we go. We needed a place to put packages that wasn't the GOPATH so vendor/ was created. The next step wasn't worked out. We now need a consistent way to handle versions. The next step(s) are being worked out. By adding SemVer it enables the community to keep moving forward to work it out.

Member

mattfarina commented Nov 23, 2015

Then what? That's the big question that needs to be answered here.

@rsc That's a great question. I think the answer is we figure it out as we go. We needed a place to put packages that wasn't the GOPATH so vendor/ was created. The next step wasn't worked out. We now need a consistent way to handle versions. The next step(s) are being worked out. By adding SemVer it enables the community to keep moving forward to work it out.

@rsc

This comment has been minimized.

Show comment
Hide comment
@rsc

rsc Nov 23, 2015

Contributor
Contributor

rsc commented Nov 23, 2015

@perillo

This comment has been minimized.

Show comment
Hide comment
@perillo

perillo Nov 23, 2015

On Mon, Nov 23, 2015 at 7:19 PM, Russ Cox notifications@github.com wrote:

[....]

My suggestion would be that people continue to experiment with SemVer
support in tools like glide and gb, to learn answers to those questions.
Those answers would inform this discussion considerably and help move the
proposal from "say it's good" to "do these specific things to support it".

A language that I know is supporting SemVer with the aid of tools is Elm.
See as an example:
elm-lang/elm-package#90
https://github.com/elm-lang/elm-package

The idea is that a version change can be handled automatically by a tool,
thanks to the use of SemVer.
The problem is that a tool can not prove that a project is using SemVer
correctly.

perillo commented Nov 23, 2015

On Mon, Nov 23, 2015 at 7:19 PM, Russ Cox notifications@github.com wrote:

[....]

My suggestion would be that people continue to experiment with SemVer
support in tools like glide and gb, to learn answers to those questions.
Those answers would inform this discussion considerably and help move the
proposal from "say it's good" to "do these specific things to support it".

A language that I know is supporting SemVer with the aid of tools is Elm.
See as an example:
elm-lang/elm-package#90
https://github.com/elm-lang/elm-package

The idea is that a version change can be handled automatically by a tool,
thanks to the use of SemVer.
The problem is that a tool can not prove that a project is using SemVer
correctly.

@kardianos

This comment has been minimized.

Show comment
Hide comment
@kardianos

kardianos Nov 23, 2015

Contributor

@perillo I think we are well aware of other languages that use semver. I would ask, what would you envision this look like in go? What would a workflow look like? What would the tool do with semver?

Contributor

kardianos commented Nov 23, 2015

@perillo I think we are well aware of other languages that use semver. I would ask, what would you envision this look like in go? What would a workflow look like? What would the tool do with semver?

@perillo

This comment has been minimized.

Show comment
Hide comment
@perillo

perillo Nov 26, 2015

On Mon, Nov 23, 2015 at 8:26 PM, Daniel Theophanes <notifications@github.com

wrote:

@perillo https://github.com/perillo I think we are well aware of other
languages that use semver. I would ask, what would you envision this look
like in go? What would a workflow look like? What would the tool do with
semver?

I took some time to read carefully the proposal and all the comments, and I
come up with an alternate specification:
https://gist.github.com/perillo/06fe1d3e86f9cbf79b28

The main difference is that my specification proposes both how to attach a
version to a package inside its source control repository, and how to
request a specified version of a package using go get. The specification
also proposes some changes to go get documentation, how a remote import path must be handled, and to the go version and go tool api commands.
All changes do not break compatibility.

Another difference is that my specification also allows partial
versions. This is a very important question.

Finally, my specification try to be compatible with current practice.

Thanks Manlio

perillo commented Nov 26, 2015

On Mon, Nov 23, 2015 at 8:26 PM, Daniel Theophanes <notifications@github.com

wrote:

@perillo https://github.com/perillo I think we are well aware of other
languages that use semver. I would ask, what would you envision this look
like in go? What would a workflow look like? What would the tool do with
semver?

I took some time to read carefully the proposal and all the comments, and I
come up with an alternate specification:
https://gist.github.com/perillo/06fe1d3e86f9cbf79b28

The main difference is that my specification proposes both how to attach a
version to a package inside its source control repository, and how to
request a specified version of a package using go get. The specification
also proposes some changes to go get documentation, how a remote import path must be handled, and to the go version and go tool api commands.
All changes do not break compatibility.

Another difference is that my specification also allows partial
versions. This is a very important question.

Finally, my specification try to be compatible with current practice.

Thanks Manlio

@Merovius

This comment has been minimized.

Show comment
Hide comment
@Merovius

Merovius Nov 26, 2015

I am sorry to only have skimmed the discussion (it's very long), but I think this point wasn't made before:
I would request that a proposal like this, that proposes semantic versions also gives a good definition of what the individual levels of compatibility as specified in the SemVer spec actually mean for go (as there seem to be either no, or very inconsistent notions of this in the community). In particular, as I've argued before, I don't think the language allows minor minor, only patch releases and major releases. You can read my argument here but it mainly comes down to the fact that due to dot-imports, type embedding and strict typing, every API change potentially breaks the build of some reverse dependency.

I might be able to agree that this is unnecessarily nit-picky, but I do firmly believe, that if the community agrees on a semantic versioning scheme, they also need to agree on the semantics of that scheme (i.e. if you arbitrarily allow some breakages in minor version, we should at least agree which) and that agreement shouldn't be just hand-wavy (so far, the strictest definition I could get out of people was "minor changes, like adding a field").

Also, I'm a bit unhappy that this proposal conflates "releases" and "versions", which I would consider two different things. I agree with lots of the cited benefits of releases but I am not at all a fan of SemVer. I would prefer a release process like "master is always the current stable release, development happens on a separate branch that is periodically merged into master when it's considered stable", which would make all the current tools pretty happy (but wouldn't provide nor be as restrictive as requiring SemVer).

I am sorry to only have skimmed the discussion (it's very long), but I think this point wasn't made before:
I would request that a proposal like this, that proposes semantic versions also gives a good definition of what the individual levels of compatibility as specified in the SemVer spec actually mean for go (as there seem to be either no, or very inconsistent notions of this in the community). In particular, as I've argued before, I don't think the language allows minor minor, only patch releases and major releases. You can read my argument here but it mainly comes down to the fact that due to dot-imports, type embedding and strict typing, every API change potentially breaks the build of some reverse dependency.

I might be able to agree that this is unnecessarily nit-picky, but I do firmly believe, that if the community agrees on a semantic versioning scheme, they also need to agree on the semantics of that scheme (i.e. if you arbitrarily allow some breakages in minor version, we should at least agree which) and that agreement shouldn't be just hand-wavy (so far, the strictest definition I could get out of people was "minor changes, like adding a field").

Also, I'm a bit unhappy that this proposal conflates "releases" and "versions", which I would consider two different things. I agree with lots of the cited benefits of releases but I am not at all a fan of SemVer. I would prefer a release process like "master is always the current stable release, development happens on a separate branch that is periodically merged into master when it's considered stable", which would make all the current tools pretty happy (but wouldn't provide nor be as restrictive as requiring SemVer).

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Nov 26, 2015

Contributor

@Merovius thanks for your comments.

re: semvar, I can see you hold strong objections on number of levels. I'm sure I mentioned above that the choice of semver is convenience. To choose some other methodology would:

a. not change this proposal in spirit.
b. just be another thing that go does different, "Will go 2.0 finally adopt semver?" threads from here to eternity.
c. Probably end up being 90% the same as semver.

Your major objection to the semver standard is a lack of clarity over how version numbers are chosen. I've read your post on backwards compatibility and I agree totally. Noting that this proposal does no give any guidance for how authors choose version numbers, Go has the tools to let people write tools that could analyse the difference between two trees and give guidance for what the next version number should be based on the changes to the public symbols. Would that satisfy your concerns ?

re: releases vs versions. This is a proposal for releasing go packages, you apply version numbers to them to release them. The target of this is machine consumption. You mention you don't like "conflating" releases and versions, do you have an alternative suggestion ?

Contributor

davecheney commented Nov 26, 2015

@Merovius thanks for your comments.

re: semvar, I can see you hold strong objections on number of levels. I'm sure I mentioned above that the choice of semver is convenience. To choose some other methodology would:

a. not change this proposal in spirit.
b. just be another thing that go does different, "Will go 2.0 finally adopt semver?" threads from here to eternity.
c. Probably end up being 90% the same as semver.

Your major objection to the semver standard is a lack of clarity over how version numbers are chosen. I've read your post on backwards compatibility and I agree totally. Noting that this proposal does no give any guidance for how authors choose version numbers, Go has the tools to let people write tools that could analyse the difference between two trees and give guidance for what the next version number should be based on the changes to the public symbols. Would that satisfy your concerns ?

re: releases vs versions. This is a proposal for releasing go packages, you apply version numbers to them to release them. The target of this is machine consumption. You mention you don't like "conflating" releases and versions, do you have an alternative suggestion ?

@perillo

This comment has been minimized.

Show comment
Hide comment
@perillo

perillo Nov 27, 2015

On Thu, Nov 26, 2015 at 7:51 PM, Axel Wagner notifications@github.com
wrote:

I am sorry to only have skimmed the discussion (it's very long), but I
think this point wasn't made before:
I would request that a proposal like this, that proposes semantic versions
also gives a good definition of what the individual levels of compatibility
as specified in the SemVer spec actually mean for go (as there seem to be
either no, or very inconsistent notions of this in the community). In
particular, as I've argued before, I don't think the language allows minor
minor, only patch releases and major releases./You can read my argument
here
http://blog.merovius.de/2015/07/29/backwards-compatibility-in-go.html
but it mainly comes down to the fact that due to dot-imports, type
embedding and strict typing, every API change potentially breaks the
build of some reverse dependency.

compatibility for Go is specified here:
https://golang.org/doc/go1compat
so the same definition of compatibility can be used for external packages.

I might be able to agree that this is unnecessarily nit-picky, but I do
firmly believe, that if the community agrees on a semantic versioning
scheme, they also need to agree on the semantics of that scheme (i.e. if
you arbitrarily allow some breakages in minor version, we should at
least agree which) and that agreement shouldn't be just hand-wavy (so
far, the strictest definition I could get out of people was "minor changes,
like adding a field").

Also, I'm a bit unhappy that this proposal conflates "releases" and
"versions", which I would consider two different things.

I agree that releases and versions are different.
Versions can be considered tagged releases.

I agree with lots of the cited benefits of releases but I am not at all a
fan of SemVer. I would prefer a release process like "master is always the
current stable release, development happens on a separate branch that is
periodically merged into master when it's considered stable", which would
make all the current tools pretty happy (but wouldn't provide nor be as
restrictive as requiring SemVer).

Note that, should versions support be added to standard Go tools, you will
never be forced to use them.
The current behavior of go get will be the same: if no version is specified
it will get the latest revision (not the latest version).
I expect that any external tool will offer you the option to get the latest
revision and not the latest version.

Also note that SemVer has nothing to do with this problem.
A release process like "master is always the current stable release," is
simply a release process where no version is actual assigned to a
revision.

perillo commented Nov 27, 2015

On Thu, Nov 26, 2015 at 7:51 PM, Axel Wagner notifications@github.com
wrote:

I am sorry to only have skimmed the discussion (it's very long), but I
think this point wasn't made before:
I would request that a proposal like this, that proposes semantic versions
also gives a good definition of what the individual levels of compatibility
as specified in the SemVer spec actually mean for go (as there seem to be
either no, or very inconsistent notions of this in the community). In
particular, as I've argued before, I don't think the language allows minor
minor, only patch releases and major releases./You can read my argument
here
http://blog.merovius.de/2015/07/29/backwards-compatibility-in-go.html
but it mainly comes down to the fact that due to dot-imports, type
embedding and strict typing, every API change potentially breaks the
build of some reverse dependency.

compatibility for Go is specified here:
https://golang.org/doc/go1compat
so the same definition of compatibility can be used for external packages.

I might be able to agree that this is unnecessarily nit-picky, but I do
firmly believe, that if the community agrees on a semantic versioning
scheme, they also need to agree on the semantics of that scheme (i.e. if
you arbitrarily allow some breakages in minor version, we should at
least agree which) and that agreement shouldn't be just hand-wavy (so
far, the strictest definition I could get out of people was "minor changes,
like adding a field").

Also, I'm a bit unhappy that this proposal conflates "releases" and
"versions", which I would consider two different things.

I agree that releases and versions are different.
Versions can be considered tagged releases.

I agree with lots of the cited benefits of releases but I am not at all a
fan of SemVer. I would prefer a release process like "master is always the
current stable release, development happens on a separate branch that is
periodically merged into master when it's considered stable", which would
make all the current tools pretty happy (but wouldn't provide nor be as
restrictive as requiring SemVer).

Note that, should versions support be added to standard Go tools, you will
never be forced to use them.
The current behavior of go get will be the same: if no version is specified
it will get the latest revision (not the latest version).
I expect that any external tool will offer you the option to get the latest
revision and not the latest version.

Also note that SemVer has nothing to do with this problem.
A release process like "master is always the current stable release," is
simply a release process where no version is actual assigned to a
revision.

@Merovius

This comment has been minimized.

Show comment
Hide comment
@Merovius

Merovius Nov 27, 2015

@davecheney

Your major objection to the semver standard is a lack of clarity over how version numbers are chosen.

Not really. My major objection is about the whole idea of defining "breakage" and "compatibility" in terms of an API in isolation, i.e. about the "semantic" part in "SemVer". I see the usefulness of versions to give human readable names to releases, I also acknowledge that it is useful to imply a total ordering on versions. But if you want to assign semantic meaning apart from that to a version, you would either a) have potentially false version-matches, e.g. I require 1.7.0 or above and 1.8.0 breaks me because the author considered a change-non-breaking but there is no such thing in go or b) have probable false version-mismatches, e.g. I require version 1.0 or above, the author did a minor change in the API that broke nothing in practice, but to avoid a) they chose to increment to version 2.0 which is assumed to break me as the major version increased.

I think it's hard to reconcile any SemVer-equivalent versioning scheme with these concerns, while a purely chronological versioning scheme (e.g. every release increments a counter) that isn't used to make semantic deductions could.

As such, a tool to help in determining the next version number can't really ameliorate my concerns. In fact, the mentioned blog posts and my position towards SemVer is born out of me trying to write exactly this tool about half a year ago. I realized half-way through that there is no useful (read: prevents breakages but does not restrict language use too much) notion of compatibility for go.

So I maintain my position that the best way would be to at least not use semantic versions, but treat release-tags as convenience identifiers that have no meaning apart from that (and possibly a purely chronological ordering). I don't think that versions are the correct way to tackle the compatibility problem, but that this should rather be done by better tooling and conventions to work around that and actually look at the code that uses an API, instead of the API itself. A set of tools that helps you to choose the correct version to install and package based on the actual API and the code of the actual users of that API, instead of some ultimately heuristic number assigned to it.

@perillo Thanks for your comments :) I have read the compatibility guarantee extensively, but I don't think it is a sufficient definition, because a) most community members I have talked to so far consider things that break this guarantee to be non-breaking and b) even the go stdlib constantly breaks it, if you are being precise. I see the go 1 compatibility guarantee more as a statement of intent then an actual definition of what compatibility means (and to be clear: That's still far better than anything else I've seen for other languages). I recommend reading the blog post I linked to earlier on why I think this needs more spelling out.

Also note that SemVer has nothing to do with this problem.

I respectfully disagree. It has everything to do with this problem, it is part of the proposal and my issues with the proposal are completely attributed to this.

A release process like "master is always the current stable release," is
simply a release process where no version is actual assigned to a
revision.

Precisely. I wanted to demonstrate that it is possible to have releases without having versions (which is what I would prefer).

@davecheney

Your major objection to the semver standard is a lack of clarity over how version numbers are chosen.

Not really. My major objection is about the whole idea of defining "breakage" and "compatibility" in terms of an API in isolation, i.e. about the "semantic" part in "SemVer". I see the usefulness of versions to give human readable names to releases, I also acknowledge that it is useful to imply a total ordering on versions. But if you want to assign semantic meaning apart from that to a version, you would either a) have potentially false version-matches, e.g. I require 1.7.0 or above and 1.8.0 breaks me because the author considered a change-non-breaking but there is no such thing in go or b) have probable false version-mismatches, e.g. I require version 1.0 or above, the author did a minor change in the API that broke nothing in practice, but to avoid a) they chose to increment to version 2.0 which is assumed to break me as the major version increased.

I think it's hard to reconcile any SemVer-equivalent versioning scheme with these concerns, while a purely chronological versioning scheme (e.g. every release increments a counter) that isn't used to make semantic deductions could.

As such, a tool to help in determining the next version number can't really ameliorate my concerns. In fact, the mentioned blog posts and my position towards SemVer is born out of me trying to write exactly this tool about half a year ago. I realized half-way through that there is no useful (read: prevents breakages but does not restrict language use too much) notion of compatibility for go.

So I maintain my position that the best way would be to at least not use semantic versions, but treat release-tags as convenience identifiers that have no meaning apart from that (and possibly a purely chronological ordering). I don't think that versions are the correct way to tackle the compatibility problem, but that this should rather be done by better tooling and conventions to work around that and actually look at the code that uses an API, instead of the API itself. A set of tools that helps you to choose the correct version to install and package based on the actual API and the code of the actual users of that API, instead of some ultimately heuristic number assigned to it.

@perillo Thanks for your comments :) I have read the compatibility guarantee extensively, but I don't think it is a sufficient definition, because a) most community members I have talked to so far consider things that break this guarantee to be non-breaking and b) even the go stdlib constantly breaks it, if you are being precise. I see the go 1 compatibility guarantee more as a statement of intent then an actual definition of what compatibility means (and to be clear: That's still far better than anything else I've seen for other languages). I recommend reading the blog post I linked to earlier on why I think this needs more spelling out.

Also note that SemVer has nothing to do with this problem.

I respectfully disagree. It has everything to do with this problem, it is part of the proposal and my issues with the proposal are completely attributed to this.

A release process like "master is always the current stable release," is
simply a release process where no version is actual assigned to a
revision.

Precisely. I wanted to demonstrate that it is possible to have releases without having versions (which is what I would prefer).

@perillo

This comment has been minimized.

Show comment
Hide comment
@perillo

perillo Nov 27, 2015

@Merovius
I have read your blog post, but this phrase: I really love go, I really hate vendoring and up until now I didn't really get, why anyone would think go should need something like that means that we have different types of workflow when working with code.

I work as a freelance developer. A client ask me a software, I develop it, release it and after some time I forget about it. Suppose after a year the client call me telling he need a new feature or a bug fix: do you think I want to spend more time than necessary trying to update everything to the latest revision, checking that it really works?

SemVer does not solve the problem, but it, with the aid of good tools, will make my life better.

perillo commented Nov 27, 2015

@Merovius
I have read your blog post, but this phrase: I really love go, I really hate vendoring and up until now I didn't really get, why anyone would think go should need something like that means that we have different types of workflow when working with code.

I work as a freelance developer. A client ask me a software, I develop it, release it and after some time I forget about it. Suppose after a year the client call me telling he need a new feature or a bug fix: do you think I want to spend more time than necessary trying to update everything to the latest revision, checking that it really works?

SemVer does not solve the problem, but it, with the aid of good tools, will make my life better.

@peterbourgon

This comment has been minimized.

Show comment
Hide comment
@peterbourgon

peterbourgon Nov 27, 2015

Member

@Merovius With respect, I believe you are letting the perfect become the enemy of the good. Semantic versioning is the de-facto standard for versioning in ~every programming ecosystem (to some order of approximation). Developers expect it, or something like it, when approaching this problem space. Its failures, which you identify, do not prevent it from being useful, as @perillo notes. And, it seems that adopting this proposal doesn't prevent package authors from opting out of versioning at their discretion, and asking their consumers to vendor at specific revisions. — What do you think?

Member

peterbourgon commented Nov 27, 2015

@Merovius With respect, I believe you are letting the perfect become the enemy of the good. Semantic versioning is the de-facto standard for versioning in ~every programming ecosystem (to some order of approximation). Developers expect it, or something like it, when approaching this problem space. Its failures, which you identify, do not prevent it from being useful, as @perillo notes. And, it seems that adopting this proposal doesn't prevent package authors from opting out of versioning at their discretion, and asking their consumers to vendor at specific revisions. — What do you think?

@perillo

This comment has been minimized.

Show comment
Hide comment
@perillo

perillo Nov 27, 2015

@peterbourgon
Right; the tools should work as they work currently when no explicit version is required.
New tools should offer support for requiring revisions, instead of versions (but IMHO, they should only support fetching the latest revision (as go get); the user can always cd into the vendor directory and git checkout at a specific revision.

perillo commented Nov 27, 2015

@peterbourgon
Right; the tools should work as they work currently when no explicit version is required.
New tools should offer support for requiring revisions, instead of versions (but IMHO, they should only support fetching the latest revision (as go get); the user can always cd into the vendor directory and git checkout at a specific revision.

@perillo

This comment has been minimized.

Show comment
Hide comment
@perillo

perillo Nov 27, 2015

s/required/specified/

perillo commented Nov 27, 2015

s/required/specified/

@Merovius

This comment has been minimized.

Show comment
Hide comment
@Merovius

Merovius Nov 27, 2015

@perillo But that isn't really an argument now, is it? SemVer does exactly zero to make that situation better. Just use the revision you used before, develop against that and if there were breakages, you need to figure that out either way. I don't understand what you are getting at with that.

@peterbourgon Note, that I started not by demanding that this should be abolished (though I think it should), but by stating that someone suggesting SemVer should also give a definition of what that means. I firmly stand by this, I will be unhappy, but can accept SemVer as an 80% solution, but only if we collectively agree what kinds of changes will require an increment of which version number. Because a simple definition leads to counterproductive version numbers.

@perillo But that isn't really an argument now, is it? SemVer does exactly zero to make that situation better. Just use the revision you used before, develop against that and if there were breakages, you need to figure that out either way. I don't understand what you are getting at with that.

@peterbourgon Note, that I started not by demanding that this should be abolished (though I think it should), but by stating that someone suggesting SemVer should also give a definition of what that means. I firmly stand by this, I will be unhappy, but can accept SemVer as an 80% solution, but only if we collectively agree what kinds of changes will require an increment of which version number. Because a simple definition leads to counterproductive version numbers.

@peterbourgon

This comment has been minimized.

Show comment
Hide comment
@peterbourgon

peterbourgon Nov 27, 2015

Member

@Merovius

someone suggesting SemVer should also give a definition of what that means . . . [I] can accept SemVer as an 80% solution, but only if we collectively agree what kinds of changes will require an increment of which version number

Good news, then: this is already defined to a commonly-accepted level of precision in the spec:

Given a version number MAJOR.MINOR.PATCH, increment the:

MAJOR version when you make incompatible API changes,
MINOR version when you add functionality in a backwards-compatible manner, and
PATCH version when you make backwards-compatible bug fixes.

Member

peterbourgon commented Nov 27, 2015

@Merovius

someone suggesting SemVer should also give a definition of what that means . . . [I] can accept SemVer as an 80% solution, but only if we collectively agree what kinds of changes will require an increment of which version number

Good news, then: this is already defined to a commonly-accepted level of precision in the spec:

Given a version number MAJOR.MINOR.PATCH, increment the:

MAJOR version when you make incompatible API changes,
MINOR version when you add functionality in a backwards-compatible manner, and
PATCH version when you make backwards-compatible bug fixes.

@Merovius

This comment has been minimized.

Show comment
Hide comment
@Merovius

Merovius Nov 27, 2015

@peterbourgon I don't understand why you'd think that's relevant to my points. I thought I made myself relatively clear before. I know and understand the SemVer spec, but if that's all the definition you are giv, then every API change in go is a major version increment. That is hardly a useful versioning scheme. To make SemVer useful for go, you need to define what "backwards-compatible" means, because that's less straightforward then you'd think and everyone seems to have a different opinion about that.

@peterbourgon I don't understand why you'd think that's relevant to my points. I thought I made myself relatively clear before. I know and understand the SemVer spec, but if that's all the definition you are giv, then every API change in go is a major version increment. That is hardly a useful versioning scheme. To make SemVer useful for go, you need to define what "backwards-compatible" means, because that's less straightforward then you'd think and everyone seems to have a different opinion about that.

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Nov 28, 2015

Contributor

@Merovius thank you for your comments. I can see you have strong views on versioning, releasing, and methodologies like semver itself. Like others who have replied, we'll have to agree to disagree, and as SemVer is key to this proposal, I'm sorry but I cannot accept your suggestions into this proposal.

Contributor

davecheney commented Nov 28, 2015

@Merovius thank you for your comments. I can see you have strong views on versioning, releasing, and methodologies like semver itself. Like others who have replied, we'll have to agree to disagree, and as SemVer is key to this proposal, I'm sorry but I cannot accept your suggestions into this proposal.

@Merovius

This comment has been minimized.

Show comment
Hide comment
@Merovius

Merovius Nov 29, 2015

To repeat myself again: My suggestion was primarily to at least define what constitutes a "breaking change", or rather a "non-breaking change". I don't understand why this should be impossible or conflict with the target of using Semantic versioning, in fact I would think that it would promote the introduction of SemVer. I have strong opinions on the merits of SemVer, yes, but I agreed to disagree on those from the very beginning. But so far no one has even attempted to give this definition, acknowledge it's necessity or to give a reason to not give such a definition. Clarity on what constitutes a breakage seems like a hard requirement for introducing SemVer to me, because after all the spec references this term and only makes sense after this term is clear.

I am going to leave this discussion now, because I feel unwanted and ignored.

To repeat myself again: My suggestion was primarily to at least define what constitutes a "breaking change", or rather a "non-breaking change". I don't understand why this should be impossible or conflict with the target of using Semantic versioning, in fact I would think that it would promote the introduction of SemVer. I have strong opinions on the merits of SemVer, yes, but I agreed to disagree on those from the very beginning. But so far no one has even attempted to give this definition, acknowledge it's necessity or to give a reason to not give such a definition. Clarity on what constitutes a breakage seems like a hard requirement for introducing SemVer to me, because after all the spec references this term and only makes sense after this term is clear.

I am going to leave this discussion now, because I feel unwanted and ignored.

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Nov 29, 2015

Contributor

@Merovius I'm sorry you feel that your views are not being respected. If I have given that impression, please accept my apologies.

With respect to your specific concerns about what defines "breaking or non breaking change", I hold it is whatever Semver says, which is clearly stated at the top of the summary section of http://semver.org/spec/v2.0.0.html.

You argue that what constitutes a breaking change is much larger than what is commonly accepted, as pointed out by your well written article. As I said above, I agree completely with what you wrote, and see this as a point which tools can help as the rules for "breaking vs non breaking" are not ambiguous -- we already have this with the Go api tool, although i think you are arguing for some even more stringent, and that is also fine.

In summary, I agree with your points on breaking changes being almost any addition to an existing Go type (sorry if I have not reproduced the depth of your argument), but that seems to me to be tangental to this proposal. This proposal is to adopt some kind of release process -- if there is no release process then arguing about the nuances of the application of Semver to Go APIs serves no purpose.

Thanks

Dave

Contributor

davecheney commented Nov 29, 2015

@Merovius I'm sorry you feel that your views are not being respected. If I have given that impression, please accept my apologies.

With respect to your specific concerns about what defines "breaking or non breaking change", I hold it is whatever Semver says, which is clearly stated at the top of the summary section of http://semver.org/spec/v2.0.0.html.

You argue that what constitutes a breaking change is much larger than what is commonly accepted, as pointed out by your well written article. As I said above, I agree completely with what you wrote, and see this as a point which tools can help as the rules for "breaking vs non breaking" are not ambiguous -- we already have this with the Go api tool, although i think you are arguing for some even more stringent, and that is also fine.

In summary, I agree with your points on breaking changes being almost any addition to an existing Go type (sorry if I have not reproduced the depth of your argument), but that seems to me to be tangental to this proposal. This proposal is to adopt some kind of release process -- if there is no release process then arguing about the nuances of the application of Semver to Go APIs serves no purpose.

Thanks

Dave

@rsc

This comment has been minimized.

Show comment
Hide comment
@rsc

rsc Nov 30, 2015

Contributor

@Merovius, a few points.

  1. While it may be true that "every API change potentially breaks the build", that's not a terribly useful stance. For example, adding a method to an existing type is one kind of change; deleting that type is qualitatively different. It is helpful to distinguish these two, even if both are potential breaking changes.
  2. Go itself is using SemVer numbering (although we'd never heard of it when we started) to good effect. So if we need a formal spec about what constitutes a breaking change and what does not, then Go's own rules (https://golang.org/doc/go1compat) might be a decent start.
  3. Although the proposal does mention SemVer explicitly, it seems to me best not to get too hung up by the specific version numbering scheme. The reason this proposal is stalled is lack of a clear idea, assuming one can create a semantically useful versioning scheme, what you do next.
Contributor

rsc commented Nov 30, 2015

@Merovius, a few points.

  1. While it may be true that "every API change potentially breaks the build", that's not a terribly useful stance. For example, adding a method to an existing type is one kind of change; deleting that type is qualitatively different. It is helpful to distinguish these two, even if both are potential breaking changes.
  2. Go itself is using SemVer numbering (although we'd never heard of it when we started) to good effect. So if we need a formal spec about what constitutes a breaking change and what does not, then Go's own rules (https://golang.org/doc/go1compat) might be a decent start.
  3. Although the proposal does mention SemVer explicitly, it seems to me best not to get too hung up by the specific version numbering scheme. The reason this proposal is stalled is lack of a clear idea, assuming one can create a semantically useful versioning scheme, what you do next.
@mattfarina

This comment has been minimized.

Show comment
Hide comment
@mattfarina

mattfarina Dec 2, 2015

Member

For anyone who wants to back SemVer into their toolchains I wrote up a quick tutorial (with a package) for the Gopher Academy 2015 Advent

Member

mattfarina commented Dec 2, 2015

For anyone who wants to back SemVer into their toolchains I wrote up a quick tutorial (with a package) for the Gopher Academy 2015 Advent

@nathany

This comment has been minimized.

Show comment
Hide comment
@nathany

nathany Dec 2, 2015

Contributor

This proposal would ratify what many people and tools are already doing. Docker, Kubernetes, and CoreOS were mentioned. When I wrote about tagging repositories over two years ago, there were already third-party tools that worked with the v<SemVer> format.

The fsnotify library currently uses gopkg.in to turn tagged releases into new import paths for each major version. One feature gopkg.in provides is the distinction between released code and what's sitting on master. The url http://gopkg.in/fsnotify.v1 retrieves the latest v1.x.y tag. Code sitting on master while preparing a release is not made available until tagged. It also has experimental support for prerelease versions. Adding a v2.0.0-unstable tag would provide an opt-in way to retrieve an upcoming release.

While gopkg.in has been useful, I think it is quickly becoming unnecessary. Many people vendor code now, and GO15VENDOREXPERIMENT is further formalizing that recommendation. Everything gopkg.in provides can be replaced with tools such as Godep and gb _if_ they can retrieve the latest tag rather than master, or pull in prerelease versions or previous versions.

Gopkg.in is built on the idea of taking a common VCS practice of tagging releases and providing multiple import paths, derived from the recommendation in the Go FAQ. @davecheney has already listed the issues his colleagues at Canonical had with the multiple import path approach.

There are situations where creating an entirely new library with a new name is appropriate. But for bumping a major version to remove deprecated APIs, there are better alternatives to "create a new package with a new import path", and I believe the FAQ deserves an update.

Providing more visibility into "a release process for Go repositories" will allow third-party tools to start expecting v<SemVer> tags, using them to benefit their users.

Updating GoDoc.org to show documentation for any version may or may not be worth the effort involved. A more reasonable start would be to simply display the version number for the most current release. That will help make this recommendation to tag releases more visible. The distinction between tagged and untagged repositories could also be used to help ranking, by boosting released libraries higher than hobby projects.

What comes after that? I don't know the future, but I believe this is a good step, and one worth taking. Thanks @davecheney.

Contributor

nathany commented Dec 2, 2015

This proposal would ratify what many people and tools are already doing. Docker, Kubernetes, and CoreOS were mentioned. When I wrote about tagging repositories over two years ago, there were already third-party tools that worked with the v<SemVer> format.

The fsnotify library currently uses gopkg.in to turn tagged releases into new import paths for each major version. One feature gopkg.in provides is the distinction between released code and what's sitting on master. The url http://gopkg.in/fsnotify.v1 retrieves the latest v1.x.y tag. Code sitting on master while preparing a release is not made available until tagged. It also has experimental support for prerelease versions. Adding a v2.0.0-unstable tag would provide an opt-in way to retrieve an upcoming release.

While gopkg.in has been useful, I think it is quickly becoming unnecessary. Many people vendor code now, and GO15VENDOREXPERIMENT is further formalizing that recommendation. Everything gopkg.in provides can be replaced with tools such as Godep and gb _if_ they can retrieve the latest tag rather than master, or pull in prerelease versions or previous versions.

Gopkg.in is built on the idea of taking a common VCS practice of tagging releases and providing multiple import paths, derived from the recommendation in the Go FAQ. @davecheney has already listed the issues his colleagues at Canonical had with the multiple import path approach.

There are situations where creating an entirely new library with a new name is appropriate. But for bumping a major version to remove deprecated APIs, there are better alternatives to "create a new package with a new import path", and I believe the FAQ deserves an update.

Providing more visibility into "a release process for Go repositories" will allow third-party tools to start expecting v<SemVer> tags, using them to benefit their users.

Updating GoDoc.org to show documentation for any version may or may not be worth the effort involved. A more reasonable start would be to simply display the version number for the most current release. That will help make this recommendation to tag releases more visible. The distinction between tagged and untagged repositories could also be used to help ranking, by boosting released libraries higher than hobby projects.

What comes after that? I don't know the future, but I believe this is a good step, and one worth taking. Thanks @davecheney.

@mattfarina

This comment has been minimized.

Show comment
Hide comment
@mattfarina

mattfarina Dec 2, 2015

Member

@nathany If I understand what @rsc is say the issue is that adding SemVer as a recommendation without having part of the go toolchain using it is the issue. I'm not sure we'd find agreement on the right way to use SemVer in the toolchain right now. I think we really want SemVer so we can start to use it in tools and figure out directions it can be useful. So we can leverage versions like other communities.

If community things that aren't going to be built directly into the tools aren't appropriate for this forum do we need something similar to the PHP FIG for the community?

Member

mattfarina commented Dec 2, 2015

@nathany If I understand what @rsc is say the issue is that adding SemVer as a recommendation without having part of the go toolchain using it is the issue. I'm not sure we'd find agreement on the right way to use SemVer in the toolchain right now. I think we really want SemVer so we can start to use it in tools and figure out directions it can be useful. So we can leverage versions like other communities.

If community things that aren't going to be built directly into the tools aren't appropriate for this forum do we need something similar to the PHP FIG for the community?

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Dec 2, 2015

Contributor

If community things that aren't going to be built directly into the tools aren't appropriate for this forum do we need something similar to the PHP FIG for the community?

That sounds well meaning, but entirely too much bureaucracy. How about people just tag their releases ?

Contributor

davecheney commented Dec 2, 2015

If community things that aren't going to be built directly into the tools aren't appropriate for this forum do we need something similar to the PHP FIG for the community?

That sounds well meaning, but entirely too much bureaucracy. How about people just tag their releases ?

@rsc

This comment has been minimized.

Show comment
Hide comment
@rsc

rsc Dec 2, 2015

Contributor

Yes, my objection is that this proposal does not propose any actual changes to any part of Go. It is not actionable in its current form. I'm still not even sure it's a good idea, in large part because one way I evaluate whether something is a good idea is to understand its effects, and this proposal has no direct effects.

Build a system that uses semantic versions effectively. Demonstrate what great tools can be built. Give people incentives, and they will tag their releases no matter what the Go team says. Certainly the Go team saying "you should do it this way" has not in the past been successful without these things (think not-vendoring).

Contributor

rsc commented Dec 2, 2015

Yes, my objection is that this proposal does not propose any actual changes to any part of Go. It is not actionable in its current form. I'm still not even sure it's a good idea, in large part because one way I evaluate whether something is a good idea is to understand its effects, and this proposal has no direct effects.

Build a system that uses semantic versions effectively. Demonstrate what great tools can be built. Give people incentives, and they will tag their releases no matter what the Go team says. Certainly the Go team saying "you should do it this way" has not in the past been successful without these things (think not-vendoring).

@davecheney

This comment has been minimized.

Show comment
Hide comment
@davecheney

davecheney Dec 3, 2015

Contributor

I feel that it is time to draw the discussion to a close. The proposal guidelines clearly state "a lack of agreement means [the proposal is] declined", as such it is appropriate that I withdraw this proposal.

I want to extend my sincere thanks to all who have contributed to this debate. I remain passionate about improving the tools Go programmers have to manage package dependencies in large Go projects and your feedback has been invaluable to me and my future efforts.

Thank you again

Dave

Contributor

davecheney commented Dec 3, 2015

I feel that it is time to draw the discussion to a close. The proposal guidelines clearly state "a lack of agreement means [the proposal is] declined", as such it is appropriate that I withdraw this proposal.

I want to extend my sincere thanks to all who have contributed to this debate. I remain passionate about improving the tools Go programmers have to manage package dependencies in large Go projects and your feedback has been invaluable to me and my future efforts.

Thank you again

Dave

@davecheney davecheney closed this Dec 3, 2015

@rsc

This comment has been minimized.

Show comment
Hide comment
@rsc

rsc Dec 3, 2015

Contributor

Fair enough Dave. Thanks for the time you spent on the proposal. I certainly agree that a solution to versioning would be great, and that SemVer seems like a very good basis for building such a solution. The devil is always in the details, of course.

Contributor

rsc commented Dec 3, 2015

Fair enough Dave. Thanks for the time you spent on the proposal. I certainly agree that a solution to versioning would be great, and that SemVer seems like a very good basis for building such a solution. The devil is always in the details, of course.

@sapioit

This comment has been minimized.

Show comment
Hide comment
@sapioit

sapioit Jul 5, 2016

For the Versioning Numbering Cheme I would, any day, choose Explicit Versioning insted of Semantic Versioning. Why? It's so simple the first four words from the title of the link explain it for most of the developers. And besides that, no more ambiguity caused by the "Major" and "Minor" fields of the version number. Here's the promissed link: Release.Breaking.Feature.Fix or why the Semantic Versioning should be replaced with Explicit Versioning as soon as possible

sapioit commented Jul 5, 2016

For the Versioning Numbering Cheme I would, any day, choose Explicit Versioning insted of Semantic Versioning. Why? It's so simple the first four words from the title of the link explain it for most of the developers. And besides that, no more ambiguity caused by the "Major" and "Minor" fields of the version number. Here's the promissed link: Release.Breaking.Feature.Fix or why the Semantic Versioning should be replaced with Explicit Versioning as soon as possible

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.