Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: allow go-import <meta> tags to specify a branch #10913

Open
alanconway opened this issue May 19, 2015 · 25 comments

Comments

Projects
None yet
9 participants
@alanconway
Copy link

commented May 19, 2015

Allow meta tags in 'go get' to specify a branch. Suggested syntax from discussion at https://groups.google.com/d/msg/golang-dev/SW8r9ODYQf0/kqofheawGWYJ

<meta name="go-import" content="qpid.apache.org/proton git https://git-wip-us.apache.org/repos/asf/qpid-proton.git/proton-c/bindings/go branch=go1">

Rationale: there are several reasons why you might want go get to retrieve from a branch that is not "master", "trunk" or the default branch for a repository.

  • master/trunk usually used for development, a project may want direct "go get" to get the latest stable release or snapshot on a branch or tag.
  • make "experimental" work available from a branch before it is considered stable/mature enough to move to the project's main branch.
  • some repos have odd branch naming conventions for historical reasons (e.g. git repos that were converted from SVN and still use "trunk" instead of "master")
@bradfitz

This comment has been minimized.

Copy link
Member

commented May 19, 2015

/cc @adg

@minux minux changed the title Allow go-import <meta> tags to specify a branch cmd/go: allow go-import <meta> tags to specify a branch May 19, 2015

@ianlancetaylor ianlancetaylor added this to the Go1.6 milestone Jun 3, 2015

@rsc

This comment has been minimized.

Copy link
Contributor

commented Nov 23, 2015

I wonder if it should be rev= instead of branch=, so that other non-branch identifiers can be used too. That said, too late for Go 1.6. Need more details. Perhaps the right next step is to write a proposal.

@rsc rsc modified the milestones: Unplanned, Go1.6 Nov 23, 2015

@alanconway

This comment has been minimized.

Copy link
Author

commented Nov 23, 2015

rev= makes sense to me, released labels could be useful as well as branches.

@elithrar

This comment has been minimized.

Copy link

commented Dec 28, 2015

I'd be interested in picking this up—which includes writing the proposal—but want to hash out the design a little more and am seeking feedback:

Q. How should the revision/branch/tag support work? Ideally we want to support all three forms but we will need ways to identify which form the go-imports tag is referring to.
Proposed Answer: <meta name="go-import" content="import-prefix vcs repo-root repo-rev"> where repo-rev is of the form branch=$name, rev=$commit or tag=$tag.

Q: What does backward compatibility look like here?
Proposed Answer: Because older versions of the Go tool will not be aware of the additional tag, they will likely (TODO: check this) pull from HEAD instead of the specified rev. This implicit behaviour will be confusing to package consumers as a package author might set up the import URL as getattest.io/pkgname tag=v2, but the consumer won't get this revision and might wonder why their program is breaking or otherwise different.

Should we fail outright here? Rely on // +build go1.7 in the package itself to prevent the behaviour (has other ramifications)? This needs more thought. One solution may be to re-order the go-imports format to force this failure—e.g. import-prefix vcs repo-rev repo-root—at the risk of a confusing error message from earlier versions of go get and other tools.

There are likely other questions here as well, but I wanted to get these down to get the discussion moving.

@alanconway

This comment has been minimized.

Copy link
Author

commented Dec 29, 2015

Q1. For git I would say rev=xxx (or commit=xxx) is sufficient for repo-rev. The git tools that take a commit will accept a branch, tag, hash or any other string that can name a commit, so I would expect this to do the same (i.e. I can say rev=mybranch, rev=mytag, rev=xxxxcommithash and all will work) I don't know if that is true for other repo types (mercurial, bzr etc.) or if they need the extra info to interpret the string as a branch/tag/etc. Possibly this needs to be different to follow the normal syntax for different repo types?

Q2. Fail outright. Anyone who needs backwards compat will have to set things up to work the old way anyway (HEAD or the magic go1 branch), so there's no point in them using the new feature. If they use the new feature it is safe to assume they have not intentionally set things up to work the old way, and if it appears to work by accident Bad Things will probably happen.

Perhaps a safe way to ensure a clean failure would be use <meta name="go-import-rev">, so old go won't even find the new directive. That would also allow you to set up both "go-import" for old go and "go-import-rev" for new go, e.g. go-import might point to a different repo that has the proper branch cloned as master for backwards compat.

@rsc

This comment has been minimized.

Copy link
Contributor

commented Dec 29, 2015

Q1. If you say rev= then that's supposed to work for any code identifier. You don't have to say branch= sometimes or tag= other times, just like you don't have to tell, say, git checkout what kind of argument you are giving it.

Q2. No, older versions of the go command will ignore the tag entirely, resulting in not being able to resolve the reference. That's fine. If you need to specify a branch, then you can't speak to them.

@elithrar

This comment has been minimized.

Copy link

commented Dec 29, 2015

Thanks for the responses @alanconway and @rsc.

The rev/branch/tag suggestion was primarily for compatibility across VCS', but since git, Mercurial, bazaar and Subversion all just take a revision of any kind it's unnecessary (as Russ points out).

I'll look to push a patch to support an updated go-import tag of the form <meta name="go-import" content="import-prefix vcs repo-root repo-rev"> when I have some free time (moving continents).

@alanconway

This comment has been minimized.

Copy link
Author

commented Dec 29, 2015

Q2: I believe there will be some set of go versions that does understand <meta name="go-import" but does not understand the new rev= Since rev= implies the author does not want you to pull master it would be bad if they simply ignored the rev= and pulled master anyway. Some kind of failure is called for in that case I think.

@elithrar

This comment has been minimized.

Copy link

commented Dec 29, 2015

I think my original proposal to just re-order the fields: repo-rev before
the URL - would be sufficient to prevent that.

I'll see how the current tool parses the meta tag when I'm in front of a PC.
On Wed, 30 Dec 2015 at 5:16 AM, alanconway notifications@github.com wrote:

Q2: I believe there will be some set of go versions that does understand <meta
name="go-import" but does not understand the new rev= Since rev= implies
the author does not want you to pull master it would be bad if they
simply ignored the rev= and pulled master anyway. Some kind of failure is
called for in that case I think.


Reply to this email directly or view it on GitHub
#10913 (comment).

@rsc

This comment has been minimized.

Copy link
Contributor

commented Jan 4, 2016

@alanconway, @elithrar regarding Q2 (again):

Q2. No, older versions of the go command will ignore the tag entirely, resulting in not being able to resolve the reference. That's fine. If you need to specify a branch, then you can't speak to them.

They ignore the meta tag because it has the wrong number of fields. See the source code (src/cmd/go/discovery.go):

    if f := strings.Fields(attrValue(e.Attr, "content")); len(f) == 3 {
        imports = append(imports, metaImport{
            Prefix:   f[0],
            VCS:      f[1],
            RepoRoot: f[2],
        })
    }
@duzy

This comment has been minimized.

Copy link

commented Oct 22, 2017

You may try editing the HEAD ref. Or just use godev.io or gopkg.in. Where godev.io allows you to select any revision of commit/branch/tag/vX.Y.Z , but gopkg.in is restricted to vX.Y.Z

Examples:

go get godev.io/golang/go.master

Or

go get gopkg.in/golang/go.v1.9

Such revision selection is very useful, unfortunately go tool don't allow such usage. It uses the vendor approach. So things like godev.io and gopkg.in came to help.

@bcmills

This comment has been minimized.

Copy link
Member

commented Jan 23, 2019

Compare #26964, which proposes to put the branch information in the go.mod file rather than the meta tag.

@bcmills

This comment has been minimized.

Copy link
Member

commented Jan 23, 2019

Looking at the use-cases again in light of modules:

  • master/trunk usually used for development, a project may want direct "go get" to get the latest stable release or snapshot on a branch or tag.

In module mode, if the repository is tagged using v-prefixed semantic versions then go get -u will get the latest stable release. go get with a branch obtains the latest commit on that branch. go get with a tag or commit hash obtains that commit as long as it is reachable from any branch.

  • make "experimental" work available from a branch before it is considered stable/mature enough to move to the project's main branch.

Modules support both branches and pre-release tags for this purpose.

  • some repos have odd branch naming conventions for historical reasons (e.g. git repos that were converted from SVN and still use "trunk" instead of "master")

This one isn't obvious to me. Is there anything we need to do to make these repos work more smoothly in module mode?

@bcmills

This comment has been minimized.

Copy link
Member

commented Jan 23, 2019

Is there anything we need to do to make these repos work more smoothly in module mode?

That seems like the only remaining question: if not, then I think we can close this issue in favor of #26964.

@alanconway

This comment has been minimized.

Copy link
Author

commented Jan 23, 2019

@zx2c4

This comment has been minimized.

Copy link
Contributor

commented Mar 3, 2019

Currently go get resolution uses whatever the default HEAD of git is. This is extremely limited, and has lead to all sorts of horrific hacks, such as gopkg.in having to proxy the git-http protocol itself in order to send a hacked HEAD for go to fetch.

The reasonable and straightforward way of solving this is by allowing go-import meta tags to specify a branch in the git repo it points to. This way various APIs and programmatic path names could be setup to point to different branches.

This old request here remains very valid. Using go.mod is not a solution to the problem faced here. There might be other merits in that approach for other use cases, but it doesn't account for the significant usefulness of being able to programmatically point a go package's name at arbitrary branches, regardless of how somebody has wired up their particular go.mod.

@alanconway

This comment has been minimized.

Copy link
Author

commented Mar 4, 2019

@zx2c4 is right, silly me - updating a .mod file serves no purpose unless 'go get' can download the .mod file, which requires a revision, which gets back to the original question.

I've been maintaining a project exported using the 'go1' branch for some time now and it works fine - it's just a bit obscure that a branch called "go1" is magically the exported latest-stable code. If it was just called "go-get-go1", and could be a tag as well as a branch, I think I would never have raised this issue.

@bcmills

This comment has been minimized.

Copy link
Member

commented Mar 4, 2019

If the user's go.mod file requires a valid version, then go get will fetch that version regardless of what branch it is on. If the user wants go get -u to track a specific branch, that is arguably up to that user (that's #26964).

If the user runs go get -u or go get $PKG@latest, then they will receive the version from the most recent release tag regardless of what branch it is on: so all you as a maintainer need to do to point go get -u to a particular branch is to apply your release tags to that branch.

@alanconway

This comment has been minimized.

Copy link
Author

commented Mar 4, 2019

@bcmills

This comment has been minimized.

Copy link
Member

commented Mar 4, 2019

The issue (as I raised it) is about how the package maintainer chooses the
code that will be downloaded by default "go get" - the latest stable
release.

In Go 1.13, the default go get will be in module mode — see #30228. (We don't plan to make further improvements to GOPATH mode.)

go get in module mode chooses the latest release tag by default, regardless of the branch on which it appears. So you will indeed be able to “just tag the release and Go.”

@zx2c4

This comment has been minimized.

Copy link
Contributor

commented Mar 4, 2019

go get in module mode chooses the latest release tag by default, regardless of the branch on which it appears.

It'd be much preferable to restrict this to the default branch, and then allow the go-import meta tag to specify which branch is pointed at by a particular package name. Otherwise git becomes significantly gimped, as there's no way to put various things in different branches and hope there will be something coherent out of it.

@bcmills

This comment has been minimized.

Copy link
Member

commented Mar 4, 2019

Not every provider of modules is a git repository: notably, servers can vend module zip files directly, just as module proxies do.

If the packages are in separate subdirectories, you can always put the go.mod file within the subdirectory and include the subdirectory as a prefix on the version tags. But I'm not sure what value you'd get from branches at that point.

See previously #26664.

@zx2c4

This comment has been minimized.

Copy link
Contributor

commented Mar 4, 2019

Not every provider of modules is a git repository:

Yes, of course. But that doesn't change the fact that supporting git branches would be very useful.

If the packages are in separate subdirectories, you can always put the go.mod file within the subdirectory and include the subdirectory as a prefix on the version tags.

Sounds like a better use case for branches, so that potentially totally different histories are not intermingled.

@alanconway

This comment has been minimized.

Copy link
Author

commented Mar 5, 2019

@gdamore

This comment has been minimized.

Copy link

commented Mar 7, 2019

See also #30647 and nanomsg/mangos#78 for some of the grief this causes. The go modules approach, which basically turns some paths into magical (the semver stuff), is breaking without any kind of escape hatch for folks who don't want to adopt modules. Go 1 promise busted. The way to get back is to offer something like proposed here so that it can be possible for existing repos to support semver, vanity names, and both legacy go get and go modules.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.