Libraries and gb

[elithrar]

Providing functionality to support library authors (vendoring deps) and library users (building with those deps) would be very useful, and would cover part of the use-case that tools/godep does without the import path re-writing.

• A user can go get github.com/you/yourpackage and pull down the master.
• They can then gb build to use the vendored packages under /vendor instead of their global GOPATH
• Else they can fall back to go build and go install and build off their global GOPATH. This (obviously) may not work, but we can't change the behaviour of go get. This is no different from how things work now with go get and pulling deps from head.
• There may also be benefit in storing the commits/tags in some kind of .gbvendors file (the name is unimportant right now!) as well that makes it clear what revisions were pulled in. Avoiding submodules/subtrees is great (IMO) but knowing what revision the vendored dependency is pinned at still has value.

My library would then look like:

github.com/me/mylib
\
 |- auth.go
 |- auth_test.go
 |- doc.go
 |- vendor/
 \
  |-- .gbvendors
  |-- github.com/someother/somepkg/
  |-- github.com/another/anotherpkg

[nathany]

Flattening the vendor/ folder of a library to that of the end-of-line project vendor/ folder seems easy enough.

The challenge is conflicts. With DVCS there would be a three-way merge and conflict resolution, but if the history is removed when vendoring, what then? (other than fail with an error)

[nathany]

Flattening the vendor/ folder of a library to that of the end-of-line project vendor/ folder seems easy enough.

The challenge is conflicts. With DVCS there would be a three-way merge and conflict resolution, but if the history is removed when vendoring, what then? (other than fail with an error)

[davecheney]

@nathany

Flattening the vendor/ folder of a library to that of the end-of-line project vendor/ folder seems easy enough.

I believe this to be out of scope for gb for the time being (maybe it's implemented in a plugin, but that is also out of scope).

This is for two reasons.

The first is gb tries to avoid boiling all the oceans at once and tackles the problem of how to provide reliable builds by providing a project based workflow; a single repository that contains all the source (neatly divided into yours, src/, and theirs vendor/src) but remains mute on the way that this project is constructed.

The most obvious way to build a gb project is to copy all of the dependencies your project requires into vendor/src/ and then check the whole shebang in. More ambitious users may attempt to use svn externals or git subthingies to automate this process.

The challenge is conflicts. With DVCS there would be a three-way merge and conflict resolution, but if the history is removed when vendoring, what then? (other than fail with an error)

I view this problem as insoluable with the tools available to us today. @peterbourgon astutely noted that the Go dependency problem is one of space; the package's import path, and time; the specific revision or copy of a package at a point in time.

Go packages do not have versions, at least not in the way that ruby gems have versions, or npm packages have versions, etc. The source of Go packages may be backed by a DVCS and you may be able to ask that DVCS for different revisions of a package, representing a set of changes over time. This may appear to be splitting hairs, but I think it is a crucial distinction.

The problem of managing the diamond dependency problem is, for the moment, entirely the domain of the project owner. gb provides a vessel to hold the source of their project, but remains mute on how or what source is selected to be placed inside a project.

In the future, maybe a plugin can manage the specific source inside vendor/src/, but it is not within the current scope of what I am planning to achieve.

[davecheney]

@nathany

Flattening the vendor/ folder of a library to that of the end-of-line project vendor/ folder seems easy enough.

I believe this to be out of scope for gb for the time being (maybe it's implemented in a plugin, but that is also out of scope).

This is for two reasons.

The first is gb tries to avoid boiling all the oceans at once and tackles the problem of how to provide reliable builds by providing a project based workflow; a single repository that contains all the source (neatly divided into yours, src/, and theirs vendor/src) but remains mute on the way that this project is constructed.

The most obvious way to build a gb project is to copy all of the dependencies your project requires into vendor/src/ and then check the whole shebang in. More ambitious users may attempt to use svn externals or git subthingies to automate this process.

The challenge is conflicts. With DVCS there would be a three-way merge and conflict resolution, but if the history is removed when vendoring, what then? (other than fail with an error)

I view this problem as insoluable with the tools available to us today. @peterbourgon astutely noted that the Go dependency problem is one of space; the package's import path, and time; the specific revision or copy of a package at a point in time.

Go packages do not have versions, at least not in the way that ruby gems have versions, or npm packages have versions, etc. The source of Go packages may be backed by a DVCS and you may be able to ask that DVCS for different revisions of a package, representing a set of changes over time. This may appear to be splitting hairs, but I think it is a crucial distinction.

The problem of managing the diamond dependency problem is, for the moment, entirely the domain of the project owner. gb provides a vessel to hold the source of their project, but remains mute on how or what source is selected to be placed inside a project.

In the future, maybe a plugin can manage the specific source inside vendor/src/, but it is not within the current scope of what I am planning to achieve.

[davecheney]

@elithrar can I ask a clarifying question. I originally thought that this was a request to use gb for packages, that is libraries that don't contain a main package, effectively anything which is not defined as a gb project today. Is this the case, or is this a different request ?

[davecheney]

@elithrar can I ask a clarifying question. I originally thought that this was a request to use gb for packages, that is libraries that don't contain a main package, effectively anything which is not defined as a gb project today. Is this the case, or is this a different request ?

[elithrar]

@davecheney Nope, you're right: a request to use this for libraries — i.e. other than just the package main support we have today.

The suggestions are potential ways to implement it (and/or things to consider) but I'm certainly open to other ideas for library support.

[elithrar]

@davecheney Nope, you're right: a request to use this for libraries — i.e. other than just the package main support we have today.

The suggestions are potential ways to implement it (and/or things to consider) but I'm certainly open to other ideas for library support.

[davecheney]

@elithrar have a look at #49 and if you think I've captured things correctly maybe close this issue.

[davecheney]

@elithrar have a look at #49 and if you think I've captured things correctly maybe close this issue.

[elithrar]

@davecheney I think you covered it pretty succinctly in #49 actually—so happy to close this one.

[elithrar]

@davecheney I think you covered it pretty succinctly in #49 actually—so happy to close this one.