Skip to content

Use Godep for dependency management? #177

vito opened this Issue Mar 9, 2014 · 21 comments

7 participants

vito commented Mar 9, 2014

Godep is a pretty popular tool for managing/vendoring dependencies. I use it for large projects and it allows us to have 100% reproducible deployments with no external dependencies, as everything we need lives in our own repo.

This would make the repo a bit bigger, but in my experience it ends up being faster to pull down everything at once, than wait on every individual repo to clone (especially launchpad, ugh). And then you can deploy anywhere, even without internet access at deploy time.

I just ran godep save ./... and it weighs in at 8.3M, 5.2M of which is go-sqlite3.

As a developer, the workflow would be:

  1. when updating dependencies, run godep save ./...
  2. commit dependency changes separately so they don't clutter the actual commit's changes: git add -A Godeps; ...
  3. commit your changes
kirs commented Mar 11, 2014

I can also recommend the gom.

In my opinion, we really need the dependency manager for Drone.

rgarcia commented Mar 11, 2014

I'll note that with Godep you don't necessarily need to commit your dependencies into git. You can do a normal go get to pull dependencies into your GOPATH, and then godep restore to check out the correct version of your dependencies.

kirs commented Mar 12, 2014

The same for gom.

luxifer commented Mar 12, 2014

The fact is, you don't have to mess up with dependency management hell in go since the guidelines tells you to only break compatibility in major versions of your software and each new major version should be in a new repository. Like the naming of go itself go1.2-2.
If you really want to lock a dependency is to install it under the project root and commit it.
Also there are tools linked in the godoc faq such as goven

Read and

vito commented Mar 13, 2014

@luxifer Yep, that's what I've heard, but I've never seen anyone model breaking changes by making new repositories.

In practice, every dependency is a moving target (including the site they're hosted on), and vendoring them is a safe way of keeping them in check, and making your application braindead easy to distribute and deploy and test.

luxifer commented Mar 13, 2014

Yup of course. That's why I mentioned goven. In production project you have to have a complete control.
The fact is it's the developer job to not introduce BC breaks.


Continuation from #237 regarding dependencies management.
Since it may take a while before getting plouc/go-gitlab-client#8 reviewed, I was proposing to get
this package vendored within drone repository, at first I was thinking to use docker's approach:

But then I remember we've been discussing about this when @bradrydzewski mentioned about godep. I haven't use any of the existing dependencies management, so I think it would be best if we continue the discussion here.

I've been stumped by this error when trying to use godep for drone:

godep: open /home/adie/go/src/ no such file or directory
godep: read /home/adie/go/src/ is a directory
godep: open /home/adie/go/src/ no such file or directory
godep: open /home/adie/go/src/ permission denied
godep: open /home/adie/go/src/ permission denied
godep: error copying source code

Haven't actually try gom, but since it looks familiar with ruby's bundler and its Gemfile, I'll bet for gom. Though it will be nice if gom could do something like this:

gom '', :git => '', :branch => 'feature/patch'

From the snippet above we can use bbb/go-awesome-pkg without changing the original import path from aaa/go-awesome-pkg

my 2 cents, I'm okay with anything that works. :D


vito commented Apr 4, 2014

@fudanchii Make sure you have the latest godep and try again; there was a bug that prevented it from dealing with symlinks properly (the docker repo has some for test fixtures).

Gom seems weird to me; unsure why they bothered implementing Ruby-style syntax. The nice thing about Godep is there's no extra 'versions' file to maintain; it just vendors it and saves off which shas/tags/branches it vendored.


+1 for gpm or gom


@vito Yup, seems that's the case. Thanks for the pointer, it works like a charm!

Gom seems weird to me; unsure why they bothered implementing Ruby-style syntax.

I think I'm a bit biased since I'm doing some ruby. :D

Anyway, I would like to know if either gom or gopm can copy package from the original $GOPATH to its vendor directory like godep does? Seems like gom and gopm will just either skip existing package or re-clone them from its original repository.

Since currently we need to vendor fudanchii/go-gitlab-client in place of plouc/go-gitlab-client, with godep I can just checkout my branch from plouc/go-gitlab-client, call godep save ./... and commit the Godeps folder.

Any idea how to do something like this with gom or gopm? Also it looks like that gom is supposed to be run in the main executable folder. It's a bit bloated for drone if we end up with two Gomfiles and two _vendor directories.


@bradrydzewski member

@fudanchii we could import in Drone and then change the import to once your pull request has been merged.

It looks like gom and godep seem to have the most traction (at the moment). We can always change if a particular solution ends up becoming the defacto standard in the Go community.

I agree the ruby syntax used by gom is a little awkward.

I'm fine with godep if someone wants to submit a patch.

@bradrydzewski member

Also, we don't need to commit the dependencies to the repository, as @rgarcia mentioned. We can use -copy=false and run godep restore to clone our dependencies locally.

vito commented Apr 5, 2014

My $.02: it'd really be best to vendor the dependencies (whether or not we go with Godep). Because it's not checking in binaries it's actually pretty efficient, and faster than re-cloning dependencies, as it vendors only the package source that's used (not entire repositories). It also keeps distribution from requiring hg/bzr. godep restore is harmful; it'll go out and fetch the repos and leave all of the clones in your $GOPATH with detached heads.

@bradrydzewski member

I didn't realize that was the behavior of godep resotre. It seems to be a poor design decision. I figured it would clone to your project's Goeps/_workspace folder.

I'd prefer to avoid traditional vendoring since we would need to go through our code and change all the import paths.

If you submit a PR I'll accept.

vito commented Apr 5, 2014

To clarify, this is the workflow I've adopted around vendoring:

Given application A and dependent library X,

  1. Configure CI/automated deploys to put component A in a pristine $GOPATH and prepend its Godeps/_workspace to $GOPATH (example).
  2. Work away at A out of one big $GOPATH, often working on X at the same time
  3. When done working on X, commit (and optionally push).
  4. When I'm at a good place, godep save ./... in A, run my tests, and commit that, and then commit my actual code change (so the commit isn't too noisy).

So, all the import paths stay the same, I still work out of one $GOPATH, and the only times I think about vendored dependencies are during godep save for updating them, and one-time configuration in CI to add them to the $GOPATH.

I'll put together a PR.

@bradrydzewski member

thanks for offering to put a PR together!

Quick question ... doesn't godep handle altering the $GOAPTH?

Which is why I would need to execute godep go test? Sorry if I'm asking obvious questions, I really haven't used any of these tools, and am just trying to get a basic understanding of how they work.

vito commented Apr 5, 2014

It's not required; all that's doing is literally GOPATH=$(godep path):$GOPATH go .... In drone's case we could just run godep go test but setting $GOPATH manually is the more general solution, since there's no godep exec yet.

@bradrydzewski member

I feel like I'm being a pain here, but what would godep exec do?

I just don't want to have to remember to manually alter my $GOAPTH every time I start a new terminal session. I also want to be able to run go test (or godep go test) on a specific package, or specific unit test, without haven't to use a shell script or makefile that runs all my tests.

Is that how it would work?

vito commented Apr 5, 2014

No worries; pretty important to understand the nitty-gritty.

godep exec would be a generalized command that runs a command string with the env all set up for it, similar to bundle exec.

You'll never have to modify your $GOPATH; as a developer you should pretend Godeps/_workspace isn't even there after you bump + commit it. You'll still be working just like you do now, only you'll be vendoring the dependencies for CI/distribution whenever you change them.

Tweaking your $GOPATH or using godep go ... or godep exec ... is only important for CI or build systems that actually need the locked-down dependencies.

@bradrydzewski member

thanks for the clarification :)

vito commented Apr 5, 2014

Submitted #243 - had to submodule go.rice as nothing actually imports the executable that we depend on, so godep save ./... didn't pick it up. Making it a submodule allows it to find it.

@vito vito closed this Apr 6, 2014
@tobegit3hub tobegit3hub referenced this issue in docker/docker Nov 2, 2014

Use Godep for dependency management #8913

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.