manifest.json is empty after dep init #149

Open
dcelasun opened this Issue Jan 25, 2017 · 16 comments

Projects

None yet

7 participants

@dcelasun

I was asked to open this issue in this thread.

After running dep init with an existing project (located at $GOPATH/src/bitbucket.org/company_name/project_name), manifest.json has an empty JSON object ({}), but lock.json has all the correct dependencies.

$ go version
go version go1.7.4 linux/amd64
$ go env
GOARCH="amd64"
GOBIN="/home/dcelasun/go/bin"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/dcelasun/go"
GORACE=""
GOROOT="/usr/lib/go"
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build159479626=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"

I can provide more info if needed.

@disq
disq commented Jan 25, 2017 edited

Same thing here, project located at $GOPATH/src/github.com/company/project
Retried after rm -rf vendor, same results. lock.json looks cool and correct but manifest.json has just {} in it.

$ go version
go version go1.7 darwin/amd64
$ go end
go: unknown subcommand "end"
Run 'go help' for usage.
$ go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/kemal/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/81/jzjhd4m15sq1cj31pt52f0gh0000gn/T/go-build586013334=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
$
@pksunkara

Just confirming that I also hit this and wrote about it in the google group. I can verify that dep init command here is in migration mode according to the spec written down previously.

@raytung
Contributor
raytung commented Jan 25, 2017 edited

Are your dependencies under $GOPATH?
Looks like dep is only adding dependencies that exist in $GOPATH/src
See init.go#L302, context.go#L143, and context.go#L128

@peterbourgon peterbourgon added the bug label Jan 25, 2017
@pksunkara

Nope, I have an empty project with a fresh install of go. So, dependencies are nowhere to be found. Cloned https://github.com/pksunkara/whitespaces and ran dep init

@raytung
Contributor
raytung commented Jan 25, 2017 edited

@pksunkara Yep, that's it I think! It wouldn't save into the manifest.json file if the source doesn't already exist under $GOPATH/src.

Steps to reproduce:

  • Remove all dependencies from $GOPATH/src
  • Run dep init

EDIT: Removed workaround - not relevant to this convo

@pksunkara

Well, dep init did pull the source into vendor repo, so, shouldn't it take that code after that step? Or shouldn't it take from github directly? I am not looking for a workaround. I am trying to determine how the tool should behave in this case.

I guess, first of all, we probably need to separate dep init and dep migrate out.

@jbrodriguez

It happened to me too.

So, I did:

  • dep init
    manifest.json: only dependencies that exist in $GOPATH/src
    lock.json: ok

  • dep ensure -update
    creates/populates vendor

  • mv vendor vendor.old
    (actually had to move it one folder up the hierarchy, otherwise the next dep ensure would get confused)
    This simulates leaving vendor out of source control, then cloning the repo back

  • dep ensure -update
    recreates vendor correctly

  • diff -r vendor vendor.old
    no differences

so, what exactly is the purpose of manifest ?

@pksunkara

This analogy with npm helps me remember.

package.json => manifest.json
npm-shrinkwrap.json => lock.json
node_modules => vendor

@jbrodriguez

Right, but package.json does hold all dependencies.

I know dep is pre-alpha, just wondering if manifest.json will also hold all dependencies as it matures.

@pksunkara

Yeah, according to the design spec I read previously, it is supposed to.

@sdboyer
Member
sdboyer commented Jan 27, 2017

(sorry, I've not had time to get to this one yet)

I know dep is pre-alpha, just wondering if manifest.json will also hold all dependencies as it matures.

Standard constraint declarations in manifest.json (under the dependencies key) are only applied to direct dependencies of the current project. It's important to understand, though, that if you declare a constraint in manifest.json, but don't actually import it, that constraint will be ignored. This is still Go - the import graph is queen. (If you want to force something to show up, even if it's not imported, then you want a require. There's no command for that yet, but you can write them into the manifest directly and they work fine ๐Ÿ˜„ )

Overrides are applied globally, superceding constraints declared by all dependencies (including any dependencies in your own manifest.json, if you did a weird thing and have both). lock.json holds a transitively complete list of all deps.

If you're accustomed to npm, especially older versions (pre-flattening), this all works a bit differently because dep doesn't allow duplication of dependencies.

We don't really have docs optimized for end users yet (pre-alpha, sorry!), but there's more info about the mechanics in the gps docs.

@pksunkara

Ah, my above statement was ambiguous. I meant that according to the spec manifest.json was supposed to hold the top level dependencies and any other overrides. Sorry about the wording "all dependencies"

@jbrodriguez

Standard constraint declarations in manifest.json (under the dependencies key) are only applied to direct dependencies of the current project

Sure, that's the most important thing. As it stands right now, this is not fulfilled, as it expects these direct dependencies to exist in $GOPATH/src.

In any case, the project is awesome and I feel it's going in the right direction, so I'll keep trying to poke holes in it (in a very friendly manner :) ๐Ÿ‘

@sdboyer
Member
sdboyer commented Jan 27, 2017

In any case, the project is awesome and I feel it's going in the right direction, so I'll keep trying to poke holes in it (in a very friendly manner :)

Thanks! please do :) ๐ŸŽ‰

As it stands right now, this is not fulfilled, as it expects these direct dependencies to exist in $GOPATH/src.

mm, that's not the case. The only part of dep that touches GOPATH, at all, is dep init - and it only does so to scan what you have on your GOPATH in order to set initial constraints, which it then records in manifest.json. However, if any of those direct (or transitive) dependencies are missing, it'll reach out and try to make a default decision on what version to use by picking the "newest", as defined by the ordering rules. So, if you run dep init on a project in an otherwise empty GOPATH, it'll reach out for everything.

Of course, this issue is about cases in which that fails, and for some reason the resulting manifest is empty...and that's a bug, and I'm not sure why it's doing it. But if you're seeing behavior that makes it seem like dep is in any way requiring anything to be on the GOPATH, please please, do explain.

@sdboyer
Member
sdboyer commented Jan 27, 2017

@raytung

Yep, that's it I think! It wouldn't save into the manifest.json file if the source doesn't already exist under $GOPATH/src.

Oh wait...maybe this is it. If this is all it is, then it's a super-easy fix. I'll look tonight :)

@sdboyer
Member
sdboyer commented Jan 28, 2017

Ah, so, it seems like we actually did this intentionally for the first round with init - if it's not on disk, we don't put a constraint in the manifest. So, I want to check in with @adg before we go changing that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment