New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

manifest.json is empty after dep init #149

Closed
dcelasun opened this Issue Jan 25, 2017 · 21 comments

Comments

Projects
None yet
8 participants
@dcelasun
Copy link

dcelasun commented Jan 25, 2017

I was asked to open this issue in this thread.

After running dep init with an existing project (located at $GOPATH/src/bitbucket.org/company_name/project_name), manifest.json has an empty JSON object ({}), but lock.json has all the correct dependencies.

$ go version
go version go1.7.4 linux/amd64
$ go env
GOARCH="amd64"
GOBIN="/home/dcelasun/go/bin"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/dcelasun/go"
GORACE=""
GOROOT="/usr/lib/go"
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build159479626=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"

I can provide more info if needed.

@disq

This comment has been minimized.

Copy link

disq commented Jan 25, 2017

Same thing here, project located at $GOPATH/src/github.com/company/project
Retried after rm -rf vendor, same results. lock.json looks cool and correct but manifest.json has just {} in it.

$ go version
go version go1.7 darwin/amd64
$ go end
go: unknown subcommand "end"
Run 'go help' for usage.
$ go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/kemal/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/81/jzjhd4m15sq1cj31pt52f0gh0000gn/T/go-build586013334=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
$
@pksunkara

This comment has been minimized.

Copy link

pksunkara commented Jan 25, 2017

Just confirming that I also hit this and wrote about it in the google group. I can verify that dep init command here is in migration mode according to the spec written down previously.

@raytung

This comment has been minimized.

Copy link
Contributor

raytung commented Jan 25, 2017

Are your dependencies under $GOPATH?
Looks like dep is only adding dependencies that exist in $GOPATH/src
See init.go#L302, context.go#L143, and context.go#L128

@peterbourgon peterbourgon added the bug label Jan 25, 2017

@pksunkara

This comment has been minimized.

Copy link

pksunkara commented Jan 25, 2017

Nope, I have an empty project with a fresh install of go. So, dependencies are nowhere to be found. Cloned https://github.com/pksunkara/whitespaces and ran dep init

@raytung

This comment has been minimized.

Copy link
Contributor

raytung commented Jan 25, 2017

@pksunkara Yep, that's it I think! It wouldn't save into the manifest.json file if the source doesn't already exist under $GOPATH/src.

Steps to reproduce:

  • Remove all dependencies from $GOPATH/src
  • Run dep init

EDIT: Removed workaround - not relevant to this convo

@pksunkara

This comment has been minimized.

Copy link

pksunkara commented Jan 25, 2017

Well, dep init did pull the source into vendor repo, so, shouldn't it take that code after that step? Or shouldn't it take from github directly? I am not looking for a workaround. I am trying to determine how the tool should behave in this case.

I guess, first of all, we probably need to separate dep init and dep migrate out.

@jbrodriguez

This comment has been minimized.

Copy link

jbrodriguez commented Jan 25, 2017

It happened to me too.

So, I did:

  • dep init
    manifest.json: only dependencies that exist in $GOPATH/src
    lock.json: ok

  • dep ensure -update
    creates/populates vendor

  • mv vendor vendor.old
    (actually had to move it one folder up the hierarchy, otherwise the next dep ensure would get confused)
    This simulates leaving vendor out of source control, then cloning the repo back

  • dep ensure -update
    recreates vendor correctly

  • diff -r vendor vendor.old
    no differences

so, what exactly is the purpose of manifest ?

@pksunkara

This comment has been minimized.

Copy link

pksunkara commented Jan 25, 2017

This analogy with npm helps me remember.

package.json => manifest.json
npm-shrinkwrap.json => lock.json
node_modules => vendor

@jbrodriguez

This comment has been minimized.

Copy link

jbrodriguez commented Jan 25, 2017

Right, but package.json does hold all dependencies.

I know dep is pre-alpha, just wondering if manifest.json will also hold all dependencies as it matures.

@pksunkara

This comment has been minimized.

Copy link

pksunkara commented Jan 25, 2017

Yeah, according to the design spec I read previously, it is supposed to.

@sdboyer

This comment has been minimized.

Copy link
Member

sdboyer commented Jan 27, 2017

(sorry, I've not had time to get to this one yet)

I know dep is pre-alpha, just wondering if manifest.json will also hold all dependencies as it matures.

Standard constraint declarations in manifest.json (under the dependencies key) are only applied to direct dependencies of the current project. It's important to understand, though, that if you declare a constraint in manifest.json, but don't actually import it, that constraint will be ignored. This is still Go - the import graph is queen. (If you want to force something to show up, even if it's not imported, then you want a require. There's no command for that yet, but you can write them into the manifest directly and they work fine 😄 )

Overrides are applied globally, superceding constraints declared by all dependencies (including any dependencies in your own manifest.json, if you did a weird thing and have both). lock.json holds a transitively complete list of all deps.

If you're accustomed to npm, especially older versions (pre-flattening), this all works a bit differently because dep doesn't allow duplication of dependencies.

We don't really have docs optimized for end users yet (pre-alpha, sorry!), but there's more info about the mechanics in the gps docs.

@pksunkara

This comment has been minimized.

Copy link

pksunkara commented Jan 27, 2017

Ah, my above statement was ambiguous. I meant that according to the spec manifest.json was supposed to hold the top level dependencies and any other overrides. Sorry about the wording "all dependencies"

@jbrodriguez

This comment has been minimized.

Copy link

jbrodriguez commented Jan 27, 2017

Standard constraint declarations in manifest.json (under the dependencies key) are only applied to direct dependencies of the current project

Sure, that's the most important thing. As it stands right now, this is not fulfilled, as it expects these direct dependencies to exist in $GOPATH/src.

In any case, the project is awesome and I feel it's going in the right direction, so I'll keep trying to poke holes in it (in a very friendly manner :) 👍

@sdboyer

This comment has been minimized.

Copy link
Member

sdboyer commented Jan 27, 2017

In any case, the project is awesome and I feel it's going in the right direction, so I'll keep trying to poke holes in it (in a very friendly manner :)

Thanks! please do :) 🎉

As it stands right now, this is not fulfilled, as it expects these direct dependencies to exist in $GOPATH/src.

mm, that's not the case. The only part of dep that touches GOPATH, at all, is dep init - and it only does so to scan what you have on your GOPATH in order to set initial constraints, which it then records in manifest.json. However, if any of those direct (or transitive) dependencies are missing, it'll reach out and try to make a default decision on what version to use by picking the "newest", as defined by the ordering rules. So, if you run dep init on a project in an otherwise empty GOPATH, it'll reach out for everything.

Of course, this issue is about cases in which that fails, and for some reason the resulting manifest is empty...and that's a bug, and I'm not sure why it's doing it. But if you're seeing behavior that makes it seem like dep is in any way requiring anything to be on the GOPATH, please please, do explain.

@sdboyer

This comment has been minimized.

Copy link
Member

sdboyer commented Jan 27, 2017

@raytung

Yep, that's it I think! It wouldn't save into the manifest.json file if the source doesn't already exist under $GOPATH/src.

Oh wait...maybe this is it. If this is all it is, then it's a super-easy fix. I'll look tonight :)

@sdboyer

This comment has been minimized.

Copy link
Member

sdboyer commented Jan 28, 2017

Ah, so, it seems like we actually did this intentionally for the first round with init - if it's not on disk, we don't put a constraint in the manifest. So, I want to check in with @adg before we go changing that.

@peterbourgon

This comment has been minimized.

Copy link
Member

peterbourgon commented Mar 14, 2017

For the record, I find it highly surprising (read: bad) that first-level deps are only in manifest.json conditionally. I expect all first-level deps to be there.

@sdboyer

This comment has been minimized.

Copy link
Member

sdboyer commented Mar 14, 2017

Agreed, and that's definitely the plan for the revisited command set - #277

@kevingo

This comment has been minimized.

Copy link

kevingo commented Mar 23, 2017

I also found this issue with empty manifest.json {}. And as for @raytung described, if a package does not in the $GOPATH/src, it will not save into the manifest.json file.

I just want to make sure that it is a bug and will be fix before release, right ? @sdboyer

By the way, it's a nice tool and we will happy for supporting dep will be put into the go toolchain in the near future 👍

@sdboyer

This comment has been minimized.

Copy link
Member

sdboyer commented Apr 1, 2017

Yes, we do intend to change this behavior with the respeccing of init.

kris-nova pushed a commit to kris-nova/dep that referenced this issue Apr 21, 2017

Merge pull request golang#149 from freeformz/edHack
Pull in parser code from godep as part of not using build
@sdboyer

This comment has been minimized.

Copy link
Member

sdboyer commented May 1, 2017

This should, finally, be fixed 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment