Skip to content
This repository has been archived by the owner on Mar 6, 2020. It is now read-only.

gb not building one of my project's main packages #632

Open
patrickxb opened this issue Jul 2, 2016 · 48 comments
Open

gb not building one of my project's main packages #632

patrickxb opened this issue Jul 2, 2016 · 48 comments
Labels

Comments

@patrickxb
Copy link

I'm trying out gb by copying one service out of a large repo and putting it into a gb project. I've been working on getting all of its dependencies in place, but now, inexplicably, gb stopped trying to build the service. It does continue to build the dependencies, and if I create a hello app it builds that.

Directory layout:

project/src/srv/sapid/sapid.go, sapid_test.go (the service I'm trying to build, package main)
project/src/sh/...(all packages needed by sapid.go)
project/src/std/... (some more packages need by sapid.go)
project/vendor/src/... (all vendored packages needed by sapid.go)

gb list shows everything under sh but not srv/sapid. If I add a hello directory to project/src/srv/ and place a main package with fmt.Println("hello"), the next invoccation of gb list will show srv/sapid orgb build all` will build hello.

env DEBUG=. gb build all shows srv/sapid in args.

Any idea what is going on? project/src/srv/sapid/sapid.go, sapid_test.go were the first files I put in a gb project, and gb was definitely attempting to build them. There are no build tags or anything that would cause it not to be built.

Thanks!

@davecheney
Copy link
Contributor

Is the source online somewhere I can look at ?

@davecheney
Copy link
Contributor

Can you please post the complete env DEBUG=. gb build output. Thanks.

@davecheney
Copy link
Contributor

And confirm your gb revision. Thanks.

@patrickxb
Copy link
Author

Sorry, the source isn't online.

~/hats> env DEBUG=. gb build all
2016/07/01 22:16:31 cmd.go:82: project root "/Users/patrick/hats"
2016/07/01 22:16:31 depfile.go:185: loading depfile at "/Users/patrick/hats/depfile"
2016/07/01 22:16:31 depfile.go:33: no depfile, nothing to do.
2016/07/01 22:16:31 match.go:18: /Users/patrick/hats/src .
2016/07/01 22:16:31 match.go:108: matchPackages: ...
2016/07/01 22:16:32 main.go:118: args: [sh sh/cache sh/cache/bstash sh/cache/buffer sh/cache/clientcache sh/constants sh/crypto sh/crypto/keys sh/environment sh/ext sh/ext/jaws sh/glob sh/glog sh/net sh/net/cpr sh/net/entdeckung sh/net/eroute sh/net/qob sh/net/rconn sh/net/router sh/time sh/time/timeutil sh/tools sh/tools/ln sh/xbase64 sh/xrpc sh/xrpc/xping srv srv/hello srv/sapid std std/net std/net/rpc]
2016/07/01 22:16:32 build.go:50: build duration: 85.177µs map[]
2016/07/01 22:16:32 context.go:288: removing work directory: /var/folders/z1/78yn48v5411fz029z5s6t7q00000gn/T/gb800143756

gb commit: f25e86b

@davecheney
Copy link
Contributor

As far as gb is concerned, there is nothing to do. What happens if you
remote $PROJECT/bin ?

On Sat, Jul 2, 2016 at 1:34 PM, Patrick Crosby notifications@github.com
wrote:

Sorry, the source isn't online.

~/hats> env DEBUG=. gb build all
2016/07/01 22:16:31 cmd.go:82: project root "/Users/patrick/hats"
2016/07/01 22:16:31 depfile.go:185: loading depfile at
"/Users/patrick/hats/depfile"
2016/07/01 22:16:31 depfile.go:33: no depfile, nothing to do.
2016/07/01 22:16:31 match.go:18: /Users/patrick/hats/src .
2016/07/01 22:16:31 match.go:108: matchPackages: ...
2016/07/01 22:16:32 main.go:118: args: [sh sh/cache sh/cache/bstash
sh/cache/buffer sh/cache/clientcache sh/constants sh/crypto sh/crypto/keys
sh/environment sh/ext sh/ext/jaws sh/glob sh/glog sh/net sh/net/cpr
sh/net/entdeckung sh/net/eroute sh/net/qob sh/net/rconn sh/net/router
sh/time sh/time/timeutil sh/tools sh/tools/ln sh/xbase64 sh/xrpc
sh/xrpc/xping srv srv/hello srv/sapid std std/net std/net/rpc]
2016/07/01 22:16:32 build.go:50: build duration: 85.177µs map[]
2016/07/01 22:16:32 context.go:288: removing work directory:
/var/folders/z1/78yn48v5411fz029z5s6t7q00000gn/T/gb800143756

gb commit: f25e86b
f25e86b


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#632 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAAcA4pL3i_w4sXFSSLzi9vsYSFgameyks5qRdw0gaJpZM4JDlvh
.

@patrickxb
Copy link
Author

I edited $PROJECT/src/srv/sapid/sapid.go and now it is (attempting) to build it again. But there was never anything in bin for sapid.

Thanks for prompt response...I'll keep going, but I was in a weird state...

@davecheney
Copy link
Contributor

davecheney commented Jul 2, 2016

Are you sure $PROJECT/src/srv/sapid/sapid.go is package main ?

On Sat, Jul 2, 2016 at 1:41 PM, Patrick Crosby notifications@github.com
wrote:

I edited $PROJECT/src/srv/sapid/sapid.go and now it is (attempting) to
build it again. But there was never anything in bin for sapid.

Thanks for prompt response...I'll keep going, but I was in a weird state...


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#632 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAAcA0xwrlVhl5_6eWrueHxrbCE1Aq8Wks5qRd3_gaJpZM4JDlvh
.

@patrickxb
Copy link
Author

patrickxb commented Jul 2, 2016

Yes. It's been a package main since about 2011 :) ...

~/hats> head src/srv/sapid/sapid.go
// sapid is the web service handling all incoming api calls.
package main

import (
"crypto/md5"
"encoding/json"
"errors"
"flag"
"fmt"
"io"

@davecheney
Copy link
Contributor

So, to be clear, gb is not building this particular package main? Do you have any symlinks in your $PROJECT?

@patrickxb
Copy link
Author

Yes, it isn't building it. No symlinks. I've been copying code from existing repo to this project to try out gb. I just fixed some more import paths, and now it is no longer building sapid:

~/hats> env DEBUG=. gb build all
2016/07/01 22:47:37 cmd.go:82: project root "/Users/patrick/hats"
2016/07/01 22:47:37 depfile.go:185: loading depfile at "/Users/patrick/hats/depfile"
2016/07/01 22:47:37 depfile.go:33: no depfile, nothing to do.
2016/07/01 22:47:37 match.go:18: /Users/patrick/hats/src .
2016/07/01 22:47:37 match.go:108: matchPackages: ...
2016/07/01 22:47:37 main.go:118: args: [sh sh/cache sh/cache/bstash sh/cache/buffer sh/cache/buffercache sh/cache/clientcache sh/constants sh/crypto sh/crypto/keys sh/environment sh/ext sh/ext/jaws sh/glob sh/glog sh/net sh/net/cpr sh/net/entdeckung sh/net/eroute sh/net/qob sh/net/rconn sh/net/router sh/time sh/time/timeutil sh/tools sh/tools/ln sh/xbase64 sh/xrpc sh/xrpc/xbuckd sh/xrpc/xbufcached sh/xrpc/xping srv srv/hello srv/sapid std std/net std/net/rpc]
2016/07/01 22:47:37 install.go:70: /Users/patrick/hats/pkg/darwin-amd64/sh/xrpc/xbuckd.a is missing
2016/07/01 22:47:37 install.go:122: /Users/patrick/hats/pkg/darwin-amd64/srv/hello.a is newer than /Users/patrick/hats/bin/hello
2016/07/01 22:47:37 context.go:309: cd /Users/patrick/hats/src/sh/xrpc/xbuckd; [/usr/local/go/pkg/tool/darwin_amd64/compile -p sh/xrpc/xbuckd -pack -o /var/folders/z1/78yn48v5411fz029z5s6t7q00000gn/T/gb957130655/sh/xrpc/xbuckd.a -I /var/folders/z1/78yn48v5411fz029z5s6t7q00000gn/T/gb957130655 -I /Users/patrick/hats/pkg/darwin-amd64 -complete xbuckd.go]
2016/07/01 22:47:37 context.go:309: cd /Users/patrick/hats/src/srv/hello; [/usr/local/go/pkg/tool/darwin_amd64/compile -p srv/hello -pack -o /var/folders/z1/78yn48v5411fz029z5s6t7q00000gn/T/gb957130655/srv/hello/main.a -I /var/folders/z1/78yn48v5411fz029z5s6t7q00000gn/T/gb957130655 -I /Users/patrick/hats/pkg/darwin-amd64 -complete hello.go]
2016/07/01 22:47:38 context.go:309: cd .; [/usr/local/go/pkg/tool/darwin_amd64/link -o /Users/patrick/hats/bin/.gb-link547108978 -L /var/folders/z1/78yn48v5411fz029z5s6t7q00000gn/T/gb957130655 -L /Users/patrick/hats/pkg/darwin-amd64 -extld clang -buildmode exe /var/folders/z1/78yn48v5411fz029z5s6t7q00000gn/T/gb957130655/srv/hello/main.a]
sh/xrpc/xbuckd
srv/hello
2016/07/01 22:47:38 build.go:50: build duration: 246.34882ms map[gc:36.35015ms link:232.870703ms]
2016/07/01 22:47:38 context.go:288: removing work directory: /var/folders/z1/78yn48v5411fz029z5s6t7q00000gn/T/gb957130655

~/hats> ls bin
hello

~/hats> env DEBUG=. gb build all
2016/07/01 22:50:50 cmd.go:82: project root "/Users/patrick/hats"
2016/07/01 22:50:50 depfile.go:185: loading depfile at "/Users/patrick/hats/depfile"
2016/07/01 22:50:50 depfile.go:33: no depfile, nothing to do.
2016/07/01 22:50:50 match.go:18: /Users/patrick/hats/src .
2016/07/01 22:50:50 match.go:108: matchPackages: ...
2016/07/01 22:50:50 main.go:118: args: [sh sh/cache sh/cache/bstash sh/cache/buffer sh/cache/buffercache sh/cache/clientcache sh/constants sh/crypto sh/crypto/keys sh/environment sh/ext sh/ext/jaws sh/glob sh/glog sh/net sh/net/cpr sh/net/entdeckung sh/net/eroute sh/net/qob sh/net/rconn sh/net/router sh/time sh/time/timeutil sh/tools sh/tools/ln sh/xbase64 sh/xrpc sh/xrpc/xbuckd sh/xrpc/xbufcached sh/xrpc/xping srv srv/hello srv/sapid std std/net std/net/rpc]
2016/07/01 22:50:50 build.go:50: build duration: 71.061µs map[]
2016/07/01 22:50:50 context.go:288: removing work directory: /var/folders/z1/78yn48v5411fz029z5s6t7q00000gn/T/gb103059960

@davecheney
Copy link
Contributor

What i think has happened is sapid used to have more files, but they were deleted or removed, so from the POV of gb, the compiled .a file is still newer than the source files that are in that directory, so it doesn't think there is anything to do. If you touch a file in sapid/*.go that might fix it.

@patrickxb
Copy link
Author

There are only two files in sapid/: sapid.go and sapid_test.go. touch didn't do anything.

@davecheney
Copy link
Contributor

Can you please run this command at $PROJECT

gb list -f '{{.GoFiles}}' srv/sapid

And paste the result.

@patrickxb
Copy link
Author

No output...

~/hats> ls -l src/srv/sapid
total 176
-rw-r--r-- 1 patrick staff 23368 Jul 1 22:57 sapid.go
-rw-r--r-- 1 patrick staff 61772 Jul 1 22:57 sapid_test.go

@davecheney
Copy link
Contributor

Can you share those files with me ?

@patrickxb
Copy link
Author

I wish I could, but those two happen to be pretty private. They've been building fine on all versions of go since pre 1.0...

Anything else I could try?

@davecheney
Copy link
Contributor

What happens if you replace the contents of sapid.go with

package main

func main() { 
        println("hello world')
}

Does that change the output from gb list (above), etc ?

@patrickxb
Copy link
Author

patrickxb commented Jul 2, 2016

yes:

~/hats> gb list -f '{{.GoFiles}}' srv/sapid
[sapid.go]

@davecheney
Copy link
Contributor

So there is something about sapid.go. Are you able to share the contents of the file up to the first line after the import statements ?

@patrickxb
Copy link
Author

patrickxb commented Jul 2, 2016

~/hats/src/srv/sapid> head -40 sapid.go

// sapid is the web service handling all incoming api calls.
package main

import (
        "crypto/md5"
        "encoding/json"
        "errors"
        "flag"
        "fmt"
        "io"
        "net"
        "net/http"
        _ "net/http/pprof"
        "net/rpc"
        "os"
        "strconv"
        "strings"
        "sync"
        "time"

        "sh/cache/bstash"
        "sh/constants"
        "sh/crypto/keys"
        "sh/environment"
        "sh/glob"
        "sh/glog"
        "sh/net/eroute"
        "sh/net/qob"
        "sh/net/router"
        "sh/time/timeutil"
        "sh/tools/ln"
        "sh/xrpc/xbuckd"
        "sh/xrpc/xbufcached"
        "sh/xrpc/xhgram"
        "sh/xrpc/xsapid"
)

const (
        port = "9999"
)

@davecheney
Copy link
Contributor

That looks pretty reasonable. I thought there was a huge comment block or something that would throw the importer off.

Can you check that there are no byte order marks at the front of the file.

@patrickxb
Copy link
Author

All those sh packages I have been copying in to the gb project when gb complains they don't exist. Then it mysteriously stopped complaining, but the binary wasn't built. The last two, xhgram, xsapid are not in the project.

@davecheney
Copy link
Contributor

Can you please run this in the root of your project.

gb list -f '{{ .ImportPath }} {{ .IgnoredGoFiles }}' 

It will show with packages have files which are being ignored.

@patrickxb
Copy link
Author

No byte order marks.

sh/cache/bstash []
sh/cache/buffer []
sh/cache/clientcache []
sh/constants []
sh/crypto/keys []
sh/environment []
sh/ext/jaws []
sh/glob []
sh/glog []
sh/net/cpr []
sh/net/entdeckung []
sh/net/eroute []
sh/net/qob []
sh/net/rconn []
sh/net/router []
sh/time/timeutil []
sh/tools/ln []
sh/xbase64 []
sh/xrpc/xbuckd []
sh/xrpc/xping []
srv/hello []
std/net/rpc []

@davecheney
Copy link
Contributor

Yet, srv/sapid is missing completely. What happens if you run

gb list -f '{{ .ImportPath }} {{ .IgnoredGoFiles }}' srv/sapid

@davecheney
Copy link
Contributor

Just to check, you haven't accidentally created a $GOROOT/src/srv/ directory ?

@patrickxb
Copy link
Author

Nope.

@patrickxb
Copy link
Author

No output to gb list -f '{{ .ImportPath }} {{ .IgnoredGoFiles }}' srv/sapid

@davecheney
Copy link
Contributor

What if you copy the files to a different directory?

@patrickxb
Copy link
Author

I tried that before posting the original issue. But just tried again. I put them in src/what/sapid.go, src/what/sapid_test.go. No change. The gb list ...commands above don't show them.

@patrickxb
Copy link
Author

I swear there's nothing unusual in these files...I can't think of why they would be different than anything else.

@davecheney
Copy link
Contributor

There is something about sapid,go that cannot be parsed. Can I ask you to try to reduce the file to the point where gb will try to compile it (it's ok if it fails) -- or -- if you can reduce it to the point where it is not longer confidential.

@patrickxb
Copy link
Author

Commenting out the import ( ... ) block makes it compile. Trying to narrow it down further.

@patrickxb
Copy link
Author

ok, for some reason it is the last one, "sh/xrpc/xbufcached". With it commented out, gb tries to compile. With it there, it doesn't. The package exists in the project.

@patrickxb
Copy link
Author

Ok, so one package imports "sh/tools" and there were no .go files in there. But there was "sh/tools/ln/ln.go".

That's my best guess right now...

@davecheney
Copy link
Contributor

Awesome, that gives me something to go on, I'll try to produce a repo now.

@davecheney
Copy link
Contributor

davecheney commented Jul 2, 2016

Hmm, so if something imports "sh/tools", but there are no go files in $PROJECT/src/sh/tools, how does it import that file without it being a compile error ?

@patrickxb
Copy link
Author

The thing was that gb wasn't even trying to compile and showing the error.

On Jul 2, 2016, at 1:00 AM, Dave Cheney notifications@github.com wrote:

Hmm, so if something import "sh/tools", but there are no go file sin $PROJECT/src/sh/tools, how does it import that file without it being a compile error ?


You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

@davecheney
Copy link
Contributor

@patrickxb I'm trying to create a test for this problem. I need your help to locate the file in that imports a package that doesn't exist yet doesn't use it. It's likely that that package is using the side effect _ underscore import.

@patrickxb
Copy link
Author

I made a small gb project that reproduces the issue, I believe.

gbrepro.tar.gz

Doing gb build all doesn't show any errors, doesn't build cmd/app. But cmd/app/app.go imports parent which is an empty package (although it does have a subdirectory with a valid package in it).

@patrickxb
Copy link
Author

And the go command shows the error:

~/gbrepro/src> GOPATH=~/gbrepro go install ./...
sibling/sibling.go:3:8: no buildable Go source files in /Users/patrick/gbrepro/src/parent

Hope that helps...

@davecheney
Copy link
Contributor

Thank you very much for producing this repo. This is exactly what I need to
chase down the issue.

On Wed, 6 Jul 2016, 15:14 Patrick Crosby notifications@github.com wrote:

And the go command shows the error:

/gbrepro/src> GOPATH=/gbrepro go install ./...
sibling/sibling.go:3:8: no buildable Go source files in /Users/patrick/gbrepro/src/parent

Hope that helps...


You are receiving this because you commented.

Reply to this email directly, view it on GitHub
#632 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAAcA2-JT1Jw6K8fm39eob9i6Uaon0ysks5qSznNgaJpZM4JDlvh
.

@davecheney
Copy link
Contributor

Well, the good news is I broke this after 0.4.2, so a workaround is to build from that tag.

The bad news is I broke this and didn't have a test, but now I do, so it should be fixed in 0.4.3.

@patrickxb
Copy link
Author

Cool, glad it helped!

Really enjoying my experiment with gb so far. Great work, Dave!

@ak2344
Copy link

ak2344 commented Sep 25, 2016

Just wanted to drop in and say that I'm having a similar issue. gb just doesn't like a package I have, and it just stops building any file that imports it.

It's a private file so not sure if I can easily break it out so it can be reproduced.

Any idea when this issue might be resolved?

Btw, @patrickxb, big fan on stathat. 👍

@ak2344
Copy link

ak2344 commented Sep 25, 2016

Arghhh. Turns out I'm not that special.

I had changed a package's path and the import block of file with the issue hadn't been updated.

I fixed the path and things seem to work again. Though what's weird is that this error wasn't printed when building.

Either way, thanks for all your work on this, @davecheney.

@davecheney davecheney added the bug label Nov 13, 2016
@davecheney
Copy link
Contributor

I'm pretty sure this is fixed now that 0.4.3 is out. Please reopen if this is not the case.

@superdump
Copy link

I'm using:

commit d4ba64b0b29760e90322851fb793b04665b0c3c9
Author: Dave Cheney <dave@cheney.net>
Date:   Wed Feb 22 23:02:41 2017 +1100

    Remove filepath.Rel code from gc helper. (#693)

    GoFiles in gb come in two kinds.

    1. relative to pkg.Dir
    2. absolute

    In this case the work to absify then relify the path is a noop, which is
    further proven by removing it.

And I still see this issue. If any import path in a main package points to a directory with no .go files in it then that main package is silently not built. Also manually pointing gb at that main package gives the message: FATAL: command "build" failed: no packages supplied

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

4 participants