Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/cmd/goimports: can't find proto packages #16600

Open
danielchatfield opened this issue Aug 4, 2016 · 9 comments

Comments

@danielchatfield
Copy link

commented Aug 4, 2016

Please answer these questions before submitting your issue. Thanks!

  1. What version of Go are you using (go version)?

go version go1.6.2 darwin/amd64

  1. What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/danielchatfield"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.6.2/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.6.2/libexec/pkg/tool/darwin_amd64"
GO15VENDOREXPERIMENT="1"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common"
CXX="clang++"
CGO_ENABLED="1"

Package fooproto in github.com/danielchatfield/repo/foo/proto

Running goimports on the following fails to find the import.

package main

func main() {
    _ := &fooproto.SomeStruct{}
}

The first commit that broke this is https://github.com/golang/tools/commit/e047ae774b027ae958a270598c5ac2591e457afc

@danielchatfield danielchatfield changed the title x/tools/cmd/goimports x/tools/cmd/goimports Can't find proto packages Aug 4, 2016

@quentinmit quentinmit added this to the Unreleased milestone Aug 4, 2016

@josharian josharian changed the title x/tools/cmd/goimports Can't find proto packages x/tools/cmd/goimports: can't find proto packages Aug 4, 2016

@bradfitz

This comment has been minimized.

Copy link
Member

commented Aug 4, 2016

This is a side-effect of the pruning of the search space we now do. We have a function which determines whether a package like "fooproto" might appear in a directory as a function of its directory name, considering mostly its final two path components.

It ignores case, and drops hyphens and periods, and then looks for substrings. But "fooproto" doesn't appear in "foo/proto". I suppose I could also try concatenating the last two path components and searching that too.

But does that solve the general problem?

How do protobuf package names & directory names look like in the wild? I haven't been following what the conventions are outside of Google or at least outside of Google open source projects.

@akalin-keybase

This comment has been minimized.

Copy link

commented Aug 22, 2016

Is this one instance of the general problem that a package whose name differs from its containing directory won't be found by goimports anymore?

i.e. if I have $GOPATH/src/github.com/akalin/mymain/main.go with:

package main

import (
    "github.com/akalin/mypkg"
    "github.com/akalin/mypkg2"
)

func main() {
    mypkg.Foo()
    myotherpkg.Bar()
}

where the mypkg package is in github.com/akalin/mypkg, but the myotherpkg package is in github.com/akalin/mypkg2, then goimports will automatically insert the first import if omitted, but won't insert the second import if omitted? (This is the current behavior, according to my tests.)

Sounds like this is a casualty of the new speed improvements. If so, it's unfortunate but understandable. I didn't run into this with protobuf packages, but with similar auto-generated code, namely https://github.com/keybase/client/tree/master/go/protocol , which has package name keybase1 but directory name protocol.

It's probably impossible to solve this in general without slowing down goimports again, but I wonder if we could have some sort of convention for auto-generated code, such that the directory name is some function of the package name, e.g. pkgName + "Autogen".

@bradfitz

This comment has been minimized.

Copy link
Member

commented Aug 23, 2016

Sounds like this is a casualty of the new speed improvements.

Yup.

If so, it's unfortunate but understandable.

It's at least fortunate in one way: it gives yet another reason to name your packages the conventional way, making it a bit tedious if you want to violate the norms.

I definitely wouldn't penalize people like that on purpose, but if it's necessary to make goimports fast, I think it's quite fine.

I wouldn't support any proposal (like pkgName + "Autogen") to make violating the norm easier. There's no reason for autogen packages to declare that they're auto-generated in their package name. If it really needs to be in the import path, it can be in the second to last path component or something.

@bradfitz

This comment has been minimized.

Copy link
Member

commented Aug 25, 2016

@dsymonds, can you answer my question in #16600 (comment)? Are there conventions for how people name their proto packages vs. how they end up on disk in their import path?

@dsymonds

This comment has been minimized.

Copy link
Member

commented Aug 25, 2016

I'm not sure there's a well-established convention. We encourage people to always rename proto imports for this kind of reason.

If you're going to apply any hack, I'd suggest bypassing the pruning if the import name ends in "pb", which should rarely happen except for protos.

@cespare

This comment has been minimized.

Copy link
Contributor

commented Aug 25, 2016

At $COMPANY our protobuf build scripts move our generated code to a directory that matches the package name specifically to avoid violating the Go convention and to make sure that tools like goimports work.

(As far as the special names thing goes, we also name our packages specially using option go_package = "foopb"; to make it easier to identify which packages are generated code; this seems orthogonal to the package name != path issue, though.)

@bradfitz

This comment has been minimized.

Copy link
Member

commented Aug 25, 2016

If you're going to apply any hack, I'd suggest bypassing the pruning if the import name ends in "pb", which should rarely happen except for protos.

Oh, that's an interesting idea, to selectively prune. Or I can at least filter off "pb", and then prune.

Does some tool default to the "pb" suffix, or is that just convention?

@dsymonds

This comment has been minimized.

Copy link
Member

commented Aug 25, 2016

It's a strong convention. The part before the "pb" (and sometimes there's nothing before the pb) is human chosen, and isn't always consistent for a given imported package. From memory, golint and other similar tools know the "pb" suffix convention and modify their behaviour a bit.

@bradfitz bradfitz removed their assignment May 29, 2019

@bradfitz

This comment has been minimized.

Copy link
Member

commented May 29, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.