Expose generated Go files to editors and IDEs

[jayconrod]

When a library is created with go_proto_library or some other mechanism that generates Go code (.pb.go files in this case), the generated sources files are not exposed to editors and IDEs. Consequently, code completion and refactoring tools don't work, lowering developer productivity.

The Go team is developing a workspace abstraction mechanism that will allow editors and Bazel (and other build systems) to communicate with each other without the need for direct integration. This is the long-term solution to this problem.

This issue will track progress on the workspace abstraction and will track editor awareness of code generated with Bazel.

[jayconrod]

This issue is in response to Bazel build, protobuf and code completion.

TODO: update answer when this issue is resolved.

[excavador]

What the status of the task?

[excavador]

So, I have proposal how to close this issue.

Suppose you generated code:
bazel-genfiles/somedir/somefile.go
bazel-genfiles/somedir/anotherfile.go

Let's add symlink
src/somedir => ../bazel-genfiles/somedir

Use in your files
import "somedir"

As result you will get workable code-completition AND workable build.

[jayconrod]

@excavador We're still developing a more complete solution to this (the workspace abstraction I mentioned above). Ideally, tools won't need to be aware of the project layout in use. I don't have any ETA for that yet, but we are making progress.

As you showed, it's possible to work around this with symlinks. You can also copy or symlink generated files into your source directory. Bazel will prefer a generated file if a source file of the same name is present. I'd recommend against checking generated files into VCS though.

[excavador]

Thank you for notify! Will wait. You can consider me as closed beta-tester :)

[raliste]

Any updates?

[excavador]

@raliste it works for me in IDEA Ultimate + Bazel plugin with following limitation: autocomplete works only if all files inside golang package are auto-generated. If mix together auto-generated and manually written file, autogenerated file would not be inspected

[jayconrod]

The workspace abstraction mentioned above is being actively worked on. We've gone through a lot of internal prototypes, and we're in the process of reviewing, testing, and merging one that we're pretty happy with. It's a huge body of code though, and it will take time. Following that, a large number of Go tools will need to be updated to use the workspace abstraction so they can function in "go build", vgo, Bazel, and other kinds of workspaces.

I know it seems like progress is slow, but it's an interface that many, many tools will interact with, and we want to be sure we get it right. It's a huge amount of work, but it is our highest priority task.

[burdiyan]

@jayconrod Are there any public discussions about that? Design docs perhaps?

[jayconrod]

Not yet, but soon. We're still working on simplifying and narrowing the interface.

[vitarb]

Can you briefly explain how this workspaces feature is going to work?
Are there any upcoming changes in the 1.11 related to this?

[jayconrod]

The first public piece of this is golang.org/x/tools/go/packages. At the moment, that is based on go list and is intended to be used by tools that support go build and vgo. We're also working on an workspace abstraction layer that go/packages may eventually be built on. That workspace abstraction would be extensible and could include Bazel. Lot of work still happening there.

[akshayjshah]

@jayconrod Any updates on the proposed workspace abstraction layer?

[jayconrod]

@akshayjshah We've made some progress. The golang.org/x/tools/go/packages API is nearing completion. Our team has migrated a few tools to use it already, but the effort is ongoing.

If you set the GOPACKAGESDRIVER environment variable, or if you have a binary named gopackagesdriver in PATH, go/packages will shell out to that binary for queries. I have an implementation of that binary about 70% done. It works in most situations, but it needs polish, documentation, and tests.

So ideally, once that change is in, you'd be able to build and install the driver into your PATH, and tools that use go/packages will just work in Bazel workspaces (and they'll fall back to the default behavior if there's no WORKSPACE file).

[akshayjshah]

Thanks for the update! Would you be willing to push your driver to a branch so I can take a look? I'm not in a huge rush for you to officially release it, but I'd love to use your rough draft to better understand how these pieces should fit together.

[jayconrod]

@akshayjshah Sure, it's at https://github.com/jayconrod/rules_go/tree/dev-gopackagesdriver if you'd like to take a look. As I said, it's very rough right now.

[pierreis]

That looks awesome, thanks! Is there any update on this?

[jayconrod]

@pierreis Sorry, no updates since my last comment. I'm still planning to work on this this quarter. There are a lot of things going on, but this is one of the higher priority projects.