Generate symlinks for vendor/ directory

[samuraisam]

We use bazel and rules_go to build our infrastructure repo composed of various golang services. We've specced out all of our go dependencies using WORKSPACE, new_git_repository, and go_library in a third_party dir. I like this a lot because it allows us to be extremely explicit with all of our dependencies. This works great for building our binaries and docker files, however it gets difficult for tool support. It would be awesome if we could symlink all of our dependencies underneath vendor/ so whatever IDE you're using that supports the vendor directory can automatically see the dependent sources.

For example a WORKSPACE file that defines:

git_repository(
    name = "io_bazel_rules_go",
    remote = "https://github.com/bazelbuild/rules_go.git",
    tag = "0.4.1",
)

load("@io_bazel_rules_go//go:def.bzl", "go_repositories")

go_repositories()

new_git_repository(
    name = "go_x_crypto",
    build_file = "third_party/go/x_crypto.BUILD",
    commit = "573951cbe80bb6352881271bb276f48749eab6f4",
    remote = "https://go.googlesource.com/crypto.git",
)

And a third_party/go/x_crypto.BUILD:

package(default_visibility = ["@//visibility:public"])

load("@//third_party:go/build.bzl", "external_go_package", "external_go_package_srcs")

external_go_package(
    base_pkg = "golang.org/x/crypto",
    deps = [
        ":nacl/secretbox",
        ":pbkdf2",
        ":poly1305",
        ":salsa20/salsa",
        ":scrypt",
    ],
)

You can see the build.bzl file here, it simply sets up an alias and calls go_library.

It would be great if we could symlink vendor/golang.org/x/crypto to wherever @go_x_crypto is checked out. I don't see a way currently to do this. Is there a way to do so? I found gobazel but it seems like there should be a simpler (and platform independent) way by just using bazel rules and symlinks... or maybe I am thinking about this entirely wrong?

[jayconrod]

There's not really a way to access external repositories in Bazel. They're stored in Bazel's cache directory, and their location within that directory tends to change between Bazel versions.

Unfortunately, I don't really have a good solution for you. There are a lot of rough edges around dependency management right now, and it's something we're looking at improving.

Probably the best thing for IDE support at the moment is to use regular vendoring (instead of listing dependencies in WORKSPACE). You can generate BUILD files for vendored packages with gazelle -external=vendored. If your dependencies are checked into your repo, the BUILD files can be checked in, too. If you get your dependencies via submodules or some other mechanism, you'll have to run Gazelle as a pre-processing step.

[samuraisam]

Thank you @jayconrod. I had considered using a vendoring tool (like glide or any one of the dozen others, many of which we've tried) and gazelle, but for now prefer to use external repositories. Since my only goal here is for tool/IDE support, I hacked together a small tool which reads the workspace and maintains the repositories in a vendor/ directory.

Here is the tool for those interested.

[jayconrod]

Glad you found something that works for you. I hope we can improve this further in the future.