Python Go Shell C C++ Objective-C Other
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.bazelci Add CI config to test on RBE. (#1638) Aug 8, 2018
extras Update deprecated single_file -> allow_single_file attribute (#1628) Aug 6, 2018
go doc: fix grammatical error (#1671) Aug 20, 2018
proto Document how to avoid proto conflicts [skip ci] (#1631) Aug 6, 2018
tests Update tests ahead of Go 1.11 (#1661) Aug 15, 2018
third_party Update org_golang_x_tools to master as of 2018-08-15 (#1662) Aug 16, 2018
tools Skylark compatability (#1175) Dec 22, 2017
.bazelrc Create .bazelrc (#1626) Aug 3, 2018
.gitignore Add .gitignore file Jul 18, 2016
.travis.yml Update minimum version of Bazel for Travis CI to 0.16.0 (#1651) Aug 10, 2018
AUTHORS Test that AUTHORS and CONTRIBUTORS are sorted (#48) Jul 19, 2016
BUILD.bazel Remove gazelle and its deps from go_rules_dependencies (#1659) Aug 15, 2018
CODEOWNERS Add CODEOWNERS file (#1042) Nov 21, 2017
CONTRIBUTING.md Update CONTRIBUTING.md (#326) Mar 23, 2017
CONTRIBUTORS Add the current team to the contributors list (#445) May 8, 2017
LICENSE.txt Move go rules to their own repository Mar 4, 2016
README.rst one character fix to README boilerplate [skip ci] (#1667) Aug 16, 2018
Vendoring.md Remove wtool (#1302) Feb 5, 2018
WORKSPACE Remove gazelle and its deps from go_rules_dependencies (#1659) Aug 15, 2018
deprecation.rst Remove gazelle and its deps from go_rules_dependencies (#1659) Aug 15, 2018
hack_go_repository.bzl Update dependencies in preparation for next release (#1583) Jul 4, 2018
roadmap.rst Minor documentation fixes (#1403) Mar 23, 2018

README.rst

Go rules for Bazel

Travis Bazel CI
travis bazelci

Mailing list: bazel-go-discuss

Announcements

August 16, 2018
Release 0.15.0 is now available.
August 8, 2018
Release 0.14.0 is now available.
July 10, 2018
Release 0.13.0 is now available.

Contents

Documentation

Quick links

Overview

The rules are in the alpha stage of development. They support:

They currently do not support (in order of importance):

  • bazel-style auto generating BUILD (where the library name is other than go_default_library)
  • C/C++ interoperation except cgo (swig etc.)
  • coverage

Note: The latest version of these rules (0.15.0) requires Bazel ≥ 0.16.0 to work.

The master branch is only guaranteed to work with the latest version of Bazel.

Setup

  • Create a file at the top of your repository named WORKSPACE and add one of the snippets below, verbatim. This will let Bazel fetch necessary dependencies from this repository and a few others.

    If you want to use the latest stable release, add the following:

    load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
    http_archive(
        name = "io_bazel_rules_go",
        urls = ["https://github.com/bazelbuild/rules_go/releases/download/0.15.0/rules_go-0.15.0.tar.gz"],
        sha256 = "56d946edecb9879aed8dff411eb7a901f687e242da4fa95c81ca08938dd23bb4",
    )
    load("@io_bazel_rules_go//go:def.bzl", "go_rules_dependencies", "go_register_toolchains")
    go_rules_dependencies()
    go_register_toolchains()

    If you want to use a specific commit (for example, something close to master), add the following instead:

    load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
    git_repository(
        name = "io_bazel_rules_go",
        remote = "https://github.com/bazelbuild/rules_go.git",
        commit = "a390e7f7eac912f6e67dc54acf67aa974d05f9c3",
    )
    load("@io_bazel_rules_go//go:def.bzl", "go_rules_dependencies", "go_register_toolchains")
    go_rules_dependencies()
    go_register_toolchains()

    You can add more external dependencies to this file later (see go_repository).

  • Add a file named BUILD.bazel in the root directory of your project. In general, you need one of these files in every directory with Go code, but you need one in the root directory even if your project doesn't have any Go code there.

  • If your project can be built with go build, you can generate your build files using Gazelle. If your project isn't compatible with go build or if you prefer not to use Gazelle, you can write build files by hand.

Generating build files

If your project can be built with go build, you can generate and update your build files automatically using gazelle.

  • Add the bazel_gazelle repository and its dependencies to your WORKSPACE file before go_rules_dependencies is called. It should look like this:

    load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
    http_archive(
        name = "io_bazel_rules_go",
        urls = ["https://github.com/bazelbuild/rules_go/releases/download/0.15.0/rules_go-0.15.0.tar.gz"],
        sha256 = "56d946edecb9879aed8dff411eb7a901f687e242da4fa95c81ca08938dd23bb4",
    )
    http_archive(
        name = "bazel_gazelle",
        urls = ["https://github.com/bazelbuild/bazel-gazelle/releases/download/0.14.0/bazel-gazelle-0.14.0.tar.gz"],
        sha256 = "c0a5739d12c6d05b6c1ad56f2200cb0b57c5a70e03ebd2f7b87ce88cabf09c7b",
    )
    load("@io_bazel_rules_go//go:def.bzl", "go_rules_dependencies", "go_register_toolchains")
    go_rules_dependencies()
    go_register_toolchains()
    load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies")
    gazelle_dependencies()
  • Add the code below to the BUILD or BUILD.bazel file in the root directory of your repository. Replace the string after prefix with the prefix you chose for your project earlier.

    load("@bazel_gazelle//:def.bzl", "gazelle")
    
    # gazelle:prefix github.com/example/project
    gazelle(name = "gazelle")
  • After adding the gazelle rule, run the command below:

    bazel run //:gazelle
    

    This will generate a BUILD.bazel file for each Go package in your repository. You can run the same command in the future to update existing build files with new source files, dependencies, and options.

Writing build files by hand

If your project doesn't follow go build conventions or you prefer not to use gazelle, you can write build files by hand.

  • In each directory that contains Go code, create a file named BUILD.bazel

  • Add a load statement at the top of the file for the rules you use.

    load("@io_bazel_rules_go//go:def.bzl", "go_binary", "go_library", "go_test")
  • For each library, add a go_library rule like the one below. Source files are listed in srcs. Other packages you import are listed in deps using Bazel labels that refer to other go_library rules. The library's import path should be specified with importpath.

    go_library(
        name = "go_default_library",
        srcs = [
            "foo.go",
            "bar.go",
        ],
        deps = [
            "//tools:go_default_library",
            "@org_golang_x_utils//stuff:go_default_library",
        ],
        importpath = "github.com/example/project/foo",
        visibility = ["//visibility:public"],
    )
  • For each test, add a go_test rule like either of the ones below. You'll need separate go_test rules for internal and external tests.

    # Internal test
    go_test(
        name = "go_default_test",
        srcs = ["foo_test.go"],
        importpath = "github.com/example/project/foo",
        embed = [":go_default_library"],
    )
    
    # External test
    go_test(
        name = "go_default_xtest",
        srcs = ["bar_test.go"],
        deps = [":go_default_library"],
        importpath = "github.com/example/project/foo",
    )
  • For each binary, add a go_binary rule like the one below.

    go_binary(
        name = "foo",
        srcs = ["main.go"],
        deps = [":go_default_library"],
    )
  • For instructions on how to depend on external libraries, see _vendoring

FAQ

Can I still use the go tool?

Yes, this setup was deliberately chosen to be compatible with go build. Make sure your project appears in GOPATH, and it should work.

Note that go build won't be aware of dependencies listed in WORKSPACE, so these will be downloaded into GOPATH. You may also need to check in generated files.

What's up with the go_default_library name?

This was used to keep import paths consistent in libraries that can be built with go build before the importpath attribute was available.

In order to compile and link correctly, rules_go must know the Go import path (the string by which a package can be imported) for each library. This is now set explicitly with the importpath attribute. Before that attribute existed, the import path was inferred by concatenating a string from a special go_prefix rule and the library's package and label name. For example, if go_prefix was github.com/example/project, for a library //foo/bar:bar, rules_go would infer the import path as github.com/example/project/foo/bar/bar. The stutter at the end is incompatible with go build, so if the label name was go_default_library, the import path would not include it. So for the library //foo/bar:go_default_library, the import path would be github.com/example/project/foo/bar.

Since go_prefix was removed and the importpath attribute became mandatory (see #721), the go_default_library name no longer serves any purpose. We may decide to stop using it in the future (see #265).

How do I access testdata?

Bazel executes tests in a sandbox, which means tests don't automatically have access to files. You must include test files using the data attribute. For example, if you want to include everything in the testdata directory:

go_test(
    name = "go_default_test",
    srcs = ["foo_test.go"],
    data = glob(["testdata/**"]),
    importpath = "github.com/example/project/foo",
)

By default, tests are run in the directory of the build file that defined them. Note that this follows the Go testing convention, not the Bazel convention followed by other languages, which run in the repository root. This means that you can access test files using relative paths. You can change the test directory using the rundir attribute. See go_test.

Gazelle will automatically add a data attribute like the one above if you have a testdata directory unless it contains buildable .go files or build files, in which case, testdata is treated as a normal package.

How do I cross-compile?

You can cross-compile by setting the --platforms flag on the command line. For example:

$ bazel build --platforms=@io_bazel_rules_go//go/toolchain:linux_amd64 //cmd

Platform-specific sources with build tags or filename suffixes are filtered automatically at compile time. You can selectively include platform-specific dependencies with select expressions (Gazelle does this automatically).

go_library(
    name = "go_default_library",
    srcs = [
        "foo_linux.go",
        "foo_windows.go",
    ],
    deps = select({
        "@io_bazel_rules_go//go/platform:linux_amd64": [
            "//bar_linux:go_default_library",
        ],
        "@io_bazel_rules_go//go/platform:windows_amd64": [
            "//bar_windows:go_default_library",
        ],
        "//conditions:default": [],
    }),
)

rules_go can generate pure Go binaries for any platform the Go SDK supports. If your project includes cgo code, has C/C++ dependencies, or requires external linking, you'll need to write a CROSSTOOL file for your toolchain and set the --cpu flag on the command line, in addition to setting --platforms. You'll also need to set pure = "off" on your go_binary. We don't fully support this yet, but people have gotten this to work in some cases.

In some cases, you may want to set the goos and goarch attributes of go_binary. This will cross-compile a binary for a specific platform. This is necessary when you need to produce multiple binaries for different platforms in a single build. However, note that select expressions will not work correctly when using these attributes.

How do I access go_binary executables from go_test?

The location where go_binary writes its executable file is not stable across rules_go versions and should not be depended upon. The parent directory includes some configuration data in its name. This prevents Bazel's cache from being poisoned when the same binary is built in different configurations. The binary basename may also be platform-dependent: on Windows, we add an .exe extension.

To depend on an executable in a go_test rule, reference the executable in the data attribute (to make it visible), then expand the location in args. The real location will be passed to the test on the command line. For example:

go_binary(
    name = "cmd",
    srcs = ["cmd.go"],
)

go_test(
    name = "cmd_test",
    srcs = ["cmd_test.go"],
    args = ["$(location :cmd)"],
    data = [":cmd"],
)

See reproducible_binary for a complete example.

Alternatively, you can set the out attribute of go_binary to a specific filename. Note that when out is set, the binary won't be cached when changing configurations.

go_binary(
    name = "cmd",
    srcs = ["cmd.go"],
    out = "cmd",
)

go_test(
    name = "cmd_test",
    srcs = ["cmd_test.go"],
    data = [":cmd"],
)

How do I run Bazel on Travis CI?

References:

In order to run Bazel tests on Travis CI, you'll need to install Bazel in the before_install script. See our configuration file linked above.

You'll want to run Bazel with a number of flags to prevent it from consuming a huge amount of memory in the test environment.

  • --host_jvm_args=-Xmx500m --host_jvm_args=-Xms500m: Set the maximum and initial JVM heap size. Keeping the same means the JVM won't spend time growing the heap. The choice of heap size is somewhat arbitrary; other configuration files recommend limits as high as 2500m. Higher values mean a faster build, but higher risk of OOM kill.
  • --bazelrc=.test-bazelrc: Use a Bazel configuration file specific to Travis CI. You can put most of the remaining options in here.
  • build --spawn_strategy=standalone --genrule_strategy=standalone: Disable sandboxing for the build. Sandboxing may fail inside of Travis's containers because the mount system call is not permitted.
  • test --test_strategy=standalone: Disable sandboxing for tests as well.
  • --local_resources=1536,1.5,0.5: Set Bazel limits on available RAM in MB, available cores for compute, and available cores for I/O. Higher values mean a faster build, but higher contention and risk of OOM kill.
  • --noshow_progress: Suppress progress messages in output for cleaner logs.
  • --verbose_failures: Get more detailed failure messages.
  • --test_output=errors: Show test stderr in the Travis log. Normally, test output is written log files which Travis does not save or report.

Downloads on Travis are relatively slow (the network is heavily contended), so you'll want to minimize the amount of network I/O in your build. Downloading Bazel and a Go SDK is a huge part of that. To avoid downloading a Go SDK, you may request a container with a preinstalled version of Go in your .travis.yml file, then call go_register_toolchains(go_version = "host") in a Travis-specific WORKSPACE file.

You may be tempted to put Bazel's cache in your Travis cache. Although this can speed up your build significantly, Travis stores its cache on Amazon, and it takes a very long time to transfer. Clean builds seem faster in practice.

How do I test a beta version of the Go SDK?

rules_go only supports official releases of the Go SDK. However, we do have an easy way for developers to try out beta releases.

In your WORKSPACE file, add a call go_download_sdk like the one below. This must be named go_sdk, and it must come before the call to go_register_toolchains.

load("@io_bazel_rules_go//go:def.bzl",
    "go_download_sdk",
    "go_register_toolchains",
    "go_rules_dependencies",
)

go_rules_dependencies()

go_download_sdk(
    name = "go_sdk",
    sdks = {
        "darwin_amd64": ("go1.10beta1.darwin-amd64.tar.gz", "8c2a4743359f4b14bcfaf27f12567e3cbfafc809ed5825a2238c0ba45db3a8b4"),
        "linux_amd64":  ("go1.10beta1.linux-amd64.tar.gz", "ec7a10b5bf147a8e06cf64e27384ff3c6d065c74ebd8fdd31f572714f74a1055"),
    },
)

go_register_toolchains()

How do I get information about the Go SDK used by rules_go?

You can run: bazel build @io_bazel_rules_go//:go_info which outputs go_info_report with information like the used Golang version.

How do I avoid conflicts with protocol buffers?

See Avoiding conflicts in the proto documentation.

How do I use a specific version of golang.org/x/sys, net or text?

Several of the golang.org/x repositories are declared in go_rules_dependencies. We declare these automatically because they're needed by gRPC.

See Overriding dependencies for an example of how to replace these with specific commits.