diff --git a/README.md b/README.md index 7e1ac466..97275388 100644 --- a/README.md +++ b/README.md @@ -3,17 +3,31 @@ LLVM toolchain for Bazel [![Tests](https://github.com/grailbio/bazel-toolchain/a ------- -Required minimum bazel version: 4.0.0 +The project is in a relatively stable state and in use for all code development +at GRAIL and other organizations. Having said that, I am unable to give time to +it at any regular cadence. + +I rely on the community for maintenance and new feature implementations. If you +are interested in being part of this project, please let me know and I can give +you write access, so you can merge your changes directly. + +If you feel like you have a better maintained fork or an alternative/derived +implementation, please let me know and I can redirect people there. + +– @siddharthab ------- +## Quickstart + +Minimum bazel version: **4.0.0** + To use this toolchain, include this section in your WORKSPACE: ```python -# Change master to the git tag you want. http_archive( name = "com_grail_bazel_toolchain", - strip_prefix = "bazel-toolchain-master", - urls = ["https://github.com/grailbio/bazel-toolchain/archive/master.tar.gz"], + strip_prefix = "bazel-toolchain-0.6", + urls = ["https://github.com/grailbio/bazel-toolchain/archive/0.6.tar.gz"], ) load("@com_grail_bazel_toolchain//toolchain:deps.bzl", "bazel_toolchain_dependencies") @@ -24,7 +38,7 @@ load("@com_grail_bazel_toolchain//toolchain:rules.bzl", "llvm_toolchain") llvm_toolchain( name = "llvm_toolchain", - llvm_version = "8.0.0", + llvm_version = "12.0.0", ) load("@llvm_toolchain//:toolchains.bzl", "llvm_register_toolchains") @@ -32,6 +46,14 @@ load("@llvm_toolchain//:toolchains.bzl", "llvm_register_toolchains") llvm_register_toolchains() ``` +And add the following section to your .bazelrc file (not needed after +this [issue](https://github.com/bazelbuild/bazel/issues/7260) is closed): +``` +build --incompatible_enable_cc_toolchain_resolution +``` + +## Basic Usage + The toolchain can automatically detect your OS and arch type, and use the right pre-built binary distribution from llvm.org. The detection is currently based on host OS and is not perfect, so some distributions, docker based @@ -42,49 +64,136 @@ grow through community contributions. We welcome PRs! :smile: See in-code documentation in [rules.bzl](toolchain/rules.bzl) for available attributes to `llvm_toolchain`. -For making changes to default settings for these toolchains, edit the -cc_toolchain_config template. See [tutorial]( -https://github.com/bazelbuild/bazel/blob/master/site/docs/tutorial/cc-toolchain-config.md). +## Advanced Usage + +#### Customizations + +We currently offer limited customizability through attributes of the +[llvm_toolchain_\* rules](toolchain/rules.bzl). You can send us a PR to add +more configuration attributes. + +A majority of the complexity of this project is to make it generic for multiple +use cases. For one-off experiments with new architectures, cross-compilations, +new compiler features, etc., my advice would be to look at the toolchain +configurations generated by this repo, and copy-paste/edit to make your own in +any package in your own workspace. + +``` +bazel query --output=build @llvm_toolchain//:all | grep -v -e '^#' -e '^ generator' +``` + +Besides defining your toolchain in your package BUILD file, and until this +[issue](https://github.com/bazelbuild/bazel/issues/7746) is resolved, you would +also need a way for bazel to access the tools in LLVM distribution as relative +paths from your package without using `..` up-references. For this, you can +create a symlink that uses up-references to point to the LLVM distribution +directory, and also create a wrapper script for clang such that the actual +clang invocation is not through the symlinked path. See the files in the +`@llvm_toolchain//:` package as a reference. + +``` +# See generated files for reference. +ls -lR "$(bazel info output_base)/external/llvm_toolchain" + +# Create symlink to LLVM distribution. +cd _your_package_directory_ +ln -s ../....../external/llvm_toolchain_llvm llvm + +# Create CC wrapper script. +mkdir bin +cp "$(bazel info output_base)/external/llvm_toolchain/bin/cc_wrapper.sh" bin/cc_wrapper.sh +vim bin/cc_wrapper.sh # Review to ensure relative paths, etc. are good. +``` + +See [bazel +tutorial](https://docs.bazel.build/versions/main/tutorial/cc-toolchain-config.html) +for how CC toolchains work in general. + +#### Selecting Toolchains + +If toolchains are registered (see Quickstart section above), you do not need to +do anything special for bazel to find the toolchain. You may want to check once +with the `--toolchain_resolution_debug` flag to see which toolchains were +selected by bazel for your target platform. For overriding toolchains on the command line, please use the -`--extra_toolchains` flag in lieu of the deprecated `--crosstool_top` flag. -For example, `--extra_toolchains=@llvm_toolchain//:cc-toolchain-linux`. You -may need to also set the `--incompatible_enable_cc_toolchain_resolution` flag. +`--extra_toolchains` flag. For example, +`--extra_toolchains=@llvm_toolchain//:cc-toolchain-x86_64-linux`. If you would like to use the older method of selecting toolchains, you can -continue to do so with `--crosstool_top=@llvm_toolchain//:toolchain`. - -Notes: - -- The LLVM toolchain archive is downloaded and extracted as a separate - repository with the suffix `_llvm`. Alternatively, you can also specify your - own repositories for each host os-arch pair through the `toolchain_roots` - attributes. Each of these repositories is typically configured through - `local_repository` or `http_archive` (with `build_file` attribute as - `@com_grail_bazel_toolchain//toolchain:BUILD.llvm_repo`). - -- A sysroot can be specified through the `sysroot` attribute. This can be either - a path on the user's system, or a bazel `filegroup` like label. One way to - create a sysroot is to use `docker export` to get a single archive of the - entire filesystem for the image you want. Another way is to use the build - scripts provided by the - [Chromium project](https://chromium.googlesource.com/chromium/src/+/HEAD/docs/linux_sysroot.md). - -- Sandboxing the toolchain introduces a significant overhead (100ms per - action, as of mid 2018). To overcome this, one can use - `--experimental_sandbox_base=/dev/shm`. However, not all environments might - have enough shared memory available to load all the files in memory. If this - is a concern, you may set the attribute for using absolute paths, which will - substitute templated paths to the toolchain as absolute paths. When running - bazel actions, these paths will be available from inside the sandbox as part of - the / read-only mount. Note that this will make your builds non-hermetic. - -- The toolchain is known to also work with `rules_go`. - -- The LLVM toolchain also provides several tools like `clang-format`. You can - depend on these tools directly in the bin directory of the toolchain. For - example, `@llvm_toolchain_llvm//:bin/clang-format` is a valid and visible - target. +continue to do so with `--crosstool_top=@llvm_toolchain//:toolchain` instead +of the `--incompatible_enable_cc_toolchain_resolution` flag. Follow +[bazelbuild/bazel#7260](https://github.com/bazelbuild/bazel/issues/7260) for +any changes in this behavior. + +#### Bring Your Own LLVM + +The LLVM toolchain archive is downloaded and extracted as a separate repository +with the suffix `_llvm`. Alternatively, you can also specify your own +repositories for each host os-arch pair through the `toolchain_roots` +attributes. Each of these repositories is typically configured through +`local_repository` or `http_archive` (with `build_file` attribute as +`@com_grail_bazel_toolchain//toolchain:BUILD.llvm_repo`). + +#### Sysroots + +A sysroot can be specified through the `sysroot` attribute. This can be either +a path on the user's system, or a bazel `filegroup` like label. One way to +create a sysroot is to use `docker export` to get a single archive of the +entire filesystem for the image you want. Another way is to use the build +scripts provided by the [Chromium +project](https://chromium.googlesource.com/chromium/src/+/HEAD/docs/linux_sysroot.md). + +#### Cross-compilation + +The toolchain supports cross-compilation if you bring your own sysroot. When +cross-compiling, we link against the libstdc++ from the sysroot +(single-platform build behavior is to link against libc++ bundled with LLVM). +The following pairs have been tested to work for some hello-world binaries: +- {linux, x86_64} -> {linux, aarch64} +- {linux, aarch64} -> {linux, x86_64} +- {darwin, x86_64} -> {linux, x86_64} +- {darwin, x86_64} -> {linux, aarch64} + +A recommended approach would be to define two toolchains, one without sysroot +for single-platform builds, and one with sysroot for cross-compilation builds. +Then, when cross-compiling, explicitly specify the toolchain with the sysroot +and the target platform. For example, see the [WORKSPACE](WORKSPACE) file and +the [test script](tests/scripts/run_xcompile_tests.sh) for cross-compilation. +``` +bazel build \ + --platforms=@com_grail_bazel_toolchain//platforms:linux-x86_64 \ + --extra_toolchains=@llvm_toolchain_with_sysroot//:cc-toolchain-x86_64-linux \ + //... +``` + +#### Sandbox + +Sandboxing the toolchain introduces a significant overhead (100ms per action, +as of mid 2018). To overcome this, one can use +`--experimental_sandbox_base=/dev/shm`. However, not all environments might +have enough shared memory available to load all the files in memory. If this is +a concern, you may set the attribute for using absolute paths, which will +substitute templated paths to the toolchain as absolute paths. When running +bazel actions, these paths will be available from inside the sandbox as part of +the / read-only mount. Note that this will make your builds non-hermetic. + +#### Compatibility + +The toolchain is tested to work with `rules_go` and `rules_foreign_cc`. + +#### Accessing tools + +The LLVM distribution also provides several tools like `clang-format`. You can +depend on these tools directly in the bin directory of the distribution. When +using the auto-configured download (not using `toolchain_roots`), the +distribution is available in the repo with the suffix `_llvm` appended to the +name you used for the `llvm_toolchain` rule. For example, +`@llvm_toolchain_llvm//:bin/clang-format` is a valid and visible target in the +quickstart example above. + + +## Prior Art Other examples of toolchain configuration: