[TOC]
Bazel is an open-source build and test tool similar to Make.
Bazel builds software from source code organized in a directory called a workspace. Source files in the workspace are organized in a nested hierarchy of packages, where each package is a directory that contains a set of related source files and one BUILD file. The BUILD file specifies what software outputs can be built from the source.
A workspace is a directory on your filesystem that contains the source files for the software you want to build, as well as symbolic links to directories that contain the build outputs.
Each workspace directory has a text file named WORKSPACE which may be empty, or may contain references to external dependencies required to build the outputs.
A package is defined as a directory containing a file named BUILD, residing beneath the top-level directory in the workspace.
A package includes all files in its directory, plus all subdirectories beneath it, except those which themselves contain a BUILD file.
The elements of a package are called targets. Most targets are one of two principal kinds, files and rules.
All targets belong to exactly one package. The name of a target is called its label, and a typical label in canonical form looks like this:
//path/to/package:target
for example:
//my/app/main:app_binaryEach label has two parts, a package name (my/app/main) and a target name (app_binary).
The label for a file without package:
//:path/to/file
WORKSPACE rules (mainly for external dependencies) include:
- bind
- git_repository
- http_archive
- http_file
- http_jar
- local_repository
- maven_jar
- maven_server
- new_git_repository
- new_http_archive
- new_local_repository
- xcode_config
- xcode_version
Refer to:
Every rule has a name, specified by the name attribute, of type string.
BUILD rules include:
- filegroup
- genquery
- test_suite
- alias
- config_setting
- genrule
Refer to:
A build consists of three phases.
- Loading phase. First, we load and evaluate all extensions and all BUILD files that are needed for the build. The execution of the BUILD files simply instantiates rules (each time a rule is called, it gets added to a graph). This is where macros are evaluated.
- Analysis phase. The code of the rules is executed (their implementation function), and actions are instantiated. An action describes how to generate a set of outputs from a set of inputs, e.g. "run gcc on hello.c and get hello.o". It is important to note that we have to list explicitly which files will be generated before executing the actual commands. In other words, the analysis phase takes the graph generated by the loading phase and generates an action graph.
- Execution phase. Actions are executed, when at least one of their outputs is required. If a file is missing or if a command fails to generate one output, the build fails. Tests are also run during this phase.
- -s: show subcommands
- --test_output=<summary|errors|all|streamed> (default: "summary")
Ref: https://docs.bazel.build/versions/master/test-encyclopedia.html
--test_timeout
(a single integer or comma-separated list of 4 integers; default: "-1")
Override the default test timeout values for test timeouts (in secs). If a
single positive integer value is specified it will override all
categories. If 4 comma-separated integers are specified, they will
override the timeouts for short, moderate, long and eternal (in that
order). In either form, a value of -1 tells blaze to use its default
timeouts for that category.
Generated output directories:
bazel-soc -> /data/.../execroot/soc #symlink to execroot, work dir for all actions
bazel-out -> /data/.../execroot/soc/bazel-out #symlink to output path
bazel-bin -> /data/.../execroot/soc/bazel-out/k8-fastbuild/bin #symlink to written bin dir
bazel-genfiles -> /data/.../execroot/soc/bazel-out/k8-fastbuild/genfiles #symlink to written genfile dir
bazel-testlogs -> /data/.../execroot/soc/bazel-out/k8-fastbuild/testlogs #symlink to test logs
Refer to:
Bazel extensions are files ending in .bzl. Use the load statement to import a symbol from an extension.
load("//build_tools/rules:maprule.bzl", "maprule")This code will load the file build_tools/rules/maprule.bzl and add the maprule symbol to the environment.
This can be used to load new rules, functions or constants (e.g. a string, a list, etc.).
Skylark, the language used in Bazel.
A macro is a function called from the BUILD file that can instantiate rules. Macros don't give additional power, they are just used for encapsulation and code reuse.
Native rules (i.e. rules that don't need a load() statement) can be instantiated from the native module:
def my_macro(name, visibility=None):
native.cc_library(
name = name,
srcs = ["main.cc"],
visibility = visibility,
)Refer to:
A rule defines a series of actions that Bazel should perform on inputs to get a set of outputs.
In a .bzl file, use the rule function to create a new rule and store it in a global variable:
def _empty_impl(ctx):
# This function is called when the rule is analyzed.
print("This rule does nothing")
empty = rule(implementation = _empty_impl)The rule can then be loaded in BUILD files:
load("//empty:empty.bzl", "empty")
# Run 'bazel build :nothing'. The target will be successfully analyzed
empty(name = "nothing")Refer to:
- docs: https://docs.bazel.build/versions/master/skylark/rules.html
- examples: https://github.com/bazelbuild/examples/tree/master/rules
An attribute is a rule argument, such as srcs or deps. You must list the attributes and their types when you define a rule. Create attributes using the attr module.
def _impl(ctx):
# Print debug information about the target.
print("Target {} has {} deps".format(ctx.label, len(ctx.attr.deps)))
# For each target in deps, print its label and files.
for i, d in enumerate(ctx.attr.deps):
print(" {}. label = {}".format(i + 1, d.label))
# A label can represent any number of files (possibly 0).
print(" files = " + str([f.path for f in d.files.to_list()]))
# For debugging, consider using `dir` to explore the existing fields.
print(dir(ctx)) # prints all the fields and methods of ctx
print(dir(ctx.attr)) # prints all the attributes of the rule
printer = rule(
implementation = _impl,
attrs = {
# Do not declare "name": It is added automatically.
"number": attr.int(default = 1),
"deps": attr.label_list(allow_files = True),
},
)The most common way to access attribute values is by using ctx.attr.<attribute_name>, though there are several other fields besides attr that provide more convenient ways of accessing file handles, such as ctx.file and ctx.outputs. The name and the package of a rule are available with ctx.label.name and ctx.label.package.
Every rule requires an implementation function. Rule implementation functions are usually private (i.e., named with a leading underscore) because they tend not to be reused.
Implementation functions take exactly one parameter: a rule context, conventionally named ctx. It can be used to:
- access attribute values and obtain handles on declared input and output files;
- create actions; and
- pass information to other targets that depend on this one, via providers.
Each call to a build rule has the side effect of defining a new target (also called instantiating the rule).
An action describes how to generate a set of outputs from a set of inputs.
All functions that create actions are defined in ctx.actions:
- ctx.actions.run, to run an executable.
- ctx.actions.run_shell, to run a shell command.
- ctx.actions.write, to write a string to a file.
- ctx.actions.expand_template, to generate a file from a template.
For example, in a .bzl file, use the rule function to create a new rule:
"""Execute a binary.
The example below executes the binary target "//actions_run:merge" with
some arguments. The binary will be automatically built by Bazel.
The rule must declare its dependencies. To do that, we pass the target to
the attribute "_merge_tool". Since it starts with an underscore, it is private
and users cannot redefine it.
"""
def _impl(ctx):
# The list of arguments we pass to the script.
args = [ctx.outputs.out.path] + [f.path for f in ctx.files.chunks]
# Action to call the script.
ctx.actions.run(
inputs = ctx.files.chunks,
outputs = [ctx.outputs.out],
arguments = args,
progress_message = "Merging into %s" % ctx.outputs.out.short_path,
executable = ctx.executable._merge_tool,
)
concat = rule(
implementation = _impl,
attrs = {
"chunks": attr.label_list(allow_files = True),
"out": attr.output(mandatory = True),
"_merge_tool": attr.label(
executable = True,
cfg = "host",
allow_files = True,
default = Label("//actions_run:merge"),
),
},
)The rule can then be loaded in BUILD files:
load(":execute.bzl", "concat")
concat(
name = "sh",
out = "page.html",
chunks = [
"header.html",
"body.html",
"footer.html",
],
)
# This target is used by the shell rule.
sh_binary(
name = "merge",
srcs = ["merge.sh"],
)Providers are pieces of information that a rule exposes to other rules that depend on it.
Providers are the only mechanism to exchange data between rules, and can be thought of as part of a rule's public interface (loosely analogous to a function's return value).
A rule can only see the providers of its direct dependencies.
For example, in a .bzl file, create a rule with a mandatory provider:
"""Rule with a mandatory provider.
In this example, rules have a number attribute. Each rule adds its number
with the numbers of its transitive dependencies, and write the result in a
file. This shows how to transfer information from a dependency to its
dependents.
"""
NumberInfo = provider("number")
def _impl(ctx):
result = ctx.attr.number
for dep in ctx.attr.deps:
result += dep[NumberInfo].number
ctx.file_action(output = ctx.outputs.out, content = str(result))
# Return the provider with result, visible to other rules.
return [NumberInfo(number = result)]
sum = rule(
implementation = _impl,
attrs = {
"number": attr.int(default = 1),
# All deps must provide all listed providers.
"deps": attr.label_list(providers = [NumberInfo]),
},
outputs = {"out": "%{name}.sum"},
)The rule can then be loaded in BUILD files:
load(":sum.bzl", "sum")
sum(
name = "n",
deps = [
":n2",
":n5",
],
)
sum(
name = "n2",
number = 2,
)
sum(
name = "n5",
number = 5,
)Runfiles are a set of files used by the (often executable) output of a rule during runtime (as opposed to build time, i.e. when the binary itself is generated). During the execution phase, Bazel creates a directory tree containing symlinks pointing to the runfiles. This stages the environment for the binary so it can access the runfiles during runtime.
Runfiles can be added manually during rule creation and/or collected transitively from the rule's dependencies:
def _rule_implementation(ctx):
...
transitive_runfiles = depset(transitive=
[dep.transitive_runtime_files for dep in ctx.attr.special_dependencies])
runfiles = ctx.runfiles(
# Add some files manually.
files = [ctx.file.some_data_file],
# Add transitive files from dependencies manually.
transitive_files = transitive_runfiles,
# Collect runfiles from the common locations: transitively from srcs,
# deps and data attributes.
collect_default = True,
)
# Add a field named "runfiles" to the DefaultInfo provider in order to actually
# create the symlink tree.
return [DefaultInfo(runfiles=runfiles)]Depsets are a specialized data structure for efficiently collecting data across a target’s transitive dependencies.
The main feature of depsets is that they support a time- and space-efficient merge operation, whose cost is independent of the size of the existing contents. Depsets also have well-defined ordering semantics.
For example, in a .bzl file:
# A provider with one field, transitive_sources.
FooFiles = provider(fields = ["transitive_sources"])
def get_transitive_srcs(srcs, deps):
"""Obtain the source files for a target and its transitive dependencies.
Args:
srcs: a list of source files
deps: a list of targets that are direct dependencies
Returns:
a collection of the transitive sources
"""
return depset(
srcs,
transitive = [dep[FooFiles].transitive_sources for dep in deps])
def _foo_library_impl(ctx):
trans_srcs = get_transitive_srcs(ctx.files.srcs, ctx.attr.deps)
return [FooFiles(transitive_sources=trans_srcs)]
foo_library = rule(
implementation = _foo_library_impl,
attrs = {
"srcs": attr.label_list(allow_files=True),
"deps": attr.label_list(),
},
)
def _foo_binary_impl(ctx):
foocc = ctx.executable._foocc
out = ctx.outputs.out
trans_srcs = get_transitive_srcs(ctx.files.srcs, ctx.attr.deps)
srcs_list = trans_srcs.to_list()
ctx.actions.run(executable = foocc,
arguments = [out.path] + [src.path for src in srcs_list],
inputs = srcs_list + [foocc],
outputs = [out])
foo_binary = rule(
implementation = _foo_binary_impl,
attrs = {
"srcs": attr.label_list(allow_files=True),
"deps": attr.label_list(),
"_foocc": attr.label(default=Label("//depsets:foocc"),
allow_files=True, executable=True, cfg="host")
},
outputs = {"out": "%{name}.out"},
)If you don't need the merge operation, consider using another type, such as list or dict.
Refer to:
Aspects allow augmenting build dependency graphs with additional information and actions.
Aspects are similar to rules in that they have an implementation function that generates actions and returns providers. However, their power comes from the way the dependency graph is built for them. An aspect has an implementation and a list of all attributes it propagates along.
The following example demonstrates using an aspect from a target rule that recursively counts files in targets, potentially filtering them by extension. It shows how to use a provider to return values, how to use parameters to pass an argument into an aspect implementation, and how to invoke an aspect from a rule.
FileCount.bzl file:
# provider
FileCount = provider(
# It is best practice to explicitly define the fields of a provider
# using the 'fields' attribute
fields = {
'count' : 'number of files'
}
)
# aspect implementation
# takes two arguments:
# * target - the target the aspect is being applied to
# * ctx - ctx object that can be used to access attributes and generate outputs and actions
def _file_count_aspect_impl(target, ctx):
count = 0
# Make sure the rule has a srcs attribute.
if hasattr(ctx.rule.attr, 'srcs'):
# Iterate through the sources counting files
for src in ctx.rule.attr.srcs:
for f in src.files:
if ctx.attr.extension == '*' or ctx.attr.extension == f.extension:
count = count + 1
# Get the counts from our dependencies.
for dep in ctx.rule.attr.deps:
count = count + dep[FileCount].count
# aspects are required to return a list of providers
return [FileCount(count = count)]
# aspect definition
file_count_aspect = aspect(implementation = _file_count_aspect_impl,
# 'attr_aspects' is a list of rule attributes along which the aspect propagates
attr_aspects = ['deps'],
# 'attrs' defines a set of attributes
attrs = {
'extension' : attr.string(values = ['*', 'h', 'cc']),
}
)
# invoke the aspect from a rule
def _file_count_rule_impl(ctx):
for dep in ctx.attr.deps:
print(dep[FileCount].count)
file_count_rule = rule(
implementation = _file_count_rule_impl,
attrs = {
'deps' : attr.label_list(aspects = [file_count_aspect]),
# set default value of parameter
'extension' : attr.string(default = '*'),
},
)BUILD.bazel file:
load('//file_count.bzl', 'file_count_rule')
cc_library(
name = 'lib',
srcs = [
'lib.h',
'lib.cc',
],
)
cc_binary(
name = 'app',
srcs = [
'app.h',
'app.cc',
'main.cc',
],
deps = ['lib'],
)
file_count_rule(
name = 'file_count',
deps = ['app'],
extension = 'h',
)Refer to: