Python Scala Java Shell Thrift
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.bazelci Add CI config to test on RBE. (#578) Aug 13, 2018
jmh Compile with unused dependency checker (#579) Aug 14, 2018
junit Format BUILD files and check format during CI (#489) Jun 12, 2018
private Format BUILD files and check format during CI (#489) Jun 12, 2018
scala Propagate source jars through to IntelliJ metadata. Aug 18, 2018
scala_proto Compile with unused dependency checker (#579) Aug 14, 2018
scripts moved plugin under third_party in ij.bazelproject Aug 8, 2017
specs2 Support for selecting scala version (#544) Jul 17, 2018
src Compile with unused dependency checker (#579) Aug 14, 2018
test Compile with unused dependency checker (#579) Aug 14, 2018
test_expect_failure Unused dependency checking (#568) Aug 7, 2018
test_version Compile with unused dependency checker (#579) Aug 14, 2018
third_party Compile with unused dependency checker (#579) Aug 14, 2018
thrift Aspect-based scrooge implementation (#524) Jun 28, 2018
tools Bump to using min version of 0.14.1 (#541) Jun 27, 2018
tut_rule Support for selecting scala version (#544) Jul 17, 2018
twitter_scrooge Fix Intellij Bazel Plugin support for scrooge_scala_library rule (#562) Jul 24, 2018
.bazelrc.travis Bump to using min version of 0.14.1 (#541) Jun 27, 2018
.gitignore Enable local cache by default (#478) Apr 11, 2018
.travis.yml Update bazel to 0.16.1 (#586) Aug 16, 2018
AUTHORS Add missing authors Feb 23, 2016
CONTRIBUTING.md Format BUILD files and check format during CI (#489) Jun 12, 2018
CONTRIBUTORS Use a scala compiler plugin to throw errors on missing direct-deps (#243 Jul 25, 2017
LICENSE.txt Add boilerplate for open sourcing Feb 17, 2016
README.md Update bazel to 0.16.1 (#586) Aug 16, 2018
WORKSPACE Compile with unused dependency checker (#579) Aug 14, 2018
lint.sh Format BUILD files and check format during CI (#489) Jun 12, 2018
test_all.sh Add notes about updates and usage with bazel-deps to readme (#566) Aug 6, 2018
test_intellij_aspect.sh added scala_toolchain with scalacopts support (#364) Jan 4, 2018
test_lint.sh Format BUILD files and check format during CI (#489) Jun 12, 2018
test_reproducibility.sh Bump to using min version of 0.14.1 (#541) Jun 27, 2018
test_rules_scala.sh Unused dependency checking (#568) Aug 7, 2018
test_runner.sh Integrate intellij scala aspect integration testing into rules_scala (#… Dec 27, 2017
test_version.sh Support for selecting scala version (#544) Jul 17, 2018

README.md

Scala Rules for Bazel

Build Status Build status Gitter chat

Overview

This rule is used for building Scala projects with Bazel. There are currently four rules, scala_library, scala_macro_library, scala_binary and scala_test.

Getting started

In order to use scala_library, scala_macro_library, and scala_binary, you must have bazel 0.5.3 or later and add the following to your WORKSPACE file:

rules_scala_version="63eab9f4d80612e918ba954211f377cc83d27a07" # update this as needed

http_archive(
             name = "io_bazel_rules_scala",
             url = "https://github.com/bazelbuild/rules_scala/archive/%s.zip"%rules_scala_version,
             type = "zip",
             strip_prefix= "rules_scala-%s" % rules_scala_version
             )

load("@io_bazel_rules_scala//scala:scala.bzl", "scala_repositories")
scala_repositories()

To use a particular tag, use the tagged number in tag = and omit the commit attribute. Note that these plugins are still evolving quickly, as is bazel, so you may need to select the version most appropriate for you.

In addition, you must register scala_toolchain - To register default empty toolcahin simply add those lines to WORKSPACE file:

load("@io_bazel_rules_scala//scala:toolchains.bzl", "scala_register_toolchains")
scala_register_toolchains()

read more here

Then in your BUILD file just add the following so the rules will be available:

load("@io_bazel_rules_scala//scala:scala.bzl", "scala_library", "scala_binary", "scala_test")

You may wish to have these rules loaded by default using bazel's prelude. You can add the above to the file tools/build_rules/prelude_bazel in your repo (don't forget to have a, possibly empty, BUILD file there) and then it will be automatically prepended to every BUILD file in the workspace.

To run with a persistent worker (much faster), you need to add

build --strategy=Scalac=worker
test --strategy=Scalac=worker

to your command line, or to enable by default for building/testing add it to your .bazelrc.

Selecting Scala version

Rules scala supports all minor versions of Scala 2.11/2.12. By default Scala 2.11.12 is used and to use another version you need to specify it when calling scala_repositories. scala_repositories takes a tuple (scala_version, scala_version_jar_shas) as a parameter where scala_version is the scala version and scala_version_jar_shas is a dict with sha256 hashes for the maven artifacts scala_library, scala_reflect and scala_compiler:

scala_repositories(("2.12.6", {
    "scala_compiler": "3023b07cc02f2b0217b2c04f8e636b396130b3a8544a8dfad498a19c3e57a863",
    "scala_library": "f81d7144f0ce1b8123335b72ba39003c4be2870767aca15dd0888ba3dab65e98",
    "scala_reflect": "ffa70d522fc9f9deec14358aa674e6dd75c9dfa39d4668ef15bb52f002ce99fa"
}))

If you're using any of the rules twitter_scrooge, tut_repositories, scala_proto_repositories or specs2_junit_repositories you also need to specify scala_version for them. See ./test_version/WORKSPACE.template for an example workspace using another scala version.

Bazel compatible versions

bazel rules_scala gitsha
0.16.x HEAD
0.15.x 3b9ab9be31ac217d3337c709cb6bfeb89c8dcbb1
0.14.x 3b9ab9be31ac217d3337c709cb6bfeb89c8dcbb1
0.13.x 3c987b6ae8a453886759b132f1572c0efca2eca2

scala_library / scala_macro_library

scala_library(name, srcs, deps, runtime_deps, exports, data, main_class, resources, resource_strip_prefix, scalacopts, jvm_flags, scalac_jvm_flags, javac_jvm_flags, unused_dependency_checker_mode)
scala_macro_library(name, srcs, deps, runtime_deps, exports, data, main_class, resources, resource_strip_prefix, scalacopts, jvm_flags, scalac_jvm_flags, javac_jvm_flags, unused_dependency_checker_mode)

scala_library generates a .jar file from .scala source files. This rule also creates an interface jar to avoid recompiling downstream targets unless their interface changes.

scala_macro_library generates a .jar file from .scala source files when they contain macros. For macros, there are no interface jars because the macro code is executed at compile time. For best performance, you want very granular targets until such time as the zinc incremental compiler can be supported.

In order to make a java rule use this jar file, use the java_import rule.

Attributes
name

Name, required

A unique name for this target

srcs

List of labels, required

List of Scala .scala source files used to build the library. These may be .srcjar jar files that contain source code.

deps

List of labels, optional

List of other libraries to linked to this library target. These must be jvm targets (scala_library, java_library, java_import, etc...)

runtime_deps

List of labels, optional

List of other libraries to put on the classpath only at runtime. This is rarely needed in Scala. These must be jvm targets (scala_library, java_library, java_import, etc...)

exports

List of labels, optional

List of targets to add to the dependencies of those that depend on this target. Similar to the `java_library` parameter of the same name. Use this sparingly as it weakens the precision of the build graph. These must be jvm targets (scala_library, java_library, java_import, etc...)

data

List of labels, optional

List of files needed by this rule at runtime.

main_class

String, optional

Name of class with main() method to use as an entry point

The value of this attribute is a class name, not a source file. The class must be available at runtime: it may be compiled by this rule (from srcs) or provided by direct or transitive dependencies (through deps). If the class is unavailable, the binary will fail at runtime; there is no build-time check.

resources

List of labels; optional

A list of data files to be included in the JAR.

resource_strip_prefix

String; optional

The path prefix to strip from Java resources. If specified, this path prefix is stripped from every file in the `resources` attribute. It is an error for a resource file not to be under this directory.

scalacopts

List of strings; optional

Extra compiler options for this library to be passed to scalac. Subject to Make variable substitution and Bourne shell tokenization.

jvm_flags

List of strings; optional; deprecated

Deprecated, superseded by scalac_jvm_flags and javac_jvm_flags. Is not used and is kept as backwards compatibility for the near future. Effectively jvm_flags is now an executable target attribute only.

scalac_jvm_flags

List of strings; optional

List of JVM flags to be passed to scalac after the scalacopts. Subject to Make variable substitution and Bourne shell tokenization.

javac_jvm_flags

List of strings; optional

List of JVM flags to be passed to javac after the javacopts. Subject to Make variable substitution and Bourne shell tokenization.

unused_dependency_checker_mode

String; optional

Enable unused dependency checking (see Unused dependency checking). Possible values are: off, warn and error.

scala_binary

scala_binary(name, srcs, deps, runtime_deps, data, main_class, resources, resource_strip_prefix, scalacopts, jvm_flags, scalac_jvm_flags, javac_jvm_flags, unused_dependency_checker_mode)

scala_binary generates a Scala executable. It may depend on scala_library, scala_macro_library and java_library rules.

A scala_binary requires a main_class attribute.

Attributes
name

Name, required

A unique name for this target

srcs

List of labels, required

List of Scala .scala source files used to build the binary

deps

List of labels, optional

List of other libraries to linked to this binary target. These must be jvm targets (scala_library, java_library, java_import, etc...)

runtime_deps

List of labels, optional

List of other libraries to put on the classpath only at runtime. This is rarely needed in Scala. These must be jvm targets (scala_library, java_library, java_import, etc...)

data

List of labels, optional

List of files needed by this rule at runtime.

main_class

String, required

Name of class with main() method to use as an entry point

The value of this attribute is a class name, not a source file. The class must be available at runtime: it may be compiled by this rule (from srcs) or provided by direct or transitive dependencies (through deps). If the class is unavailable, the binary will fail at runtime; there is no build-time check.

resources

List of labels; optional

A list of data files to be included in the JAR.

resource_strip_prefix

String; optional

The path prefix to strip from Java resources. If specified, this path prefix is stripped from every file in the `resources` attribute. It is an error for a resource file not to be under this directory.

scalacopts

List of strings; optional

Extra compiler options for this binary to be passed to scalac. Subject to Make variable substitution and Bourne shell tokenization.

jvm_flags

List of strings; optional

List of JVM flags to be passed to the executing JVM. Subject to Make variable substitution and Bourne shell tokenization.

scalac_jvm_flags

List of strings; optional

List of JVM flags to be passed to scalac after the scalacopts. Subject to Make variable substitution and Bourne shell tokenization.

javac_jvm_flags

List of strings; optional

List of JVM flags to be passed to javac after the javacopts. Subject to Make variable substitution and Bourne shell tokenization.

unused_dependency_checker_mode

String; optional

Enable unused dependency checking (see Unused dependency checking). Possible values are: off, warn and error.

scala_test

scala_test(name, srcs, suites, deps, data, main_class, resources, resource_strip_prefix, scalacopts, jvm_flags, scalac_jvm_flags, javac_jvm_flags, unused_dependency_checker_mode)

scala_test generates a Scala executable which runs unit test suites written using the scalatest library. It may depend on scala_library, scala_macro_library and java_library rules.

A scala_test by default runs all tests in a given target. For backwards compatibility it accepts a suites attribute which is ignored due to the ease with which that field is not correctly populated and tests are not run.

scala_repl

scala_repl(name, deps, scalacopts, jvm_flags, scalac_jvm_flags, javac_jvm_flags, unused_dependency_checker_mode)

A scala repl allows you to add library dependencies (not currently scala_binary targets) to generate a script to run which starts a REPL. Since bazel run closes stdin, it cannot be used to start the REPL. Instead, you use bazel build to build the script, then run that script as normal to start a REPL session. An example in this repo:

bazel build test:HelloLibRepl
bazel-bin/test/HelloLibRepl

scala_library_suite

The scala library suite allows you to define a glob or series of targets to generate sub scala libraries for. The outer target will export all of the inner targets. This allows splitting up of a series of independent files in a larger target into smaller ones. This lets us cache outputs better and also build the individual targets in parallel. Downstream targets should not be aware of its presence.

scala_test_suite

The scala test suite allows you to define a glob or series of targets to generate sub scala tests for. The outer target defines a native test suite to run all the inner tests. This allows splitting up of a series of independent tests from one target into several. This lets us cache outputs better and also build and test the individual targets in parallel.

thrift_library

load("@io_bazel_rules_scala//thrift:thrift.bzl", "thrift_library")
thrift_library(name, srcs, deps, absolute_prefix, absolute_prefixes)

thrift_library generates a thrift source zip file. It should be consumed by a thrift compiler like scrooge_scala_library (in its deps).

Attributes
name

Name, required

A unique name for this target

srcs

List of labels, required

List of Thrift .thrift source files used to build the target

deps

List of labels, optional

List of other thrift dependencies that this thrift depends on. Also can include `scroogle_scala_import` targets, containing additional `thrift_jars` (which will be compiled) and/or `scala_jars` needed at compile time (such as Finagle).

absolute_prefix

string; optional (deprecated in favor of absolute_prefixes)

This string acts as a wildcard expression of the form *`string_value` that is removed from the start of the path. Example: thrift is at `a/b/c/d/e/A.thrift` , prefix of `b/c/d`. Will mean other thrift targets can refer to this thrift at `e/A.thrift`.

absolute_prefixes

List of strings; optional

Each of these strings acts as a wildcard expression of the form *string_value that is removed from the start of the path. Example: thrift is at a/b/c/d/e/A.thrift , prefix of b/c/d. Will mean other thrift targets can refer to this thrift at e/A.thrift. Exactly one of these must match all thrift paths within the target, more than one or zero will fail the build. The main use case to have several here is to make a macro target you can share across several indvidual thrift_library, if source path is /src/thrift or /src/main/thrift it can strip off the prefix without users needing to configure it per target.

scalapb_proto_library

You first need to add the following to your WORKSPACE file:

load("@io_bazel_rules_scala//scala_proto:scala_proto.bzl", "scala_proto_repositories")
scala_proto_repositories()

Then you can import scalapb_proto_library in any BUILD file like this:

load("@io_bazel_rules_scala//scala_proto:scala_proto.bzl", "scalapb_proto_library")
scalapb_proto_library(name, deps, with_grpc, with_java, with_flat_package, with_single_line_to_string)

scalapb_proto_library generates a scala library of scala proto bindings generated by the ScalaPB compiler.

Attributes
name

Name, required

A unique name for this target

deps

List of labels, required

List of dependencies for this target. Must either be of type proto_library or java_proto_library (allowed only if with_java is enabled)

with_grpc

boolean; optional (default False)

Enables generation of grpc service bindings for services defined in deps

with_java

boolean; optional (default False)

Enables generation of converters to and from java protobuf bindings. If you set this to True make sure that you pass the corresponding java_proto_library target in deps

with_flat_package

boolean; optional (default False)

When true, ScalaPB will not append the protofile base name to the package name

with_single_line_to_string

boolean; optional (default False)

Enables generation of toString() methods that use the single line format

scala_toolchain

Scala toolchain allows you to define global configuration to all scala targets. Currently the only option that can be set is scalacopts but the plan is to expand it to other options as well.

some scala_toolchain must be registered!

Several options to configure scala_toolchain:

A) Use default scala_toolchain:

In your workspace file add the following lines:

# WORKSPACE
# register default scala toolchain
load("@io_bazel_rules_scala//scala:toolchains.bzl", "scala_register_toolchains")
scala_register_toolchains()

B) Defining your own scala_toolchain requires 2 steps:

  1. Add your own definition to scala_toolchain to a BUILD file:
# //toolchains/BUILD
load("//scala:scala_toolchain.bzl", "scala_toolchain")

scala_toolchain(
    name = "my_toolchain_impl",
    scalacopts = ["-Ywarn-unused"],
    unused_dependency_checker_mode = "off",
    visibility = ["//visibility:public"]
)

toolchain(
    name = "my_scala_toolchain",
    toolchain_type = "@io_bazel_rules_scala//scala:toolchain_type",
    toolchain = "my_toolchain_impl",
    visibility = ["//visibility:public"]
)
  1. register your custom toolchain from WORKSPACE:
# WORKSPACE
# ...
register_toolchains("//toolchains:my_scala_toolchain")

Usage with bazel-deps

Bazel-deps allows you to generate bazel dependencies transitively for maven artifacts. Generally we don't want bazel-deps to fetch scala artifacts from maven but instead use the ones we get from calling scala_repositories. The artifacts can be overridden in the dependencies file used by bazel-deps:

replacements:
  org.scala-lang:
    scala-library:
      lang: scala/unmangled
      target: "@io_bazel_rules_scala_scala_library//:io_bazel_rules_scala_scala_library"
    scala-reflect:
      lang: scala/unmangled
      target: "@io_bazel_rules_scala_scala_reflect//:io_bazel_rules_scala_scala_reflect"
    scala-compiler:
      lang: scala/unmangled
      target: "@io_bazel_rules_scala_scala_compiler//:io_bazel_rules_scala_scala_compiler"

  org.scala-lang.modules:
    scala-parser-combinators:
      lang: scala
      target:
        "@io_bazel_rules_scala_scala_parser_combinators//:io_bazel_rules_scala_scala_parser_combinators"
    scala-xml:
      lang: scala
      target:
        "@io_bazel_rules_scala_scala_xml//:io_bazel_rules_scala_scala_xml"

[Experimental] Using strict-deps

Bazel pushes towards explicit and minimal dependencies to keep BUILD file hygiene and allow for targets to refactor their dependencies without fear of downstream breaking. Currently rules_scala does this at the cost of having cryptic scalac errors when one mistakenly depends on a transitive dependency or, as more often the case for some, a transitive dependency is needed to please scalac itself. To learn more about the motivation of strict-deps itself you can visit this Bazel blog post on the subject.

To use it just add --strict_java_deps=WARN|ERROR to your bazel invocation. In both cases of WARN or ERROR you will get the following text in the event of a violation:

...
Target '//some_package:transitive_dependency' is used but isn't explicitly declared, please add it to the deps.
You can use the following buildozer command:
buildozer 'add deps //some_package:transitive_dependency' //some_other_package:transitive_dependency_user

Note that if you have buildozer installed you can just run the last line and have it automatically apply the fix for you.

Caveats:

  • Extra builds- when strict-deps is on the transitive dependencies are inputs to the compilation action which means you can potentially have more build triggers for changes the cross the ijar boundary
  • Label propagation- since label of targets are needed for the clear message and since it's not currently supported by JavaInfo from bazel we manually propagate it. This means that the error messages have a significantly lower grade if you don't use one of the scala rules or scala_import (since they don't propagate these labels)
  • javac outputs incorrect targets due to a problem we're tracing down. Practically we've noticed it's pretty trivial to understand the correct target (i.e. it's almost a formatting problem)

Note: Currently strict-deps is protected by a feature toggle but we're strongly considering making it the default behavior as java_* rules do.

[Experimental] Unused dependency checking

To allow for better caching and faster builds we want to minimize the direct dependencies of our targets. Unused dependency checking makes sure that all targets specified as direct dependencies are actually used. If unused_dependency_checker_mode is set to either error or warn you will get the following message for any dependencies that are not used:

error: Target '//some_package:unused_dep' is specified as a dependency to //target:target but isn't used, please remove it from the deps.
You can use the following buildozer command:
buildozer 'remove deps //some_package:unused_dep' //target:target

Currently unused dependency checking and strict-deps can't be used simultaneously, if both are set only strict-deps will run.

Unused dependency checking can either be enabled globally for all targets using a scala toolchain or for individual targets using the unused_dependency_checker_mode attribute. The feature is still experimental and there can thus be cases where it works incorrectly, in these cases you can enable unused dependency checking globally through a toolchain and override individual misbehaving targets using the attribute.

Building from source

Test & Build:

bash test_all.sh

You can also use:

bazel test //test/...

Note bazel test //... will not work since we have a sub-folder on the root folder which is meant to be used in a failure scenario in the integration tests. Similarly to only build you should use bazel build //src/... due to that folder.

Updates

This section contains a list of updates that might require action from the user.

Contributing

See CONTRIBUTING.md for more info.