From 9159a307d45da9a44f06e52a31b77facb21a5fb0 Mon Sep 17 00:00:00 2001 From: Jon Brandvein Date: Thu, 10 Oct 2019 14:51:02 -0400 Subject: [PATCH 1/3] README improvements Follow-up to #237. Moves development workflow info to CONTRIBUTING.md. Fixes the instructions for how to use the Bazel Federation. Also updates the project WORKSPACE to use a released version of the Federation. --- CONTRIBUTING.md | 40 +++++++++++++++++++++++ README.md | 85 ++++++++++++++++++++++--------------------------- WORKSPACE | 6 +++- 3 files changed, 83 insertions(+), 48 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0786fdf434..8d63557c66 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -22,3 +22,43 @@ use GitHub pull requests for this purpose. Consult [GitHub Help] for more information on using pull requests. [GitHub Help]: https://help.github.com/articles/about-pull-requests/ + +## Generated files + +Some checked-in files are generated and need to be updated when a new PR is +merged. + +### Documentation + +To regenerate the content under the `docs/` directory, run this script in the +repository root: + +```shell +./update_docs.sh +``` + +This needs to be done whenever the docstrings in the corresponding .bzl files +are changed; see `docs/BUILD`. + +### Precompiled tools + +The packaging rules invoke two precompiled binaries, `tools/piptool.par` and +`tools/whltool.par`. We need these to be precompiled because they are invoked +during `WORKSPACE` evaluation, before Bazel itself is able to build anything +from source. The .par files can be regenerated by running this script in the +repository root: + +```shell +# You can pass --nodocker if Docker is not available on your system. +./update_tools.sh +``` + +This needs to be done whenever the corresponding sources in the `packaging/` +directory are updated. + +Since these are binary files and therefore unreviewable, for security +reasons1 we will regenerate the .par files for you when merging your +pull request. + +1 See "[Reflections on Trusting Trust](https://en.wikipedia.org/wiki/Backdoor_(computing)#Compiler_backdoors)". + diff --git a/README.md b/README.md index 08ea820672..b0ab38552b 100644 --- a/README.md +++ b/README.md @@ -31,21 +31,45 @@ The packaging rules (`pip_import`, etc.) are less stable. We may make breaking changes as they evolve. There are no guarantees for rules underneath the `experimental/` directory. +See the [How to contribute](CONTRIBUTING.md) page for information on our +devlopment workflow. + ## Getting started To import rules_python in your project, you first need to add it to your `WORKSPACE` file. If you are using the [Bazel -Federation](https://github.com/bazelbuild/bazel-federation), you will want to -copy the boilerplate in the rules_python release's notes, under the "WORKSPACE -setup" heading. This will look something like the following: +Federation](https://github.com/bazelbuild/bazel-federation), you just need to +[import the Federation](https://github.com/bazelbuild/bazel-federation#example-workspace) +and call the rules_python setup methods: ```python load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") + +http_archive( + name = "bazel_federation", + url = "https://github.com/bazelbuild/bazel_federation/releases/download/0.0.1/bazel_federation-0.0.1.tar.gz", + sha256 = "506dfbfd74ade486ac077113f48d16835fdf6e343e1d4741552b450cfc2efb53", +) + +load("@bazel_federation//:repositories.bzl", "rules_python_deps") + +rules_python_deps() +load("@bazel_federation//setup:rules_python.bzl", "rules_python_setup") +rules_python_setup(use_pip=True) +``` + +Note the `use_pip` argument: rules_python may be imported either with or +without support for the packaging rules. + +If you are not using the Federation, you can simply import rules_python +directly and call its initialization methods as follows: + +```python http_archive( name = "rules_python", # NOT VALID: Replace with actual version and SHA. - url = "https://github.com/bazelbuild/rules_python/releases/download//rules_python-.tar.gz", - sha256 = "", + url = "https://github.com/bazelbuild/rules_python/releases/download/0.0.1/rules_python-0.0.1.tar.gz", + sha256 = "aa96a691d3a8177f3215b14b0edc9641787abaaa30363a080165d06ab65e1161", ) load("@rules_python//python:repositories.bzl", "py_repositories") py_repositories() @@ -54,32 +78,32 @@ load("@rules_python//python:pip.bzl", "pip_repositories") pip_repositories() ``` -Otherwise, you may import rules_python in a standalone way by copying the -following: +To depend on a particular unreleased version (not recommended), you can +use `git_repository` instead of `http_archive`: ```python load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository") + git_repository( name = "rules_python", remote = "https://github.com/bazelbuild/rules_python.git", # NOT VALID: Replace with actual Git commit SHA. commit = "{HEAD}", ) -load("@rules_python//python:repositories.bzl", "py_repositories") -py_repositories() -# Only needed if using the packaging rules. -load("@rules_python//python:pip.bzl", "pip_repositories") -pip_repositories() + +# Then load and call py_repositories() and possibly pip_repositories() as +# above. ``` -Either way, you can then load the core rules in your `BUILD` files with: +Once you've imported the rule set into your WORKSPACE using any of these +methods, you can then load the core rules in your `BUILD` files with: ``` python load("@rules_python//python:defs.bzl", "py_binary") py_binary( name = "main", - ... + srcs = ["main.py"], ) ``` @@ -168,36 +192,3 @@ started](#Getting-started) above. Note that Starlark-defined bundled symbols underneath `@bazel_tools//tools/python` are also deprecated. These are not yet rewritten by buildifier. - -## Development - -### Documentation - -The content underneath `docs/` is generated. To update the documentation, -simply run this in the root of the repository: - -```shell -./update_docs.sh -``` - -### Precompiled .par files - -The `piptool.par` and `whltool.par` files underneath `tools/` are compiled -versions of the Python scripts under the `packaging/` directory. We need to -check in built artifacts because they are executed during `WORKSPACE` -evaluation, before Bazel itself is able to build anything from source. - -The .par files need to be regenerated whenever their sources are updated. This -can be done by running - -```shell -# You can pass --nodocker if Docker is not available on your system. -./update_tools.sh -``` - -from the repository root. However, since these files contain compiled code, -we do not accept commits that modify them from untrusted sources.1 -If you submit a pull request that modifies the sources and we accept the -changes, we will regenerate these files for you before merging. - -1 See "[Reflections on Trusting Trust](https://en.wikipedia.org/wiki/Backdoor_(computing)#Compiler_backdoors)". diff --git a/WORKSPACE b/WORKSPACE index 6cd3456721..786da922c9 100644 --- a/WORKSPACE +++ b/WORKSPACE @@ -16,9 +16,13 @@ workspace(name = "rules_python") load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") +# Load our production dependencies using the federation, except that instead of +# calling rules_python() (which would define the @rules_python repo), we just +# call rules_python_deps(). + http_archive( name = "bazel_federation", - url = "https://github.com/bazelbuild/bazel-federation/releases/download/0.0.1/bazel_federation-0.0.1.tar.gz", + url = "https://github.com/bazelbuild/bazel_federation/releases/download/0.0.1/bazel_federation-0.0.1.tar.gz", sha256 = "506dfbfd74ade486ac077113f48d16835fdf6e343e1d4741552b450cfc2efb53", ) From 56c4e5264b2068177c11e1cba30e72f0d2d9e111 Mon Sep 17 00:00:00 2001 From: Jon Brandvein Date: Thu, 10 Oct 2019 15:02:37 -0400 Subject: [PATCH 2/3] Minor fixes --- CONTRIBUTING.md | 2 +- README.md | 1 - WORKSPACE | 2 +- 3 files changed, 2 insertions(+), 3 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8d63557c66..d06937f716 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -42,7 +42,7 @@ are changed; see `docs/BUILD`. ### Precompiled tools -The packaging rules invoke two precompiled binaries, `tools/piptool.par` and +The packaging rules depend on two precompiled binaries, `tools/piptool.par` and `tools/whltool.par`. We need these to be precompiled because they are invoked during `WORKSPACE` evaluation, before Bazel itself is able to build anything from source. The .par files can be regenerated by running this script in the diff --git a/README.md b/README.md index b0ab38552b..ea347ec0cc 100644 --- a/README.md +++ b/README.md @@ -67,7 +67,6 @@ directly and call its initialization methods as follows: ```python http_archive( name = "rules_python", - # NOT VALID: Replace with actual version and SHA. url = "https://github.com/bazelbuild/rules_python/releases/download/0.0.1/rules_python-0.0.1.tar.gz", sha256 = "aa96a691d3a8177f3215b14b0edc9641787abaaa30363a080165d06ab65e1161", ) diff --git a/WORKSPACE b/WORKSPACE index 786da922c9..b9ccadd1cd 100644 --- a/WORKSPACE +++ b/WORKSPACE @@ -22,7 +22,7 @@ load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") http_archive( name = "bazel_federation", - url = "https://github.com/bazelbuild/bazel_federation/releases/download/0.0.1/bazel_federation-0.0.1.tar.gz", + url = "https://github.com/bazelbuild/bazel-federation/releases/download/0.0.1/bazel_federation-0.0.1.tar.gz", sha256 = "506dfbfd74ade486ac077113f48d16835fdf6e343e1d4741552b450cfc2efb53", ) From 777c978ef39457fd6aa2c73aa73bc66b8d77cfc2 Mon Sep 17 00:00:00 2001 From: Jon Brandvein Date: Fri, 18 Oct 2019 10:58:33 -0400 Subject: [PATCH 3/3] Change underscore to hyphen --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index ea347ec0cc..1c0c9da396 100644 --- a/README.md +++ b/README.md @@ -47,7 +47,7 @@ load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") http_archive( name = "bazel_federation", - url = "https://github.com/bazelbuild/bazel_federation/releases/download/0.0.1/bazel_federation-0.0.1.tar.gz", + url = "https://github.com/bazelbuild/bazel-federation/releases/download/0.0.1/bazel_federation-0.0.1.tar.gz", sha256 = "506dfbfd74ade486ac077113f48d16835fdf6e343e1d4741552b450cfc2efb53", )