Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bazel/ci: unify developer-local and CI build generation. #716

Closed
wants to merge 4 commits into from
Closed

bazel/ci: unify developer-local and CI build generation. #716

wants to merge 4 commits into from

Commits on Apr 7, 2017

  1. bazel/ci: unify developer-local and CI build generation. 

    This patch dedupes the distinct approaches that existed previously, where we assembled hand curated
BUILD files for external dependencies to be used in developer-local builds and used prebuilt
artifacts compiled under the external dependency's native build system for CI.

In the new approach, the CI flow continues to prebuild artifacts with the build recipes and
recursive make in ci/build_container/{Makefile,build_recipes}, ahead of time and prior to any
invocation of Bazel.

Developer-local builds will not prebuild, but instead invoke the same build recipes and recursive
make under a Bazel genrule. The trick here is that we can no longer use globs to refer to the build
artifacts in BUILD rules, since Bazel doesn't know about the outputs of the genrule ahead of time,
so we need to use a generated BUILD, produced by the bazel/gen_prebuilt.sh script, which does an
offline build first, analyzes the outputs and then produces a BUILD file stating what the genrule
outputs. This is only done once by a developer making an external dependency change, prior to
checking in the change.

There's a change to how external dependencies are specified as a result. Instead of adding new
external dependencies to ci/build_container/BUILD, they are added to the definitions at the start
of bazel/gen_prebuilt.py. The developer who is adding the external dependency then runs:

ENVOY_SRC_DIR=$PWD ./bazel/gen_prebuilt.sh

and checks in the resulting generated bazel/prebuilt.bzl file.

Existing users of the CI flow (Lyft) will need to make a slight tweak to how the prebuilt artifacts
are generated - the thirdparty/ and thirdparty_build/ directories will need to be generated with
the BUILD_DISTINCT=1 Makefile setting in envoyproxy#714.

A significant improvement that is also provided by this patch is the automatic workspace population
via prebuilt.bzl. See the changes to WORKSPACE and ci/WORKSPACE. Projects that consume Envoy, e.g.
to link in additional filters, no longer need to track Envoy's dependencies and maintain their own
bind rules, this is now automagic.
    @htuch
    htuch committed Apr 7, 2017
    Configuration menu
    Copy the full SHA
    7168b63 View commit details
    Browse the repository at this point in the history

  2. Use includes rather than strip_include_prefix for external deps. 

    Allows Bazel to treat it with -isystem and avoid compiler warnings for headers from deps.
    @htuch
    htuch committed Apr 7, 2017
    Configuration menu
    Copy the full SHA
    b2f0dad View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'upstream/master' into developer-local-p… 

    …rebuilt
    @htuch
    htuch committed Apr 7, 2017
    Configuration menu
    Copy the full SHA
    97ee5dd View commit details
    Browse the repository at this point in the history

  4. Move ci/build_container/build_container.sh changes to envoyproxy#722.

    @htuch
    htuch committed Apr 7, 2017
    Configuration menu
    Copy the full SHA
    6497c1d View commit details
    Browse the repository at this point in the history