ci: support building recipes in distinct --prefix directories (#415). #714
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…roxy#415). The plan is to build the dependencies for the developer-local flow from the same build recipes as the CI image and using the same recursive make. This patch makes this is a bit easier, as one of the things we need to do is figure out which headers each dependency exports for Bazel, and building each recipe in its own --prefix makes this trivial. This is a nop for the CI Docker image. Validated by building a CI Docker image and performing a debug build/test against it.
mattklein123
reviewed
Apr 6, 2017
gredner
approved these changes
Apr 7, 2017
htuch
added a commit
to htuch/envoy
that referenced
this pull request
Apr 7, 2017
This patch dedupes the distinct approaches that existed previously, where we assembled hand curated
BUILD files for external dependencies to be used in developer-local builds and used prebuilt
artifacts compiled under the external dependency's native build system for CI.
In the new approach, the CI flow continues to prebuild artifacts with the build recipes and
recursive make in ci/build_container/{Makefile,build_recipes}, ahead of time and prior to any
invocation of Bazel.
Developer-local builds will not prebuild, but instead invoke the same build recipes and recursive
make under a Bazel genrule. The trick here is that we can no longer use globs to refer to the build
artifacts in BUILD rules, since Bazel doesn't know about the outputs of the genrule ahead of time,
so we need to use a generated BUILD, produced by the bazel/gen_prebuilt.sh script, which does an
offline build first, analyzes the outputs and then produces a BUILD file stating what the genrule
outputs. This is only done once by a developer making an external dependency change, prior to
checking in the change.
There's a change to how external dependencies are specified as a result. Instead of adding new
external dependencies to ci/build_container/BUILD, they are added to the definitions at the start
of bazel/gen_prebuilt.py. The developer who is adding the external dependency then runs:
ENVOY_SRC_DIR=$PWD ./bazel/gen_prebuilt.sh
and checks in the resulting generated bazel/prebuilt.bzl file.
Existing users of the CI flow (Lyft) will need to make a slight tweak to how the prebuilt artifacts
are generated - the thirdparty/ and thirdparty_build/ directories will need to be generated with
the BUILD_DISTINCT=1 Makefile setting in envoyproxy#714.
A significant improvement that is also provided by this patch is the automatic workspace population
via prebuilt.bzl. See the changes to WORKSPACE and ci/WORKSPACE. Projects that consume Envoy, e.g.
to link in additional filters, no longer need to track Envoy's dependencies and maintain their own
bind rules, this is now automagic.
|
Thanks @gredner, comments added. |
mattklein123
approved these changes
Apr 7, 2017
rshriram
pushed a commit
to rshriram/envoy
that referenced
this pull request
Oct 30, 2018
Automatic merge from submit-queue. [DO NOT MERGE] Auto PR to update dependencies of proxy This PR will be merged automatically once checks are successful. ```release-note none ```
jpsim
pushed a commit
that referenced
this pull request
Nov 28, 2022
Description: updates envoy ref to include #10137. Exposes client builder function to add dns failure refresh rate. Risk Level: low, new configuration option Testing: local apps, CI Docs Changes: created #715 to create a section in the docs for all the builder options. Fixes #673 Signed-off-by: JP Simard <jp@jpsim.com>
jpsim
pushed a commit
that referenced
this pull request
Nov 29, 2022
Description: updates envoy ref to include #10137. Exposes client builder function to add dns failure refresh rate. Risk Level: low, new configuration option Testing: local apps, CI Docs Changes: created #715 to create a section in the docs for all the builder options. Fixes #673 Signed-off-by: JP Simard <jp@jpsim.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The plan is to build the dependencies for the developer-local flow from the same build recipes as
the CI image and using the same recursive make. This patch makes this is a bit easier, as one of the
things we need to do is figure out which headers each dependency exports for Bazel, and building
each recipe in its own --prefix makes this trivial.
This is a nop for the CI Docker image. Validated by building a CI Docker image and performing a
debug build/test against it.