Update the policy-controller release build process #6672
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kube v0.59 depends on k8s-openapi v0.13, which includes breaking changes. This change updates these dependencies and modifies our code to account for these changes. Furthermore, we now use the k8s-openapi feature `v1_16` so that we use an API version that is compatible with Linkerd's minimum support kubernetes version. Closes #6657 #6658 #6659
The policy-controller fails to build on arm32 due to a file system error. crazy-max/ghaction-docker-buildx#172 describes a workaround: create a smaller temporary filesystem for docker builds. This change employs this workaround to unblock releasing linkerd.
We can't use the typical multiarch docker build with the proxy: qemu-hosted arm64/arm builds take 45+ minutes before failing due to missing tooling--specifically `protoc`. (While there is a `protoc` binary available for arm64, there are no binaries available for 32-bit arm hosts). To fix this, this change updates the release process to cross-build the policy-controller on an amd64 host to the target architecture. We separate the policy-controller's dockerfiles as `amd64.dockerfile`, `arm64.dockerfile`, and `arm.dockerfile`. Then, in CI we build and push each of these images individually (in parallel, via a build matrix). Once all of these are complete, we use the `docker manifest` CLI tools to unify these images into a single multi-arch manifest. This cross-building approach requires that we move from using `native-tls` to `rustls`, as we cannot build against the platform- appropriate native TLS libraries. The policy-controller is now feature- flagged to use `rustls` by default, though it may be necessary to use `native-tls` in local development, as `rustls` cannot validate TLS connections that target IP addresses. The policy-controller has also been updated to pull in `tracing-log` for compatibility crates that do not use `tracing` natively. This was helpful while debugging connectivity issue with the Kubernetes cluster. The `bin/docker-build-policy-controller` helper script now *only* builds the amd64 variant of the policy controller. It fails when asked to build multiarch images.
olix0r
changed the title
mateiidavid
approved these changes
Aug 13, 2021
alpeb
approved these changes
Aug 13, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We can't use the typical multiarch docker build with the proxy:
qemu-hosted arm64/arm builds take 45+ minutes before failing due to
missing tooling--specifically
protoc. (While there is aprotocbinary available for arm64, there are no binaries available for 32-bit
arm hosts).
To fix this, this change updates the release process to cross-build the
policy-controller on an amd64 host to the target architecture. We
separate the policy-controller's dockerfiles as
amd64.dockerfile,arm64.dockerfile, andarm.dockerfile. Then, in CI we build and pusheach of these images individually (in parallel, via a build matrix).
Once all of these are complete, we use the
docker manifestCLI toolsto unify these images into a single multi-arch manifest.
This cross-building approach requires that we move from using
native-tlstorustls, as we cannot build against the platform-appropriate native TLS libraries. The policy-controller is now feature-
flagged to use
rustlsby default, though it may be necessary to usenative-tlsin local development, asrustlscannot validate TLSconnections that target IP addresses.
The policy-controller has also been updated to pull in
tracing-logforcompatibility with crates that do not use
tracingnatively. This washelpful while debugging connectivity issue with the Kubernetes cluster.
The
bin/docker-build-policy-controllerhelper script now only buildsthe amd64 variant of the policy controller. It fails when asked to build
multiarch images.