Release workflow: Use GH Action for packaging sources, cross-compile addr2line on host #44
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Instead of re-implementing source packaging for multiple projects, we've come up with a dedicated GitHub Action. Let's use it for the release workflow. This Action should take into account the export-ignore rules from .gitattributes, modulo some adjustment that was just merged into libbpf and has not been propagated to retsnoop's submodules yet. In the meantime, this means that we have no difference with the previous behaviour, where the release workflow would ignore these exclusion rules anyway. Signed-off-by: Quentin Monnet <quentin@isovalent.com>
This commit contains a few clean-ups for the release GitHub Workflow: - Mention arch in arch-specific workflow steps - Add string to grep in "file"'s output to matrix directly, instead of crafting environment variables - Miscellaneous indent and spacing nits No functional change. Signed-off-by: Quentin Monnet <quentin@isovalent.com>
Although the release workflow used to succeed, the arm64 is now failing
because the update for Cargo's registry index gets OOM-killed, when
happening in the Docker container we use for cross-compiling. I'm not
sure what caused that change, but this is something we want to address.
I tested several options: using the "sparse" protocol to download less
data for the index, or using the git CLI to update it ("cargo update
--config net.git-fetch-with-cli=true"), which is supposed to require
less memory. None of these solutions was enough. I managed to fix the
workflow by: 1) running "cargo update" ouside of the container, 2)
mounting Cargo's home directory into the container:
docker run --platform linux/arm64 --rm -v $(pwd):/build \
-v ${HOME}/.cargo/git:/usr/local/cargo/git \
-v ${HOME}/.cargo/registry:/usr/local/cargo/registry \
ubuntu:22.04 \
...
and 3) running another "cargo update" inside of the container (using
sparse and git CLI for both updates).
This solution seems heavy and not optimal, and the job would still take
around 20 minutes to complete.
Instead, we can build addr2line outside of the container, directly on
the runner. This is not super-clean in the sense that part of retsnoop
is cross-compiled on the host, and another part of it inside of the
container; also, we set "CARGO=true" in the container to avoid
re-triggering a build. But it works, and it's efficient: the job now
lasts about 6 minutes.
Signed-off-by: Quentin Monnet <quentin@isovalent.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Follow-up to #42.
This PR updates the release workflow to use a dedicated GitHub Action for packaging the source files for retsnoop and its submodules. Contrarily to the previous commands, this Action should take into account the
export-ignoredirectives from the.gitattributesfiles. Although it may not show immediately for the libbpf submodule, because we need libbpf/libbpf@3f591a6 to be synced before that.When testing the change above, I observed that the job for building the arm64 binary would no longer pass. This is due to the Rust registry index update getting OOM-killed in the Docker container that we use to cross-compile. I'm not sure what caused this change, but I found a way to address it: by cross-compiling
addr2linedirectly on the host, we solve the memory issue, and we cut down the duration of the job from ~20 to ~6 minutes.