bootstrap_sdk: enable arm64 sdk bootstrap #134
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The arm64 profiles don't specify SYMLINK_LIB=yes, which makes sense since arm64 systems don't support multilib in the way that we are used to from x86. What this means is that build artifacts are installed into separate lib and lib64 directories. The root overlay installed in stage4 needs to check for SYMLINK_LIB before trying to create a symlink, otherwise it fails to be applied because it collides with the directory in the rootfs. This uncovered a second minor issues - the rust toolchain bootstrap scripts checked for /usr/lib64/rust*, but the ebuild installs to /usr/lib/rust. Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
The rust ebuild has some magic to detect cross-toolchains present on the system and enable building additional cross targets. The code to trigger the rebuild of rust is part of install_cross_rust, and checks whether the cross directories exist in the rust installation. If they don't, then rust is removed and rebuilt to allow for the auto-detection to happen. Right now there are two issues with the code. Firstly, the path that is checked is wrong, which leads to rust always being removed and rebuilt. The path checked is /usr/lib/rust-*/rustlib but /usr/lib/rustlib is where the files are installed. The second issue is that it checks for aarch64 dirs when CHOST is aarch64-cros-linux-gnu. However, on an aarch64 host the aarch64 dirs will already exist from building the sdk itself. The rust ebuild is not ready to handle aarch64 hosts yet and blows up. The correct behavior is to combine the check for CHOST with a check for the right CBUILD. On an aarch64 host we should presumably check for the x86 CHOST and rust dirs, but that can be added later, because it needs more work. Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
This was referenced Jul 23, 2021
|
CI running http://jenkins.infra.kinvolk.io:8080/job/os/job/manifest/3139/ This change is actually independent of arm64 and is also needed for the profile upgrade (17.1) so it would be nice to get it merged soonish. It doesn't change anything functionally right now. |
pothos
reviewed
Jul 27, 2021
For consistency with code further down in the file: aarch64 cross compilation only applies when CBUILD is x86, for native aarch64 builds rust is guaranteed to have aarch64 rustlibs.
pothos
approved these changes
Jul 28, 2021
dongsupark
reviewed
Aug 2, 2021
There was a problem hiding this comment.
As I said in the meeting today, I managed to build an arm64 SDK from the PRs. 🎉
Though I am not sure when we should merge the PRs, as we could start building another set of releases soon.
I will let @sayanchowdhury share some thoughts.
|
Finished a full amd64 bootstrap with this and all looks good. Talked to @sayanchowdhury and told him about merging this before release. |
dongsupark
pushed a commit
that referenced
this pull request
Aug 10, 2021
Previously before #134, `bootstrap_sdk` was looking at the wrong path (/usr/lib/rust-*/rustlib/aarch64-unknown-linux-gnu instead of /usr/lib/rustlib/aarch64-unknown-linux-gnu). As a result, Rust got always removed and rebuilt in `install_cross_rust`, which resulted in `flatcar-sdk/crossdev/dev-lang/rust/rust-1.54.0-1.xpak` being created. Now legitimate changes of #134 prevent the rebuild from happening. The path already exists in a stage4 SDK build, because the seed stage already has cross-compilers so the Rust upgrade has all the right cross-paths. That's why SDK builds with only stage4 failed when it tries uploading Rust packages like the following. On the other hand, full SDK builds with stage1 to 4 worked well, because in that case Rust is rebuilt anyway. ``` INFO bootstrap_sdk: Uploading cross toolchain packages to gs://flatcar-jenkins/developer/sdk/amd64/2021.08.04+dev-flatcar-master-3209 CommandException: No URLs matched: /mnt/host/source/src/build/catalyst/packages/flatcar-sdk/crossdev/* CommandException: No URLs matched: /tmp/tmp.xyjXbCFhUc//mnt/host/source/src/build/catalyst/packages/flatcar-sdk/crossdev/*.sig CommandException: 2 files/objects could not be transferred. ``` To fix that, we have to skip uploading packages when the crossdev directory does not exist. Debugged and suggested by @jepio
t-lo
pushed a commit
that referenced
this pull request
Nov 4, 2021
Previously before #134, `bootstrap_sdk` was looking at the wrong path (/usr/lib/rust-*/rustlib/aarch64-unknown-linux-gnu instead of /usr/lib/rustlib/aarch64-unknown-linux-gnu). As a result, Rust got always removed and rebuilt in `install_cross_rust`, which resulted in `flatcar-sdk/crossdev/dev-lang/rust/rust-1.54.0-1.xpak` being created. Now legitimate changes of #134 prevent the rebuild from happening. The path already exists in a stage4 SDK build, because the seed stage already has cross-compilers so the Rust upgrade has all the right cross-paths. That's why SDK builds with only stage4 failed when it tries uploading Rust packages like the following. On the other hand, full SDK builds with stage1 to 4 worked well, because in that case Rust is rebuilt anyway. ``` INFO bootstrap_sdk: Uploading cross toolchain packages to gs://flatcar-jenkins/developer/sdk/amd64/2021.08.04+dev-flatcar-master-3209 CommandException: No URLs matched: /mnt/host/source/src/build/catalyst/packages/flatcar-sdk/crossdev/* CommandException: No URLs matched: /tmp/tmp.xyjXbCFhUc//mnt/host/source/src/build/catalyst/packages/flatcar-sdk/crossdev/*.sig CommandException: 2 files/objects could not be transferred. ``` To fix that, we have to skip uploading packages when the crossdev directory does not exist. Debugged and suggested by @jepio
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
enable arm64 sdk bootstrap
Some issues with lib(64) handling and rust were found during attempts to bootstrap the SDK on arm64. This PR resolves them.
How to use
Provision powerful arm64 machine.
Fetch flatcar-archive/coreos-overlay/pull/1130
Fetch stage4 from flatcar/Flatcar#319 (comment).
./bootstrap_sdk --seed_tarball