Skip to content

Backport toolkit container detection using systemd-detect-virt#11135

Merged
dmcilvaney merged 3 commits intomicrosoft:mainfrom
dmcilvaney:damcilva/2.0/tools/docker_detect_backport
Nov 26, 2024
Merged

Backport toolkit container detection using systemd-detect-virt#11135
dmcilvaney merged 3 commits intomicrosoft:mainfrom
dmcilvaney:damcilva/2.0/tools/docker_detect_backport

Conversation

@dmcilvaney
Copy link
Copy Markdown
Contributor

@dmcilvaney dmcilvaney commented Nov 19, 2024

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

  • The toolchain has been rebuilt successfully (or no changes were made to it)
  • The toolchain/worker package manifests are up-to-date
  • Any updated packages successfully build (or no packages were changed)
  • Packages depending on static components modified in this PR (Golang, *-static subpackages, etc.) have had their Release tag incremented.
  • Package tests (%check section) have been verified with RUN_CHECK=y for existing SPEC files, or added to new SPEC files
  • All package sources are available
  • cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json, .github/workflows/cgmanifest.json)
  • LICENSE-MAP files are up-to-date (./LICENSES-AND-NOTICES/SPECS/data/licenses.json, ./LICENSES-AND-NOTICES/SPECS/LICENSES-MAP.md, ./LICENSES-AND-NOTICES/SPECS/LICENSE-EXCEPTIONS.PHOTON)
  • All source files have up-to-date hashes in the *.signatures.json files
  • sudo make go-tidy-all and sudo make go-test-coverage pass
  • Documentation has been updated to match any changes to the build system
  • Ready to merge

Summary

Backport of #11039. Removed the calls to logger.PrintMessageBox() and replaced with logger.Log.Warn().

Previous PR description follows:

There have been several issues with a mismatch between the build environment and the detected state by the toolkit. When running in docker, the chroots generally need to be configured externally with their mounts and re-used. This can be done via setting `CHROOT_DIR=/path/to/reusable/chroots`, and if the toolkit thinks it's in a container, it will switch modes.

See https://github.com/microsoft/azurelinux-tutorials/tree/main/build-in-container for more details.

Some builds are currently failing because what is ostensibly a container environment does not have /.dockerenv present.

In the opposite direction, there are also situations where WSL images (which should work fine as a normal build) are reporting as docker because they have a /.dockerenv file present.

systemd has a tool (systemd-detect-virt) which is designed to detect what sort of virtualization is being used to run the current environment. Instead of designing a new system to re-implement this behavior, we can just use this tool. We already have an implicit build dependency on systemd (we run the docker service etc.) so adding it as an explicit requirement shouldn't change anything.

Also, to help people self-diagnose, sanity check the configurations and warn the user:

  • If the CHROOT_DIR is set, but the tool thinks it's in a normal environment, print a warning. A fatal error will follow immediately after if this is actually broken.
  • If the systemd-detect-virt tool is not present, print a warning but fallback to the old behavior.

To validate this we will need to add a new testcase to the toolkit sanity test pipeline that ensures the chroots keep working.

Change Log
  • Prefer systemd-detect-virt over /.dockerenv for container detection
  • Print warnings if misconfiguration is detected
Does this affect the toolchain?

NO

Associated issues
Links to CVEs
Test Methodology

@microsoft-github-policy-service microsoft-github-policy-service Bot added the main PR Destined for main label Nov 19, 2024
@dmcilvaney dmcilvaney added bug Something isn't working Tools labels Nov 19, 2024
@dmcilvaney dmcilvaney force-pushed the damcilva/2.0/tools/docker_detect_backport branch from 7f1f225 to 837efd7 Compare November 22, 2024 00:30
@dmcilvaney dmcilvaney force-pushed the damcilva/2.0/tools/docker_detect_backport branch from 837efd7 to 8fab474 Compare November 22, 2024 00:32
@dmcilvaney dmcilvaney marked this pull request as ready for review November 22, 2024 00:32
@dmcilvaney dmcilvaney requested a review from a team as a code owner November 22, 2024 00:32
@dmcilvaney dmcilvaney merged commit 15b043a into microsoft:main Nov 26, 2024
kodiakhq Bot pushed a commit to Azure/iot-identity-service that referenced this pull request Dec 19, 2024
)

Our Azure Linux builds started failing in the past week or so because Azure Linux recently made a change to how they detect container builds (see microsoft/azurelinux#11135), so the trick we employed to force a regular build (even though we're building in a container) stopped working. This change removes the trick and takes the recommended approach of adding `/.mariner-toolkit-ignore-dockerenv` to the container.
kodiakhq Bot pushed a commit to Azure/iotedge that referenced this pull request Dec 20, 2024
…7409)

Our Azure Linux builds started failing in the past week or so because Azure Linux recently made a change to how they detect container builds (see microsoft/azurelinux#11135), so the trick we employed to force a regular build (even though we're building in a container) stopped working. This change removes the trick and takes the recommended approach of adding `/.mariner-toolkit-ignore-dockerenv` to the container.

Other changes were also needed to resolve problems that came up while testing this PR:
- Snap builds began failing, seemingly due to a bad version combination of snapcraft and lxd. I updated the build task to do `snap refresh` before building.
- Our CI build for Azure Linux on arm64 uses the build scripts from the iot-identity-service repo, so I updated this repo's references to the iot-identity-service repo to fix the container build detection problem there.
- A recent update to a dependency of cross requires a newer rustc version and was causing the edgelet CI runs to fail in this PR because our rustc version is fixed at 1.73, I added the '--locked' flag to the cargo command that installs cross in the edgelet CI build.

To test, I confirmed that the CI Build passes with these changes.

## Azure IoT Edge PR checklist:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working main PR Destined for main Tools

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants