Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bib: check that architecture is expected arch #347

Merged
merged 2 commits into from
Apr 23, 2024

Conversation

mvo5
Copy link
Collaborator

@mvo5 mvo5 commented Apr 12, 2024

When trying to build an image with an incompatible target arch
bib will currently not error because the container resolver is
not very strict about the architecture request.

This commit fixes this by double checking that the resolved
container is actually of the expected architecture.

This requires osbuild/images#585

@mvo5 mvo5 force-pushed the check-resolved-image-arches branch from 4d6bf90 to e7f4721 Compare April 12, 2024 14:29
@mvo5 mvo5 force-pushed the check-resolved-image-arches branch 2 times, most recently from 9d23cd9 to b5f5f66 Compare April 18, 2024 05:08
@mvo5 mvo5 marked this pull request as ready for review April 18, 2024 05:09
@mvo5 mvo5 force-pushed the check-resolved-image-arches branch 2 times, most recently from a93a394 to f4bf2ac Compare April 18, 2024 06:58
test/test_manifest.py Outdated Show resolved Hide resolved
@mvo5 mvo5 force-pushed the check-resolved-image-arches branch 4 times, most recently from 995d0e2 to 00fbe01 Compare April 19, 2024 14:19
test/test_manifest.py Outdated Show resolved Hide resolved
@mvo5 mvo5 force-pushed the check-resolved-image-arches branch 3 times, most recently from 3c84943 to c3225be Compare April 23, 2024 07:20
@mvo5 mvo5 force-pushed the check-resolved-image-arches branch 4 times, most recently from 0de96ec to 6bce9a9 Compare April 23, 2024 09:40
ondrejbudai
ondrejbudai previously approved these changes Apr 23, 2024
Copy link
Member

@ondrejbudai ondrejbudai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love this!

@ondrejbudai ondrejbudai added this pull request to the merge queue Apr 23, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to a conflict with the base branch Apr 23, 2024
@kingsleyzissou
Copy link
Contributor

Thanks so much for this @mvo5 this was much needed!

This is useful for cross arch container buildtesting. Note that
this always pass `--arch` when doing `podman build` because
without that the default behavior is to pull whatever arch was
pulled for this image ref last but we want "native" if nothing
else is specified.

Note: This just passes `platform.uname().machine` which uses the
kernel architecture names. podman seems to translate those kernel
arch names to go arches automatically.
When trying to build an image with an incompatible target arch
bib will currently not error because the container resolver is
not very strict about the architecture request.

This commit fixes this by double checking that the resolved
container is actually of the expected architecture.

This requires osbuild/images#585
@mvo5 mvo5 force-pushed the check-resolved-image-arches branch from 6bce9a9 to bb82266 Compare April 23, 2024 12:55
@kingsleyzissou kingsleyzissou added this pull request to the merge queue Apr 23, 2024
Merged via the queue into osbuild:main with commit fd264fa Apr 23, 2024
8 of 9 checks passed
@mvo5 mvo5 deleted the check-resolved-image-arches branch April 23, 2024 15:50
return nil, nil, fmt.Errorf("failed to pull container image: %w\n%s", err, output)
}
}

// TODO: check arch compat before pulling
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a TODO added, but not removed after the commit, I have a feeling this may have broken manifest compatibility, see: osbuild/images#635 (review)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cdrage
Copy link
Contributor

cdrage commented Apr 26, 2024

Reverting this commit and using @kingsleyzissou 's fixes on images, I am able to build the manifest correctly:

AMD64 test:

podman run \
    --rm \
    -it \
    --privileged \
    --pull=newer \
    --security-opt label=type:unconfined_t \
    -v $(pwd)/output:/output \
    -v /var/lib/containers/storage:/var/lib/containers/storage \
    localhost/bib:latest \
    --type qcow2 --target-arch amd64 \
    --local \
    localhost/manifest-test
Generating manifest manifest-qcow2.json
WARNING: target-arch is experimental and needs an installed 'qemu-user' package
DONE
Building manifest-qcow2.json
starting -Pipeline source org.osbuild.containers-storage: 826e75e852fd623ebaada51841b4d7c18d34fb5c1e6b3d3366e61d46c03362aa
Build
  root: <host>
Pipeline build: 5a903d3def952988f3a70228eaf59fab65dfd9bb1ba7ade9888095c49542c30f
Build
  root: <host>
  runner: org.osbuild.fedora38 (org.osbuild.fedora38)
org.osbuild.container-deploy: a8458324e7a563125fe33295ec70ba1b4eef907efa55a60d08c42c1a0ef639d3 {}
time="2024-04-26T13:54:37Z" level=info msg="Image operating system mismatch: image uses OS \"linux\"+architecture \"amd64\"+\"\", expecting one of \"linux+arm64+\\\"v8\\\", linux+arm64+\\\"\\\"\""
Getting image source signatures

ARM64 test:

podman run \
    --rm \
    -it \
    --privileged \
    --pull=newer \
    --security-opt label=type:unconfined_t \
    -v $(pwd)/output:/output \
    -v /var/lib/containers/storage:/var/lib/containers/storage \
    localhost/bib:latest \
    --type qcow2 --target-arch arm64 \
    --local \
    localhost/manifest-test
Generating manifest manifest-qcow2.json
DONE
Building manifest-qcow2.json
starting -Pipeline source org.osbuild.containers-storage: 64a862db1094290c75a091a9ac18f8d0af9df2d1e0ab87df9196a89c3a609e23
Build
  root: <host>
Pipeline build: 2643ac240b43408a2a11885a3519dfd4dc479eca24e788dbe80eb0c108dd6aa7
Build
  root: <host>
  runner: org.osbuild.fedora38 (org.osbuild.fedora38)
org.osbuild.container-deploy: 8e028e2f5a57a932f7ae74a596feae2f523429af019b086251760b4df61e4fb9 {}
Getting image source signatures
Copying blob sha256:159348fa9cfbb75c5cb57e9fac24b9f28477412149e901bdadb909bfaeb84dad
Copying blob sha256:88aab01220f9684503fb67e4ac6c7548d5503bec19e1b9935e04f86119f72f5c
Copying blob sha256:30e48485073f94510efb03f4a9ef5c2bcb6c88e42299ae57691758bea6dce4fc
Copying blob sha256:3327039033b21c313ace25702ec9a706876cad3e0d92ac16eaa096f6c80c3161
Copying blob sha256:1398857a59385bf21b05036c99b41b3ffa2ccf989a99dd66092c8d17d927d47c
Copying blob sha256:bc6a2ddc91e10b1a426be528ef61ccab8eaddd5fbc7f294fdfc315c0439c3df8
Copying blob sha256:ea21f664c19ce57ea2cbe3b21be9a5c4069c096e8f01afaa4fb0a14fed6918ee
Copying blob sha256:7afb53fb053f92a1555ddb31ad31773bb983ed98e012cca6e426f0c67935d976
Copying blob sha256:ed6e50b45a9355d57f5593c64d35b4e8eb7b953bc402815a201cb4986a2f7d03
Copying blob sha256:962b010d608fb3e37a22d5da75dd21a98f43e0d4a548256b8c2615132b8da976
Copying blob sha256:d14dafd0d84f06d1724f6fa5fcb78c438b53ca1e605dc3668779f5a39347e16d
Copying blob sha256:6ef238f5096277c47cac442c1cfc0091958b92391dec441d7121114f2ce9224c
Copying blob sha256:143569275e12d9aa97c245fa02a077129bf3eefa65550b588822139979dd2423
Copying blob sha256:5b8cb2c5a76d77f168728d7ec9e8140d969299446b088fd76386277d54b7622e
Copying blob sha256:1e4036afc4e15b9f799bcdf6157fdf1e788dae66c64ef6f46b411094ffc8ed25
Copying blob sha256:ff18962ccfa3557da881ec43f19936d9add6053fbaea093bf549fce38e4c82fd
^C%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants