Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update qemu-user-static to version 8.0.4 #246

Merged
merged 8 commits into from
Nov 22, 2023

Conversation

mbargull
Copy link
Member

@mbargull mbargull commented Nov 21, 2023

Checklist

  • Used a personal fork of the feedstock to propose changes
  • Bumped the build number (if the version is unchanged)
  • Reset the build number to 0 (if the version changed)
  • Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
  • Ensured the license file is being packaged.

closes gh-245
closes gh-228

The GitHub repo we pulled the binaries from before is not updated as mentioned by @h-vetinari in #245 (comment) .
This now downloads the binaries from Fedora (or Debian).

Signed-off-by: Marcel Bargull <marcel.bargull@udo.edu>
@mbargull mbargull marked this pull request as draft November 21, 2023 20:14
The runner apparently doesn't have bsdtar installed.
Let's just use dpkg-deb to extract Debian packages instead then.

Signed-off-by: Marcel Bargull <marcel.bargull@udo.edu>
@mbargull mbargull marked this pull request as ready for review November 21, 2023 20:33
@@ -1,33 +1,43 @@
#!/bin/bash -ex

set -xe
set -eux

if [ "$(uname -m)" == "x86_64" ]; then
docker run --rm --privileged multiarch/qemu-user-static:register --reset
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

N.B.: We can leave this as is since the binfmt_misc registration will be the same for any version.

download-qemu-static.sh Outdated Show resolved Hide resolved
Signed-off-by: Marcel Bargull <marcel.bargull@udo.edu>
This can just live on in Git commit history instead of cluttering code.

Signed-off-by: Marcel Bargull <marcel.bargull@udo.edu>
Signed-off-by: Marcel Bargull <marcel.bargull@udo.edu>
Signed-off-by: Marcel Bargull <marcel.bargull@udo.edu>
download-qemu-static.sh Outdated Show resolved Hide resolved
Signed-off-by: Marcel Bargull <marcel.bargull@udo.edu>
@mbargull mbargull marked this pull request as draft November 22, 2023 08:06
@mbargull
Copy link
Member Author

So, the linux-anvil-aarch64 build stalls with 8.1.2.
Seemingly at/after

#13 182.6   Cleanup    : glibc-2.17-317.el7.aarch64                               112/113 

from the yum update -y call.
I'm not sure at which point the locale is generated, but it could very well stall during that since the aarch64-cuda ones do not stall.

I skimmed over https://gitlab.com/qemu-project/qemu/-/issues/?sort=created_date&state=all&label_name%5B%5D=target%3A%20arm but nothing jumped at me immediately (didn't look too long).


All other tried versions -- 7.2.0 (what we already use), 8.0.4, 8.2.0-rc0 -- work.
Things we could do:

  1. Go with Ubuntu's 8.0.4.
  2. Use Debians 8.2.0-rc0 (my vote: let's not use this pre-release).
  3. Wait for builds of 8.1.3 (only tagged yesterday) and try those.
  4. Wait for 8.2 release in December.

The build for linux-anvil-aarch64 stalled with 8.1.2 from Debian.
ref:
 conda-forge#246 (comment)

Signed-off-by: Marcel Bargull <marcel.bargull@udo.edu>
@mbargull mbargull changed the title Update qemu-user-static to version 8.1.2 Update qemu-user-static to version 8.0.4 Nov 22, 2023
@mbargull mbargull marked this pull request as ready for review November 22, 2023 10:02
@h-vetinari
Copy link
Member

Thanks for the investigation!

Things we could do:

IMO option 1 sounds preferable. If QEMU 8 already solves the issues we see with numpy, scipy - all the better! If not, we can always try updating again later (whether to 8.1.3, 8.2.0, or some future version).

@mbargull
Copy link
Member Author

IMO option 1 sounds preferable.

Then this only needs an approving review and we can merge ;).

curl -sL \
"https://mirrors.edge.kernel.org/ubuntu/pool/universe/q/qemu/qemu-user-static_${version}%2B${build}_amd64.deb" |
dpkg-deb --extract - ./deb-tmp
mv ./deb-tmp/usr/bin/qemu-*-static ./
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we not moving too many aarches here? I don't know how many come with the deb file, but the fact that we could remove arm and things still work sounds to me like we're blowing up the image unnecessarily here?

Perhaps it's better to go back to an explicit for-loop?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't really hurt since only those that we explicitly ADD/COPY in the build files get used.
(It's similar to having the .git directory in the build context -- it's there, but we don't care since we don't touch it.)

Even the temporary directory and moving those files isn't really necessary.
I.e., we could just dpkg-deb --extract - ./qemu-user-static and then COPY ./qemu-user-static/usr/bin/qemu-aarch64-static /usr/bin/ in the Dockerfiles.
(I just didn't do that yet since I didn't want to argue for or against it.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I was thinking this would be called during the docker build, but if it's just happening in the build context, then obviously that's not an issue.

I.e., we could just dpkg-deb --extract - ./qemu-user-static and then COPY ./qemu-user-static/usr/bin/qemu-aarch64-static /usr/bin/ in the Dockerfiles.
(I just didn't do that yet since I didn't want to argue for or against it.)

That could be a nice improvement IMO.

@isuruf isuruf merged commit 6dcacf9 into conda-forge:main Nov 22, 2023
34 checks passed
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update qemu Drop armv7l bits
3 participants