register: file exists

[pinkynrg]

I'm trying to understand how to build docker images on x86 cups for different architectures like ARM.

I tried the following command on my macbook:

sudo docker run --rm --privileged multiarch/qemu-user-static:register

and one of the output is:

Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm
sh: write error: File exists

It looks like the generated file already exists. My problem is that I can't find qemu-arm-static in /usr/bin/.

So, where was it saved?

Thanks

[agowa]

The error message is not about /usr/bin/qemu-arm/static existing, but the entry within /proc/sys/fs/binfmt_misc/ for it to exist.
you still need to install/build the package for qemu-arm-static.
For ArchLinux for example you need to build this package: https://aur.archlinux.org/packages/qemu-user-static/

And afterwards you can run sudo docker run --rm --privileged multiarch/qemu-user-static:register --reset to make sure all registrations are present.

[umarcor]

I'm trying to understand how to build docker images on x86 cups for different architectures like ARM.

@pinkynrg, you might find the information in dbhi/qus useful. The TL;DR is:

docker run --rm --privileged aptman/qus -- -r
docker run --rm --privileged aptman/qus -s -- -p aarch64

docker build -t arm/test -<<EOF
FROM arm64v8/ubuntu:bionic

# Do your ARM stuff here

EOF

You can find a more complex example (where multiple tools are built for arm, arm64 and x86_64) at dbhi/docker.

[junaruga]

@pinkynrg

sudo docker run --rm --privileged multiarch/qemu-user-static:register

is to just create or recreate rules files to host OS's /proc/sys/fs/binfmt_misc directory from container.
After running above command,

$ ls /proc/sys/fs/binfmt_misc/qemu-* | head
/proc/sys/fs/binfmt_misc/qemu-aarch64
/proc/sys/fs/binfmt_misc/qemu-aarch64_be
/proc/sys/fs/binfmt_misc/qemu-alpha
/proc/sys/fs/binfmt_misc/qemu-arm
...

Below message means

Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm
sh: write error: File exists

/proc/sys/fs/binfmt_misc/qemu-arm file already exists on your hosts.
The program can not add (override) because of that.
Yes, the message is not so helpful.

The actual logic is here.
https://github.com/multiarch/qemu-user-static/blob/master/register/Dockerfile
https://github.com/multiarch/qemu-user-static/blob/master/register/register.sh#L23

$ curl -O https://raw.githubusercontent.com/qemu/qemu/master/scripts/qemu-binfmt-conf.sh
$ chmod +x qemu-binfmt-conf.sh

Then run below command from root's prompt, sudo does not work for below command.

# ./qemu-binfmt-conf.sh --qemu-suffix "-static" --qemu-path /usr/bin

The not so friendly message "Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm" is defined at below part.
https://github.com/qemu/qemu/blob/master/scripts/qemu-binfmt-conf.sh#L269

Maybe sending a patch to improve the message to qemu project is welcome.

qemu_register_interpreter() {
    echo "Setting $qemu as binfmt interpreter for $cpu"
    qemu_generate_register > /proc/sys/fs/binfmt_misc/register
}

The solution to avoid the error, is using --reset.

$ sudo docker run --rm --privileged multiarch/qemu-user-static:register

When using --reset option, below command to remove registered entries are executed before registering new entries.

# find /proc/sys/fs/binfmt_misc -type f -name 'qemu-*' -exec sh -c 'echo -1 > {}' \;

Here is the manual of binfmt-misc that is executed from this repository's qemu-user-static or qemu-user-static RPM package.
https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html

Here is my host OS Fedora 30's result.

# find /proc/sys/fs/binfmt_misc -type f -name 'qemu-*' -exec sh -c 'echo -1 > {}' \;

# ls /proc/sys/fs/binfmt_misc
register  status

Add (register) below below file. Adding (registering) a file for each CPU is what this repository's qemu-user-static is doing. It does not install or operate /usr/bin/qemu-aarch64-static.

# echo ":qemu-aarch64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xb7\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-aarch64-static:" > /proc/sys/fs/binfmt_misc/register

# ls /proc/sys/fs/binfmt_misc/
qemu-aarch64  register  status

# cat /proc/sys/fs/binfmt_misc/qemu-aarch64 
enabled
interpreter /usr/bin/qemu-aarch64-static
flags: 
offset 0
magic 7f454c460201010000000000000000000200b700
mask ffffffffffffff00fffffffffffffffffeffffff

When you want to remove the registered entry (the rule file: /proc/sys/fs/binfmt_misc/qemu-aarch64), run below command.

Below is the logic to remove the registered file (entry).

# echo -1 > /proc/sys/fs/binfmt_misc/qemu-aarch64 

# ls /proc/sys/fs/binfmt_misc
register  status

My problem is that I can't find qemu-arm-static in /usr/bin/.

Yes, this repository's qemu-user-static does not install /usr/bin/qemu-*-static files.

if you are using Debian base Linux, run below

# yum install qemu-user-static

If you are using Fedora, CentOS (RPM base Linux), run below.

# yum install qemu-user-static

Then install below file. Again that is what this repository's qemu-user-static is doing.

# echo ":qemu-aarch64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xb7\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-aarch64-static:" > /proc/sys/fs/binfmt_misc/register

# ls /proc/sys/fs/binfmt_misc/qemu-aarch64 
/proc/sys/fs/binfmt_misc/qemu-aarch64

/usr/bin/qemu-aarch64-static is needed inside of the container image.

$ docker run --rm -t -v /usr/bin/qemu-aarch64-static:/usr/bin/qemu-aarch64-static arm64v8/fedora uname -m
aarch64

Or you can try below "Compatible Images" for popular distributions.

https://github.com/multiarch/qemu-user-static

ubuntu-debootstrap: Docker Hub, GitHub

This is the image /usr/bin/qemu-<cpu>-static is already installed inside of the image.

So, you can run it.

$ sudo docker run --rm -t  multiarch/fedora:30-aarch64 uname -m 
aarch64

I think I answered for your question.
If you need, please reopen the ticket.
Thank you.