bwrap: Can't access /newroot/buildroot/proc/sysrq-trigger: Read-only file system

[disjustin]

mkosi commit the issue has been seen with

main

Used host distribution

RHEL 8

Used target distribution

centos

Linux kernel version used

4.18.0-513.18.1.el8_9.x86_64

CPU architectures issue was seen on

x86_64

Unexpected behaviour you saw

Fresh install on RHEL 8 system. Trying to generate a centos disk image for starters but enounter a read-only access error.

git clone https://github.com/systemd/mkosi
ln -s $PWD/mkosi/bin/mkosi /usr/local/bin/mkosi
mkosi --version
# mkosi 23~devel

...
‣ + dnf --assumeyes --best --releasever=40 --installroot=/buildroot --setopt=keepcache=1 --setopt=logdir=/var/log --setopt=cachedir=/var/cache/dnf --setopt=persistdir=/var/lib/dnf --setopt=install_weak_deps=0 --setopt=check_config_file_age=0 '--disableplugin=*' --enableplugin=builddep --setopt=debuglevel=10 --setopt=metadata_expire=never --config=/etc/dnf/dnf.conf --setopt=reposdir=/etc/yum.repos.d --setopt=varsdir=/etc/dnf/vars --setopt=proxy_sslcacert=/proxy.cacert install acl attr bash bubblewrap ca-certificates coreutils cpio diffutils dnf dosfstools e2fsprogs findutils grep jq kmod less mtools nano openssl sed socat strace swtpm systemd tar util-linux virtiofsd xfsprogs zstd apt createrepo_c curl-minimal debian-keyring distribution-gpg-keys dnf-plugins-core git-core openssh-clients policycoreutils python3-cryptography qemu-img qemu-kvm-core shadow-utils squashfs-tools swtpm-tools systemd-container systemd-journal-remote systemd-udev ubu-keyring virt-firmware xz grub2-tools edk2-ovmf pesign archlinux-keyring btrfs-progs dnf5 dnf5-plugins erofs-utils pacman qemu-system-aarch64-core qemu-system-ppc-core qemu-system-s390x-core reprepro systemd-ukify zypper sbsigntools
bwrap: Can't access /newroot/buildroot/proc/sysrq-trigger: Read-only file system
‣ "bwrap --die-with-parent --proc /proc --setenv SYSTEMD_OFFLINE 1 --unsetenv TMPDIR --tmpfs /tmp --unshare-ipc --dev /dev --symlink usr/bin /bin --symlink usr/sbin /sbin --symlink usr/lib /lib --symlink usr/lib64 /lib64 --setenv PATH /scripts:/usr/lib64/openmpi/bin:/usr/share/Modules/bin:/usr/local/sbin:/usr/sbin:/sbin:/bin:/usr/bin:/usr/local/bin:/var/lib/snapd/snap/bin:/opt/mssql-tools18/bin:/root/bin:/opt/mssql-tools18/bin:/root/bin:/opt/mssql-tools18/bin:/root/bin:/opt/mssql-tools18/bin:/opt/mssql-tools18/bin:/opt/mssql-tools18/bin --uid 0 --gid 0 --cap-add ALL --dir /work/src --chdir /work/src --bind /var/tmp/mkosi-workspacedybhn7c6/root /buildroot --bind /var/tmp/mkosi-workspacedybhn7c6/pkgmngr/etc /etc --ro-bind /etc/alternatives /etc/alternatives --ro-bind /etc/pki /etc/pki --bind /etc/resolv.conf /etc/resolv.conf --ro-bind /etc/ssl /etc/ssl --ro-bind /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem /proxy.cacert --ro-bind /usr /usr --bind /var/tmp/mkosi-workspacedybhn7c6/root/var/cache/dnf /var/cache/dnf --bind '/var/cache/mkosi/fedora~40~x86-64/cache/dnf/fedora-de0bb1deee98eda5/packages' /var/cache/dnf/fedora-de0bb1deee98eda5/packages --bind '/var/cache/mkosi/fedora~40~x86-64/cache/dnf/updates-4a8088a5c17ad284/packages' /var/cache/dnf/updates-4a8088a5c17ad284/packages --bind /var/tmp/mkosi-workspacedybhn7c6/root/var/lib/dnf /var/lib/dnf --bind /var/tmp/mkosi-workspacedybhn7c6/pkgmngr/var/log /var/log --bind /var/tmp/mkosi-workspacedybhn7c6/tmp/mkosi-var-tmp-7d8bd1cd072d414a /var/tmp --bind /var/tmp/mkosi-workspacedybhn7c6/packages /work/packages sh -c 'chmod 1777 /dev/shm && chmod 755 /etc && exec $0 "$@"' bwrap --dev-bind / / --tmpfs /buildroot/run --tmpfs /buildroot/tmp --bind /var/tmp /buildroot/var/tmp --proc /buildroot/proc --dev /buildroot/dev --ro-bind-try /buildroot/etc/machine-id /buildroot/etc/machine-id --dir /buildroot/run/user/0 --ro-bind-try /buildroot/etc/passwd /etc/passwd --ro-bind-try /buildroot/etc/group /etc/group --ro-bind-try /buildroot/etc/shadow /etc/shadow --ro-bind-try /buildroot/etc/gshadow /etc/gshadow sh -c 'chmod 1777 /buildroot/tmp /buildroot/var/tmp /buildroot/dev/shm && chmod 755 /buildroot/run && mkdir -m 755 /buildroot/run/host && echo mkosi >/buildroot/run/host/container-manager && exec $0 "$@"' dnf --assumeyes --best --releasever=40 --installroot=/buildroot --setopt=keepcache=1 --setopt=logdir=/var/log --setopt=cachedir=/var/cache/dnf --setopt=persistdir=/var/lib/dnf --setopt=install_weak_deps=0 --setopt=check_config_file_age=0 '--disableplugin=*' --enableplugin=builddep --setopt=debuglevel=10 --setopt=metadata_expire=never --config=/etc/dnf/dnf.conf --setopt=reposdir=/etc/yum.repos.d --setopt=varsdir=/etc/dnf/vars --setopt=proxy_sslcacert=/proxy.cacert install acl attr bash bubblewrap ca-certificates coreutils cpio diffutils dnf dosfstools e2fsprogs findutils grep jq kmod less mtools nano openssl sed socat strace swtpm systemd tar util-linux virtiofsd xfsprogs zstd apt createrepo_c curl-minimal debian-keyring distribution-gpg-keys dnf-plugins-core git-core openssh-clients policycoreutils python3-cryptography qemu-img qemu-kvm-core shadow-utils squashfs-tools swtpm-tools systemd-container systemd-journal-remote systemd-udev ubu-keyring virt-firmware xz grub2-tools edk2-ovmf pesign archlinux-keyring btrfs-progs dnf5 dnf5-plugins erofs-utils pacman qemu-system-aarch64-core qemu-system-ppc-core qemu-system-s390x-core reprepro systemd-ukify zypper sbsigntools" returned non-zero exit code 1.
‣ + rm -rf -- /var/tmp/mkosi-workspacedybhn7c6
Traceback (most recent call last):
  File "/root/mkosi/mkosi/run.py", line 60, in uncaught_exception_handler
    yield
  File "/root/mkosi/mkosi/run.py", line 101, in fork_and_wait
    target(*args, **kwargs)
  File "/root/mkosi/mkosi/__init__.py", line 4476, in run_build
    build_image(Context(args, config, workspace=workspace, resources=resources))
  File "/root/mkosi/mkosi/__init__.py", line 3664, in build_image
    install_distribution(context)
  File "/root/mkosi/mkosi/__init__.py", line 163, in install_distribution
    context.config.distribution.install_packages(context, context.config.packages)
  File "/root/mkosi/mkosi/distributions/__init__.py", line 134, in install_packages
    return self.installer().install_packages(context, packages)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/mkosi/mkosi/distributions/fedora.py", line 92, in install_packages
    Dnf.invoke(context, "install", packages, apivfs=apivfs)
  File "/root/mkosi/mkosi/installer/dnf.py", line 173, in invoke
    return run(
           ^^^^
  File "/root/mkosi/mkosi/run.py", line 150, in run
    with spawn(
  File "/usr/lib64/python3.11/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/root/mkosi/mkosi/run.py", line 265, in spawn
    raise subprocess.CalledProcessError(rc, prefix + cmdline)
subprocess.CalledProcessError: Command '['bwrap', '--die-with-parent', '--proc', '/proc', '--setenv', 'SYSTEMD_OFFLINE', '1', '--unsetenv', 'TMPDIR', '--tmpfs', '/tmp', '--unshare-ipc', '--dev', '/dev', '--symlink', 'usr/bin', '/bin', '--symlink', 'usr/sbin', '/sbin', '--symlink', 'usr/lib', '/lib', '--symlink', 'usr/lib64', '/lib64', '--setenv', 'PATH', '/scripts:/usr/lib64/openmpi/bin:/usr/share/Modules/bin:/usr/local/sbin:/usr/sbin:/sbin:/bin:/usr/bin:/usr/local/bin:/var/lib/snapd/snap/bin:/opt/mssql-tools18/bin:/root/bin:/opt/mssql-tools18/bin:/root/bin:/opt/mssql-tools18/bin:/root/bin:/opt/mssql-tools18/bin:/opt/mssql-tools18/bin:/opt/mssql-tools18/bin', '--uid', '0', '--gid', '0', '--cap-add', 'ALL', '--dir', '/work/src', '--chdir', '/work/src', '--bind', '/var/tmp/mkosi-workspacedybhn7c6/root', '/buildroot', '--bind', '/var/tmp/mkosi-workspacedybhn7c6/pkgmngr/etc', '/etc', '--ro-bind', '/etc/alternatives', '/etc/alternatives', '--ro-bind', '/etc/pki', '/etc/pki', '--bind', '/etc/resolv.conf', '/etc/resolv.conf', '--ro-bind', '/etc/ssl', '/etc/ssl', '--ro-bind', '/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem', '/proxy.cacert', '--ro-bind', '/usr', '/usr', '--bind', '/var/tmp/mkosi-workspacedybhn7c6/root/var/cache/dnf', '/var/cache/dnf', '--bind', '/var/cache/mkosi/fedora~40~x86-64/cache/dnf/fedora-de0bb1deee98eda5/packages', '/var/cache/dnf/fedora-de0bb1deee98eda5/packages', '--bind', '/var/cache/mkosi/fedora~40~x86-64/cache/dnf/updates-4a8088a5c17ad284/packages', '/var/cache/dnf/updates-4a8088a5c17ad284/packages', '--bind', '/var/tmp/mkosi-workspacedybhn7c6/root/var/lib/dnf', '/var/lib/dnf', '--bind', '/var/tmp/mkosi-workspacedybhn7c6/pkgmngr/var/log', '/var/log', '--bind', '/var/tmp/mkosi-workspacedybhn7c6/tmp/mkosi-var-tmp-7d8bd1cd072d414a', '/var/tmp', '--bind', '/var/tmp/mkosi-workspacedybhn7c6/packages', '/work/packages', 'sh', '-c', 'chmod 1777 /dev/shm && chmod 755 /etc && exec $0 "$@"', 'bwrap', '--dev-bind', '/', '/', '--tmpfs', '/buildroot/run', '--tmpfs', '/buildroot/tmp', '--bind', '/var/tmp', '/buildroot/var/tmp', '--proc', '/buildroot/proc', '--dev', '/buildroot/dev', '--ro-bind-try', '/buildroot/etc/machine-id', '/buildroot/etc/machine-id', '--dir', '/buildroot/run/user/0', '--ro-bind-try', '/buildroot/etc/passwd', '/etc/passwd', '--ro-bind-try', '/buildroot/etc/group', '/etc/group', '--ro-bind-try', '/buildroot/etc/shadow', '/etc/shadow', '--ro-bind-try', '/buildroot/etc/gshadow', '/etc/gshadow', 'sh', '-c', 'chmod 1777 /buildroot/tmp /buildroot/var/tmp /buildroot/dev/shm && chmod 755 /buildroot/run && mkdir -m 755 /buildroot/run/host && echo mkosi >/buildroot/run/host/container-manager && exec $0 "$@"', 'dnf', '--assumeyes', '--best', '--releasever=40', '--installroot=/buildroot', '--setopt=keepcache=1', '--setopt=logdir=/var/log', '--setopt=cachedir=/var/cache/dnf', '--setopt=persistdir=/var/lib/dnf', '--setopt=install_weak_deps=0', '--setopt=check_config_file_age=0', '--disableplugin=*', '--enableplugin=builddep', '--setopt=debuglevel=10', '--setopt=metadata_expire=never', '--config=/etc/dnf/dnf.conf', '--setopt=reposdir=/etc/yum.repos.d', '--setopt=varsdir=/etc/dnf/vars', '--setopt=proxy_sslcacert=/proxy.cacert', 'install', 'acl', 'attr', 'bash', 'bubblewrap', 'ca-certificates', 'coreutils', 'cpio', 'diffutils', 'dnf', 'dosfstools', 'e2fsprogs', 'findutils', 'grep', 'jq', 'kmod', 'less', 'mtools', 'nano', 'openssl', 'sed', 'socat', 'strace', 'swtpm', 'systemd', 'tar', 'util-linux', 'virtiofsd', 'xfsprogs', 'zstd', 'apt', 'createrepo_c', 'curl-minimal', 'debian-keyring', 'distribution-gpg-keys', 'dnf-plugins-core', 'git-core', 'openssh-clients', 'policycoreutils', 'python3-cryptography', 'qemu-img', 'qemu-kvm-core', 'shadow-utils', 'squashfs-tools', 'swtpm-tools', 'systemd-container', 'systemd-journal-remote', 'systemd-udev', 'ubu-keyring', 'virt-firmware', 'xz', 'grub2-tools', 'edk2-ovmf', 'pesign', 'archlinux-keyring', 'btrfs-progs', 'dnf5', 'dnf5-plugins', 'erofs-utils', 'pacman', 'qemu-system-aarch64-core', 'qemu-system-ppc-core', 'qemu-system-s390x-core', 'reprepro', 'systemd-ukify', 'zypper', 'sbsigntools']' returned non-zero exit status 1.
‣ + tput cnorm
‣ + tput smam

Used mkosi config

default from repo

mkosi output

[root@localhost ~]# mkosi --distribution centos --tools-tree=default --debug
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf.d/10-centos-fedora/mkosi.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf.d/10-centos-fedora/mkosi.conf.d/10-grub.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf.d/10-centos-fedora/mkosi.conf.d/10-uefi.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf.d/10-centos.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf.d/10-centos-fedora/mkosi.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf.d/10-centos-fedora/mkosi.conf.d/10-grub.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf.d/10-centos-fedora/mkosi.conf.d/10-uefi.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf.d/10-fedora/mkosi.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf.d/10-fedora/mkosi.conf.d/10-uefi.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf.d/10-centos-fedora/mkosi.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf.d/10-centos-fedora/mkosi.conf.d/10-grub.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf.d/10-centos-fedora/mkosi.conf.d/10-uefi.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf.d/10-centos.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf.d/10-centos-fedora/mkosi.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf.d/10-centos-fedora/mkosi.conf.d/10-grub.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf.d/10-centos-fedora/mkosi.conf.d/10-uefi.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf.d/10-fedora/mkosi.conf
‣ Including configuration file /tmp/tmpe6hjqbr2/resources/mkosi-tools/mkosi.conf.d/10-fedora/mkosi.conf.d/10-uefi.conf
‣ + stat --file-system --format %T /var/tmp/mkosi-workspace3g2vo3e_
‣ + cp --version
‣ + cp --recursive --dereference --preserve=mode,links --reflink=auto --copy-contents /etc/crypto-policies /var/tmp/mkosi-workspace3g2vo3e_/pkgmngr/etc/crypto-policies --no-target-directory
‣ + rpm --eval '%{__plugindir}'
‣ Syncing package manager metadata for fedora-tools image
‣ Acquiring lock on /var/cache/mkosi/fedora~40~x86-64/cache/dnf
‣ Acquired lock on /var/cache/mkosi/fedora~40~x86-64/cache/dnf
‣ Acquiring lock on /var/cache/mkosi/fedora~40~x86-64/lib/dnf
‣ Acquired lock on /var/cache/mkosi/fedora~40~x86-64/lib/dnf
‣ + dnf --assumeyes --best --releasever=40 --installroot=/buildroot --setopt=keepcache=1 --setopt=logdir=/var/log --setopt=cachedir=/var/cache/dnf --setopt=persistdir=/var/lib/dnf --setopt=install_weak_deps=0 --setopt=check_config_file_age=0 '--disableplugin=*' --enableplugin=builddep --setopt=debuglevel=10 --setopt=metadata_expire=never --config=/etc/dnf/dnf.conf --setopt=reposdir=/etc/yum.repos.d --setopt=varsdir=/etc/dnf/vars --setopt=proxy_sslcacert=/proxy.cacert makecache
timer: config: 1 ms
Loaded plugins: builddep
DNF version: 4.7.0
Command: dnf --assumeyes --best --releasever=40 --installroot=/buildroot --setopt=keepcache=1 --setopt=logdir=/var/log --setopt=cachedir=/var/cache/dnf --setopt=persistdir=/var/lib/dnf --setopt=install_weak_deps=0 --setopt=check_config_file_age=0 --disableplugin=* --enableplugin=builddep --setopt=debuglevel=10 --setopt=metadata_expire=never --config=/etc/dnf/dnf.conf --setopt=reposdir=/etc/yum.repos.d --setopt=varsdir=/etc/dnf/vars --setopt=proxy_sslcacert=/proxy.cacert makecache 
Installroot: /buildroot
Releasever: 40
cachedir: /buildroot/var/cache/dnf
Base command: makecache
Extra commands: ['--assumeyes', '--best', '--releasever=40', '--installroot=/buildroot', '--setopt=keepcache=1', '--setopt=logdir=/var/log', '--setopt=cachedir=/var/cache/dnf', '--setopt=persistdir=/var/lib/dnf', '--setopt=install_weak_deps=0', '--setopt=check_config_file_age=0', '--disableplugin=*', '--enableplugin=builddep', '--setopt=debuglevel=10', '--setopt=metadata_expire=never', '--config=/etc/dnf/dnf.conf', '--setopt=reposdir=/etc/yum.repos.d', '--setopt=varsdir=/etc/dnf/vars', '--setopt=proxy_sslcacert=/proxy.cacert', 'makecache']
Making cache files for all metadata files.
fedora: has expired and will be refreshed.
updates: has expired and will be refreshed.
fedora                                                                                                                                                                              13 kB/s |  23 kB     00:01    
reviving: 'fedora' can be revived - metalink checksums match.
fedora: using metadata from Sun Apr 14 18:51:11 2024.
updates                                                                                                                                                                             31 kB/s |  22 kB     00:00    
reviving: 'updates' can be revived - metalink checksums match.
updates: using metadata from Mon May  6 02:17:22 2024.
timer: sack setup: 2837 ms
User-Agent: constructed: 'libdnf (Red Hat Enterprise Linux 8.9; generic; Linux.x86_64)'
Metadata cache created.
Cleaning up.
Plugins were unloaded.
‣ + rm -rf -- /var/tmp/mkosi-workspace3g2vo3e_
‣ + mount --make-rslave /
‣ + mount --rbind /etc /etc --options ro
‣ + mount --rbind /opt /opt --options ro
‣ + mount --rbind /boot /boot --options ro
‣ + mount --rbind /media /media --options ro
‣ + mount --rbind /usr /usr --options ro
‣ Building fedora-tools image
‣ + stat --file-system --format %T /var/tmp/mkosi-workspacedybhn7c6
‣ + cp --version
‣ + cp --recursive --dereference --preserve=mode,links --reflink=auto --copy-contents /etc/crypto-policies /var/tmp/mkosi-workspacedybhn7c6/pkgmngr/etc/crypto-policies --no-target-directory
‣ Acquiring lock on /var/cache/mkosi/fedora~40~x86-64/cache/dnf
‣ Acquired lock on /var/cache/mkosi/fedora~40~x86-64/cache/dnf
‣ Acquiring lock on /var/cache/mkosi/fedora~40~x86-64/lib/dnf
‣ Acquired lock on /var/cache/mkosi/fedora~40~x86-64/lib/dnf
‣ + cp --version
‣ + cp --recursive --no-dereference --preserve=mode,links --reflink=auto --copy-contents '/var/cache/mkosi/fedora~40~x86-64/cache/dnf' /var/tmp/mkosi-workspacedybhn7c6/root/var/cache/dnf --no-target-directory
‣ + cp --version
‣ + cp --recursive --no-dereference --preserve=mode,links --reflink=auto --copy-contents '/var/cache/mkosi/fedora~40~x86-64/lib/dnf' /var/tmp/mkosi-workspacedybhn7c6/root/var/lib/dnf --no-target-directory
‣ + rpm --eval '%{__plugindir}'
‣  Installing Fedora
‣ + dnf --assumeyes --best --releasever=40 --installroot=/buildroot --setopt=keepcache=1 --setopt=logdir=/var/log --setopt=cachedir=/var/cache/dnf --setopt=persistdir=/var/lib/dnf --setopt=install_weak_deps=0 --setopt=check_config_file_age=0 '--disableplugin=*' --enableplugin=builddep --setopt=debuglevel=10 --setopt=metadata_expire=never --config=/etc/dnf/dnf.conf --setopt=reposdir=/etc/yum.repos.d --setopt=varsdir=/etc/dnf/vars --setopt=proxy_sslcacert=/proxy.cacert install basesystem
timer: config: 1 ms
Loaded plugins: builddep
DNF version: 4.7.0
Command: dnf --assumeyes --best --releasever=40 --installroot=/buildroot --setopt=keepcache=1 --setopt=logdir=/var/log --setopt=cachedir=/var/cache/dnf --setopt=persistdir=/var/lib/dnf --setopt=install_weak_deps=0 --setopt=check_config_file_age=0 --disableplugin=* --enableplugin=builddep --setopt=debuglevel=10 --setopt=metadata_expire=never --config=/etc/dnf/dnf.conf --setopt=reposdir=/etc/yum.repos.d --setopt=varsdir=/etc/dnf/vars --setopt=proxy_sslcacert=/proxy.cacert install basesystem 
Installroot: /buildroot
Releasever: 40
cachedir: /buildroot/var/cache/dnf
Base command: install
Extra commands: ['--assumeyes', '--best', '--releasever=40', '--installroot=/buildroot', '--setopt=keepcache=1', '--setopt=logdir=/var/log', '--setopt=cachedir=/var/cache/dnf', '--setopt=persistdir=/var/lib/dnf', '--setopt=install_weak_deps=0', '--setopt=check_config_file_age=0', '--disableplugin=*', '--enableplugin=builddep', '--setopt=debuglevel=10', '--setopt=metadata_expire=never', '--config=/etc/dnf/dnf.conf', '--setopt=reposdir=/etc/yum.repos.d', '--setopt=varsdir=/etc/dnf/vars', '--setopt=proxy_sslcacert=/proxy.cacert', 'install', 'basesystem']
User-Agent: constructed: 'libdnf (Red Hat Enterprise Linux 8.9; generic; Linux.x86_64)'
repo: using cache for: fedora
fedora: using metadata from Sun Apr 14 18:51:11 2024.
repo: using cache for: updates
updates: using metadata from Mon May  6 02:17:22 2024.
Last metadata expiration check: 0:00:13 ago on Mon May  6 21:57:29 2024.
timer: sack setup: 1644 ms
--> Starting dependency resolution
---> Package basesystem.noarch 11-20.fc40 will be installed
---> Package fedora-gpg-keys.noarch 40-1 will be installed
---> Package fedora-repos.noarch 40-1 will be installed
---> Package filesystem.x86_64 3.18-8.fc40 will be installed
---> Package setup.noarch 2.14.5-2.fc40 will be installed
---> Package fedora-release.noarch 40-39 will be installed
---> Package fedora-release-common.noarch 40-39 will be installed
---> Package fedora-release-identity-basic.noarch 40-39 will be installed
--> Finished dependency resolution
timer: depsolve: 164 ms
Dependencies resolved.
===================================================================================================================================================================================================================
 Package                                                           Architecture                               Version                                            Repository                                   Size
===================================================================================================================================================================================================================
Installing:
 basesystem                                                        noarch                                     11-20.fc40                                         fedora                                      7.2 k
Installing dependencies:
 fedora-gpg-keys                                                   noarch                                     40-1                                               fedora                                      132 k
 fedora-release                                                    noarch                                     40-39                                              updates                                      11 k
 fedora-release-common                                             noarch                                     40-39                                              updates                                      21 k
 fedora-release-identity-basic                                     noarch                                     40-39                                              updates                                      12 k
 fedora-repos                                                      noarch                                     40-1                                               fedora                                      9.4 k
 filesystem                                                        x86_64                                     3.18-8.fc40                                        fedora                                      1.1 M
 setup                                                             noarch                                     2.14.5-2.fc40                                      fedora                                      155 k

Transaction Summary
===================================================================================================================================================================================================================
Install  8 Packages

Total size: 1.4 M
Installed size: 870 k
Downloading Packages:
[SKIPPED] basesystem-11-20.fc40.noarch.rpm: Already downloaded                                                                                                                                                    
[SKIPPED] fedora-gpg-keys-40-1.noarch.rpm: Already downloaded                                                                                                                                                     
[SKIPPED] fedora-repos-40-1.noarch.rpm: Already downloaded                                                                                                                                                        
[SKIPPED] filesystem-3.18-8.fc40.x86_64.rpm: Already downloaded                                                                                                                                                   
[SKIPPED] setup-2.14.5-2.fc40.noarch.rpm: Already downloaded                                                                                                                                                      
[SKIPPED] fedora-release-40-39.noarch.rpm: Already downloaded                                                                                                                                                     
[SKIPPED] fedora-release-common-40-39.noarch.rpm: Already downloaded                                                                                                                                              
[SKIPPED] fedora-release-identity-basic-40-39.noarch.rpm: Already downloaded                                                                                                                                      
Using rpmkeys executable at /bin/rpmkeys to verify signatures
warning: Generating 18 missing index(es), please wait...
fedora                                                                                                                                                                             8.5 kB/s | 4.6 kB     00:00    
Importing GPG key 0xEB10B464:
 Userid     : "Fedora (38) <fedora-38-primary@fedoraproject.org>"
 Fingerprint: 6A51 BBAB BA3D 5467 B617 1221 809A 8D7C EB10 B464
 From       : https://fedoraproject.org/fedora.gpg
Key imported successfully
Importing GPG key 0x18B8E74C:
 Userid     : "Fedora (39) <fedora-39-primary@fedoraproject.org>"
 Fingerprint: E8F2 3996 F232 1864 0CB4 4CBE 75CF 5AC4 18B8 E74C
 From       : https://fedoraproject.org/fedora.gpg
Key imported successfully
Importing GPG key 0xA15B79CC:
 Userid     : "Fedora (40) <fedora-40-primary@fedoraproject.org>"
 Fingerprint: 115D F9AE F857 853E E844 5D0A 0727 707E A15B 79CC
 From       : https://fedoraproject.org/fedora.gpg
Key imported successfully
Importing GPG key 0xE99D6AD1:
 Userid     : "Fedora (41) <fedora-41-primary@fedoraproject.org>"
 Fingerprint: 466C F2D8 B60B C305 7AA9 453E D062 2462 E99D 6AD1
 From       : https://fedoraproject.org/fedora.gpg
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
timer: transaction test: 54 ms
Running transaction
RPMDB altered outside of DNF.
RPM transaction start.
  Running scriptlet: filesystem-3.18-8.fc40.x86_64                                                                                                                                                             1/1 
  Preparing        :                                                                                                                                                                                           1/1 
Installed: fedora-release-identity-basic-40-39.noarch
  Installing       : fedora-release-identity-basic-40-39.noarch                                                                                                                                                1/8 
Installed: fedora-gpg-keys-40-1.noarch
  Installing       : fedora-gpg-keys-40-1.noarch                                                                                                                                                               2/8 
Installed: fedora-repos-40-1.noarch
  Installing       : fedora-repos-40-1.noarch                                                                                                                                                                  3/8 
Installed: fedora-release-common-40-39.noarch
  Installing       : fedora-release-common-40-39.noarch                                                                                                                                                        4/8 
Installed: fedora-release-40-39.noarch
  Installing       : fedora-release-40-39.noarch                                                                                                                                                               5/8 
Installed: setup-2.14.5-2.fc40.noarch
  Installing       : setup-2.14.5-2.fc40.noarch                                                                                                                                                                6/8 
  Running scriptlet: setup-2.14.5-2.fc40.noarch                                                                                                                                                                6/8 
Installed: filesystem-3.18-8.fc40.x86_64
  Installing       : filesystem-3.18-8.fc40.x86_64                                                                                                                                                             7/8 
Installed: basesystem-11-20.fc40.noarch
  Installing       : basesystem-11-20.fc40.noarch                                                                                                                                                              8/8 
  Running scriptlet: filesystem-3.18-8.fc40.x86_64                                                                                                                                                             8/8 
RPM transaction over.
  Verifying        : basesystem-11-20.fc40.noarch                                                                                                                                                              1/8 
  Verifying        : fedora-gpg-keys-40-1.noarch                                                                                                                                                               2/8 
  Verifying        : fedora-repos-40-1.noarch                                                                                                                                                                  3/8 
  Verifying        : filesystem-3.18-8.fc40.x86_64                                                                                                                                                             4/8 
  Verifying        : setup-2.14.5-2.fc40.noarch                                                                                                                                                                5/8 
  Verifying        : fedora-release-40-39.noarch                                                                                                                                                               6/8 
  Verifying        : fedora-release-common-40-39.noarch                                                                                                                                                        7/8 
  Verifying        : fedora-release-identity-basic-40-39.noarch                                                                                                                                                8/8 
timer: verify transaction: 237 ms
timer: transaction: 2713 ms
Installed: basesystem-11-20.fc40.noarch
Installed: fedora-gpg-keys-40-1.noarch
Installed: fedora-release-40-39.noarch
Installed: fedora-release-common-40-39.noarch
Installed: fedora-release-identity-basic-40-39.noarch
Installed: fedora-repos-40-1.noarch
Installed: filesystem-3.18-8.fc40.x86_64
Installed: setup-2.14.5-2.fc40.noarch

Installed:
  basesystem-11-20.fc40.noarch     fedora-gpg-keys-40-1.noarch    fedora-release-40-39.noarch    fedora-release-common-40-39.noarch    fedora-release-identity-basic-40-39.noarch    fedora-repos-40-1.noarch   
  filesystem-3.18-8.fc40.x86_64    setup-2.14.5-2.fc40.noarch    

Complete!
Cleaning up.
Plugins were unloaded.
‣ + dnf --assumeyes --best --releasever=40 --installroot=/buildroot --setopt=keepcache=1 --setopt=logdir=/var/log --setopt=cachedir=/var/cache/dnf --setopt=persistdir=/var/lib/dnf --setopt=install_weak_deps=0 --setopt=check_config_file_age=0 '--disableplugin=*' --enableplugin=builddep --setopt=debuglevel=10 --setopt=metadata_expire=never --config=/etc/dnf/dnf.conf --setopt=reposdir=/etc/yum.repos.d --setopt=varsdir=/etc/dnf/vars --setopt=proxy_sslcacert=/proxy.cacert install acl attr bash bubblewrap ca-certificates coreutils cpio diffutils dnf dosfstools e2fsprogs findutils grep jq kmod less mtools nano openssl sed socat strace swtpm systemd tar util-linux virtiofsd xfsprogs zstd apt createrepo_c curl-minimal debian-keyring distribution-gpg-keys dnf-plugins-core git-core openssh-clients policycoreutils python3-cryptography qemu-img qemu-kvm-core shadow-utils squashfs-tools swtpm-tools systemd-container systemd-journal-remote systemd-udev ubu-keyring virt-firmware xz grub2-tools edk2-ovmf pesign archlinux-keyring btrfs-progs dnf5 dnf5-plugins erofs-utils pacman qemu-system-aarch64-core qemu-system-ppc-core qemu-system-s390x-core reprepro systemd-ukify zypper sbsigntools
bwrap: Can't access /newroot/buildroot/proc/sysrq-trigger: Read-only file system
‣ "bwrap --die-with-parent --proc /proc --setenv SYSTEMD_OFFLINE 1 --unsetenv TMPDIR --tmpfs /tmp --unshare-ipc --dev /dev --symlink usr/bin /bin --symlink usr/sbin /sbin --symlink usr/lib /lib --symlink usr/lib64 /lib64 --setenv PATH /scripts:/usr/lib64/openmpi/bin:/usr/share/Modules/bin:/usr/local/sbin:/usr/sbin:/sbin:/bin:/usr/bin:/usr/local/bin:/var/lib/snapd/snap/bin:/opt/mssql-tools18/bin:/root/bin:/opt/mssql-tools18/bin:/root/bin:/opt/mssql-tools18/bin:/root/bin:/opt/mssql-tools18/bin:/opt/mssql-tools18/bin:/opt/mssql-tools18/bin --uid 0 --gid 0 --cap-add ALL --dir /work/src --chdir /work/src --bind /var/tmp/mkosi-workspacedybhn7c6/root /buildroot --bind /var/tmp/mkosi-workspacedybhn7c6/pkgmngr/etc /etc --ro-bind /etc/alternatives /etc/alternatives --ro-bind /etc/pki /etc/pki --bind /etc/resolv.conf /etc/resolv.conf --ro-bind /etc/ssl /etc/ssl --ro-bind /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem /proxy.cacert --ro-bind /usr /usr --bind /var/tmp/mkosi-workspacedybhn7c6/root/var/cache/dnf /var/cache/dnf --bind '/var/cache/mkosi/fedora~40~x86-64/cache/dnf/fedora-de0bb1deee98eda5/packages' /var/cache/dnf/fedora-de0bb1deee98eda5/packages --bind '/var/cache/mkosi/fedora~40~x86-64/cache/dnf/updates-4a8088a5c17ad284/packages' /var/cache/dnf/updates-4a8088a5c17ad284/packages --bind /var/tmp/mkosi-workspacedybhn7c6/root/var/lib/dnf /var/lib/dnf --bind /var/tmp/mkosi-workspacedybhn7c6/pkgmngr/var/log /var/log --bind /var/tmp/mkosi-workspacedybhn7c6/tmp/mkosi-var-tmp-7d8bd1cd072d414a /var/tmp --bind /var/tmp/mkosi-workspacedybhn7c6/packages /work/packages sh -c 'chmod 1777 /dev/shm && chmod 755 /etc && exec $0 "$@"' bwrap --dev-bind / / --tmpfs /buildroot/run --tmpfs /buildroot/tmp --bind /var/tmp /buildroot/var/tmp --proc /buildroot/proc --dev /buildroot/dev --ro-bind-try /buildroot/etc/machine-id /buildroot/etc/machine-id --dir /buildroot/run/user/0 --ro-bind-try /buildroot/etc/passwd /etc/passwd --ro-bind-try /buildroot/etc/group /etc/group --ro-bind-try /buildroot/etc/shadow /etc/shadow --ro-bind-try /buildroot/etc/gshadow /etc/gshadow sh -c 'chmod 1777 /buildroot/tmp /buildroot/var/tmp /buildroot/dev/shm && chmod 755 /buildroot/run && mkdir -m 755 /buildroot/run/host && echo mkosi >/buildroot/run/host/container-manager && exec $0 "$@"' dnf --assumeyes --best --releasever=40 --installroot=/buildroot --setopt=keepcache=1 --setopt=logdir=/var/log --setopt=cachedir=/var/cache/dnf --setopt=persistdir=/var/lib/dnf --setopt=install_weak_deps=0 --setopt=check_config_file_age=0 '--disableplugin=*' --enableplugin=builddep --setopt=debuglevel=10 --setopt=metadata_expire=never --config=/etc/dnf/dnf.conf --setopt=reposdir=/etc/yum.repos.d --setopt=varsdir=/etc/dnf/vars --setopt=proxy_sslcacert=/proxy.cacert install acl attr bash bubblewrap ca-certificates coreutils cpio diffutils dnf dosfstools e2fsprogs findutils grep jq kmod less mtools nano openssl sed socat strace swtpm systemd tar util-linux virtiofsd xfsprogs zstd apt createrepo_c curl-minimal debian-keyring distribution-gpg-keys dnf-plugins-core git-core openssh-clients policycoreutils python3-cryptography qemu-img qemu-kvm-core shadow-utils squashfs-tools swtpm-tools systemd-container systemd-journal-remote systemd-udev ubu-keyring virt-firmware xz grub2-tools edk2-ovmf pesign archlinux-keyring btrfs-progs dnf5 dnf5-plugins erofs-utils pacman qemu-system-aarch64-core qemu-system-ppc-core qemu-system-s390x-core reprepro systemd-ukify zypper sbsigntools" returned non-zero exit code 1.
‣ + rm -rf -- /var/tmp/mkosi-workspacedybhn7c6
Traceback (most recent call last):
  File "/root/mkosi/mkosi/run.py", line 60, in uncaught_exception_handler
    yield
  File "/root/mkosi/mkosi/run.py", line 101, in fork_and_wait
    target(*args, **kwargs)
  File "/root/mkosi/mkosi/__init__.py", line 4476, in run_build
    build_image(Context(args, config, workspace=workspace, resources=resources))
  File "/root/mkosi/mkosi/__init__.py", line 3664, in build_image
    install_distribution(context)
  File "/root/mkosi/mkosi/__init__.py", line 163, in install_distribution
    context.config.distribution.install_packages(context, context.config.packages)
  File "/root/mkosi/mkosi/distributions/__init__.py", line 134, in install_packages
    return self.installer().install_packages(context, packages)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/mkosi/mkosi/distributions/fedora.py", line 92, in install_packages
    Dnf.invoke(context, "install", packages, apivfs=apivfs)
  File "/root/mkosi/mkosi/installer/dnf.py", line 173, in invoke
    return run(
           ^^^^
  File "/root/mkosi/mkosi/run.py", line 150, in run
    with spawn(
  File "/usr/lib64/python3.11/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/root/mkosi/mkosi/run.py", line 265, in spawn
    raise subprocess.CalledProcessError(rc, prefix + cmdline)
subprocess.CalledProcessError: Command '['bwrap', '--die-with-parent', '--proc', '/proc', '--setenv', 'SYSTEMD_OFFLINE', '1', '--unsetenv', 'TMPDIR', '--tmpfs', '/tmp', '--unshare-ipc', '--dev', '/dev', '--symlink', 'usr/bin', '/bin', '--symlink', 'usr/sbin', '/sbin', '--symlink', 'usr/lib', '/lib', '--symlink', 'usr/lib64', '/lib64', '--setenv', 'PATH', '/scripts:/usr/lib64/openmpi/bin:/usr/share/Modules/bin:/usr/local/sbin:/usr/sbin:/sbin:/bin:/usr/bin:/usr/local/bin:/var/lib/snapd/snap/bin:/opt/mssql-tools18/bin:/root/bin:/opt/mssql-tools18/bin:/root/bin:/opt/mssql-tools18/bin:/root/bin:/opt/mssql-tools18/bin:/opt/mssql-tools18/bin:/opt/mssql-tools18/bin', '--uid', '0', '--gid', '0', '--cap-add', 'ALL', '--dir', '/work/src', '--chdir', '/work/src', '--bind', '/var/tmp/mkosi-workspacedybhn7c6/root', '/buildroot', '--bind', '/var/tmp/mkosi-workspacedybhn7c6/pkgmngr/etc', '/etc', '--ro-bind', '/etc/alternatives', '/etc/alternatives', '--ro-bind', '/etc/pki', '/etc/pki', '--bind', '/etc/resolv.conf', '/etc/resolv.conf', '--ro-bind', '/etc/ssl', '/etc/ssl', '--ro-bind', '/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem', '/proxy.cacert', '--ro-bind', '/usr', '/usr', '--bind', '/var/tmp/mkosi-workspacedybhn7c6/root/var/cache/dnf', '/var/cache/dnf', '--bind', '/var/cache/mkosi/fedora~40~x86-64/cache/dnf/fedora-de0bb1deee98eda5/packages', '/var/cache/dnf/fedora-de0bb1deee98eda5/packages', '--bind', '/var/cache/mkosi/fedora~40~x86-64/cache/dnf/updates-4a8088a5c17ad284/packages', '/var/cache/dnf/updates-4a8088a5c17ad284/packages', '--bind', '/var/tmp/mkosi-workspacedybhn7c6/root/var/lib/dnf', '/var/lib/dnf', '--bind', '/var/tmp/mkosi-workspacedybhn7c6/pkgmngr/var/log', '/var/log', '--bind', '/var/tmp/mkosi-workspacedybhn7c6/tmp/mkosi-var-tmp-7d8bd1cd072d414a', '/var/tmp', '--bind', '/var/tmp/mkosi-workspacedybhn7c6/packages', '/work/packages', 'sh', '-c', 'chmod 1777 /dev/shm && chmod 755 /etc && exec $0 "$@"', 'bwrap', '--dev-bind', '/', '/', '--tmpfs', '/buildroot/run', '--tmpfs', '/buildroot/tmp', '--bind', '/var/tmp', '/buildroot/var/tmp', '--proc', '/buildroot/proc', '--dev', '/buildroot/dev', '--ro-bind-try', '/buildroot/etc/machine-id', '/buildroot/etc/machine-id', '--dir', '/buildroot/run/user/0', '--ro-bind-try', '/buildroot/etc/passwd', '/etc/passwd', '--ro-bind-try', '/buildroot/etc/group', '/etc/group', '--ro-bind-try', '/buildroot/etc/shadow', '/etc/shadow', '--ro-bind-try', '/buildroot/etc/gshadow', '/etc/gshadow', 'sh', '-c', 'chmod 1777 /buildroot/tmp /buildroot/var/tmp /buildroot/dev/shm && chmod 755 /buildroot/run && mkdir -m 755 /buildroot/run/host && echo mkosi >/buildroot/run/host/container-manager && exec $0 "$@"', 'dnf', '--assumeyes', '--best', '--releasever=40', '--installroot=/buildroot', '--setopt=keepcache=1', '--setopt=logdir=/var/log', '--setopt=cachedir=/var/cache/dnf', '--setopt=persistdir=/var/lib/dnf', '--setopt=install_weak_deps=0', '--setopt=check_config_file_age=0', '--disableplugin=*', '--enableplugin=builddep', '--setopt=debuglevel=10', '--setopt=metadata_expire=never', '--config=/etc/dnf/dnf.conf', '--setopt=reposdir=/etc/yum.repos.d', '--setopt=varsdir=/etc/dnf/vars', '--setopt=proxy_sslcacert=/proxy.cacert', 'install', 'acl', 'attr', 'bash', 'bubblewrap', 'ca-certificates', 'coreutils', 'cpio', 'diffutils', 'dnf', 'dosfstools', 'e2fsprogs', 'findutils', 'grep', 'jq', 'kmod', 'less', 'mtools', 'nano', 'openssl', 'sed', 'socat', 'strace', 'swtpm', 'systemd', 'tar', 'util-linux', 'virtiofsd', 'xfsprogs', 'zstd', 'apt', 'createrepo_c', 'curl-minimal', 'debian-keyring', 'distribution-gpg-keys', 'dnf-plugins-core', 'git-core', 'openssh-clients', 'policycoreutils', 'python3-cryptography', 'qemu-img', 'qemu-kvm-core', 'shadow-utils', 'squashfs-tools', 'swtpm-tools', 'systemd-container', 'systemd-journal-remote', 'systemd-udev', 'ubu-keyring', 'virt-firmware', 'xz', 'grub2-tools', 'edk2-ovmf', 'pesign', 'archlinux-keyring', 'btrfs-progs', 'dnf5', 'dnf5-plugins', 'erofs-utils', 'pacman', 'qemu-system-aarch64-core', 'qemu-system-ppc-core', 'qemu-system-s390x-core', 'reprepro', 'systemd-ukify', 'zypper', 'sbsigntools']' returned non-zero exit status 1.
‣ + tput cnorm
‣ + tput smam

[disjustin]

also tested and reproduced with CentOS Stream 9

uname -r
5.14.0-432.el9.x86_64

[DaanDeMeyer]

I backported the required patch for this to centos stream 9: containers/bubblewrap@4c35d7a a month ago. Nothing more I can do on my end, it's up to red hat to make sure the new build of bubblewrap pops up in centos stream 9