builddah build still failing on unpacking rpm package

[tsailiming]

Description

Doing a yum update on Centos8-stream base image results in the following

Upgrading : filesystem-3.8-4.el8.0.1.x86_64 2/203
Error unpacking rpm package filesystem-3.8-4.el8.0.1.x86_64
Installing : cracklib-dicts-2.9.6-15.el8.x86_64 3/203
error: unpacking of archive failed on file /proc: cpio: chown failed - Inappropriate ioctl for device
error: filesystem-3.8-4.el8.0.1.x86_64: install failed

Steps to reproduce the issue:

1. buildah bud -f Dockerfile -t openshift-squid:test

My Dockerfile

FROM quay.io/centos/centos:stream8

RUN yum clean all -y && yum -y update && yum install -y squid && yum clean all -y

Describe the results you received:

Failed:
  filesystem-3.8-4.el8.x86_64          filesystem-3.8-4.el8.0.1.x86_64         Error: Transaction failed


error building at STEP "RUN yum clean all -y && yum -y update && yum install -y squid && yum clean all -y": error while running runtime: exit status 1

Describe the results you expected:
This should work based. Based on past issues, this should have already been fixed.

Tried with this and same error.

buildah bud --cap-add CAP_SETFCAP -f Dockerfile -t openshift-squid:test

Output of rpm -q buildah or apt list buildah:

$ rpm -qa | grep buildah
buildah-1.21.0-1.fc34.x86_64

Output of buildah version:

$ buildah --version
buildah version 1.21.0 (image-spec 1.0.1-dev, runtime-spec 1.0.2-dev)

Output of cat /etc/*release:

$ cat /etc/fedora-release 
Fedora release 34 (Thirty Four)

Output of uname -a:

$ uname -a
Linux build.local 5.12.10-300.fc34.x86_64 #1 SMP Thu Jun 10 14:21:36 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Output of cat /etc/containers/storage.conf:

$ cat /etc/containers/storage.conf | grep -v ^#
[storage]
driver = "overlay"
runroot = "/run/containers/storage"
graphroot = "/var/lib/containers/storage"

[storage.options]
additionalimagestores = [
]

[storage.options.overlay]
mountopt = "nodev,metacopy=on"
[storage.options.thinpool]

[tsailiming]

Using this worked.

# buildah --version
buildah version 1.19.8 (image-spec 1.0.1-dev, runtime-spec 1.0.2-dev)

# rpm -qa | grep buildah
buildah-1.19.7-2.module+el8.4.0+11311+9da8acfb.x86_64

# cat /etc/redhat-release 
Red Hat Enterprise Linux release 8.3 (Ootpa)

# uname -a
Linux ansible.home.com 4.18.0-240.el8.x86_64 #1 SMP Wed Sep 23 05:13:10 EDT 2020 x86_64 x86_64 x86_64 GNU/Linux

[rhatdan]

@nalind Any ideas?

[nalind]

When run in rootless mode, crun and runc are giving us a /proc that's owned by an unmapped user (which appears as 65534:65534) rather than by the in-container root user, so if we attempt to upgrade the filesystem package, RPM fails because it can't set the permissions on /proc to match the 0:0 specified in the package's payload.

@tsailiming, are you running as root or as a non-root user?

[tsailiming]

I tried with root and it worked on my fc34 box. On RHEL 8.3, it was already running as root.

[rhatdan]

So there is nothing we can do to fix this in rootless mode. I think the filesystem package would need to be able to handle this. Or we just mark this package as a failure in rootless mode installs. There is no way for us to change the permissions of /proc in rootless mode. It will always we undefined.

$ podman run alpine ls -ld /proc
dr-xr-xr-x 405 nobody nobody 0 Jun 18 15:22 /proc

@giuseppe Any thoughts on this?

[giuseppe]

@giuseppe Any thoughts on this?

proc is owned by root in initial namespace, so root in a user namespace doesn't own it. I don't think there is anything runc/crun can do about it, the issue can be easily reproduced just with unshare:

$ unshare -rmp -f --mount-proc=/proc ls -ldn /proc
dr-xr-xr-x. 458 65534 65534 0 Jun 21 10:53 /proc

$ unshare -rmp -f --mount-proc=/proc touch /proc
touch: setting times of '/proc': Permission denied

[rhatdan]

You can not mount /proc after creating the user namespace?

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

Since there is no way for us to fix, this needs to be fixed in the filesystem package.