Rootless podman on CircleCI fails to even build with error running container: from /usr/bin/crun: sd-bus call: Permission denied #16529

adelton · 2022-11-16T08:21:21Z

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

I select bug here but this is more a question about what the assumptions are about the setup and what could be causing the crun / sd-dbus Permission deniced / Interactive authentication required failure or what configuration change to try.

Description

I try to add testing https://github.com/freeipa/freeipa-container on CircleCI on their Ubuntu 22.04 VMs.

Compared to GitHub Actions Ubuntu 22.04 where this works without issues once a session gets created by ssh to self, on CircleCI where loginctl shows that we already have session I'm hitting

STEP 2/2: RUN date
error running container: error from /usr/bin/crun creating container for [/bin/sh -c date]: sd-bus call: Permission denied
: exit status 1
Error: error building at STEP "RUN date": error while running runtime: exit status 1
Exited with code exit status 125

or with podman from download.opensuse.org/repositories/devel:kubic:libcontainers:unstable/

STEP 2/2: RUN date
error running container: from /usr/bin/crun creating container for [/bin/sh -c date]: sd-bus call: Interactive authentication required.: Permission denied
: exit status 1
ERRO[0003] did not get container create message from subprocess: EOF 
Error: building at STEP "RUN date": while running runtime: exit status 1
Exited with code exit status 1

Steps to reproduce the issue:

Have .circleci/config.yml in your project's repo with content to install podman and build an image with it in rootless fashion

version: 2.1

jobs:
  build-image:
    machine:
      image: ubuntu-2204:edge
    resource_class: arm.medium
    steps:
      - run: sudo apt-get -y update
      - run: sudo apt-get -y install podman
      - run: podman version
      - run: podman info
      - run: apt list podman
      - run: loginctl
      - run: ( echo FROM quay.io/centos/centos:stream9 ; echo RUN date ) > Dockerfile.circleci
      - run: podman build -f Dockerfile.circleci .

workflows:
  test-podman:
    jobs:
      - build-image

Enable CircleCI, push, watch the pipeline output.

Describe the results you received:

https://app.circleci.com/pipelines/github/adelton/freeipa-container/74/workflows/9a11b1d9-4040-4d9f-a005-e50532d4bb44/jobs/503

podman build -f Dockerfile.circleci .

STEP 1/2: FROM quay.io/centos/centos:stream9
Trying to pull quay.io/centos/centos:stream9...
Getting image source signatures
…
Writing manifest to image destination
Storing signatures
STEP 2/2: RUN date
error running container: error from /usr/bin/crun creating container for [/bin/sh -c date]: sd-bus call: Permission denied
: exit status 1
Error: error building at STEP "RUN date": error while running runtime: exit status 1
Exited with code exit status 125

Describe the results you expected:

No error, image built.

Additional information you deem important (e.g. issue happens only occasionally):

The loginctl shows we are running in a session:

SESSION  UID USER     SEAT TTY
     c1 1001 circleci      

1 sessions listed.

I also tried systemd-run --scope --user and ssh to self to no avail. It seems like we have session alright, it just needs some permissions somewhere.

I get this on the CircleCI Ubuntu VMs in deterministic fashion. When I change the

      - run: sudo apt-get -y update
      - run: sudo apt-get -y install podman

steps to

      - run: sudo mkdir -p /etc/apt/keyrings
      - run: curl -fsSL https://download.opensuse.org/repositories/devel:kubic:libcontainers:unstable/xUbuntu_$(lsb_release -rs)/Release.key
             | gpg --dearmor
             | sudo tee /etc/apt/keyrings/devel_kubic_libcontainers_unstable.gpg > /dev/null
      - run: echo
             "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/devel_kubic_libcontainers_unstable.gpg]
             https://download.opensuse.org/repositories/devel:kubic:libcontainers:unstable/xUbuntu_$(lsb_release -rs)/ /"
             | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:unstable.list > /dev/null
      - run: sudo apt-get update -y
      - run: sudo apt-get install -y podman skopeo

to test with the latest podman per https://podman.io/getting-started/installation#ubuntu, the output is slightly different but still error:

https://app.circleci.com/pipelines/github/adelton/freeipa-container/73/workflows/a086e653-cd68-47f9-851c-54a79d849c7f/jobs/502

podman build -f Dockerfile.circleci .

STEP 1/2: FROM quay.io/centos/centos:stream9
Trying to pull quay.io/centos/centos:stream9...
Getting image source signatures
Writing manifest to image destination
Storing signatures
STEP 2/2: RUN date
error running container: from /usr/bin/crun creating container for [/bin/sh -c date]: sd-bus call: Interactive authentication required.: Permission denied
: exit status 1
ERRO[0003] did not get container create message from subprocess: EOF 
Error: building at STEP "RUN date": while running runtime: exit status 1
Exited with code exit status 1

Output of podman version:

Version:      3.4.4
API Version:  3.4.4
Go Version:   go1.17.3
Built:        Thu Jan  1 00:00:00 1970
OS/Arch:      linux/arm64
WARN[0000] Failed to add pause process to systemd sandbox cgroup: dbus: invalid bus address (no transport)

Output of podman info:

host:
  arch: arm64
  buildahVersion: 1.23.1
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: 'conmon: /usr/bin/conmon'
    path: /usr/bin/conmon
    version: 'conmon version 2.0.25, commit: unknown'
  cpus: 2
  distribution:
    codename: jammy
    distribution: ubuntu
    version: "22.04"
  eventLogger: journald
  hostname: ip-172-28-21-69
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1002
      size: 1
    - container_id: 1
      host_id: 165536
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1001
      size: 1
    - container_id: 1
      host_id: 165536
      size: 65536
  kernel: 5.15.0-1022-aws
  linkmode: dynamic
  logDriver: journald
  memFree: 6990278656
  memTotal: 8116731904
  ociRuntime:
    name: crun
    package: 'crun: /usr/bin/crun'
    path: /usr/bin/crun
    version: |-
      crun version 0.17
      commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    path: /run/user/1001/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: 'slirp4netns: /usr/bin/slirp4netns'
    version: |-
      slirp4netns version 1.0.1
      commit: 6a7b16babc95b6a3056b33fb45b74a6f62262dd4
      libslirp: 4.6.1
  swapFree: 0
  swapTotal: 0
  uptime: 1m 59.56s
plugins:
  log:
  - k8s-file
  - none
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries: {}
store:
  configFile: /home/circleci/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/circleci/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 0
  runRoot: /run/user/1001/containers
  volumePath: /home/circleci/.local/share/containers/storage/volumes
version:
  APIVersion: 3.4.4
  Built: 0
  BuiltTime: Thu Jan  1 00:00:00 1970
  GitCommit: ""
  GoVersion: go1.17.3
  OsArch: linux/arm64
  Version: 3.4.4

Package info (e.g. output of rpm -q podman or apt list podman or brew info podman):

Listing... Done
podman/jammy,now 3.4.4+ds1-1ubuntu1 arm64 [installed]

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes; No

Additional environment details (AWS, VirtualBox, physical, etc.):

This is on CircleCI Ubuntu 22.04 ARM VM.

The text was updated successfully, but these errors were encountered:

Luap99 · 2022-11-16T13:30:51Z

Sounds like you are missing the systemd user session. Can you try with the latest version?

adelton · 2022-11-16T22:33:08Z

When I added loginctl session-status, it shows

c1 - circleci (1001)
	   Since: Wed 2022-11-16 22:30:24 UTC; 32s ago
	  Leader: 916 (sudo)
	  Remote: user root
	 Service: sudo-i; type unspecified; class background
	   State: active
	    Unit: session-c1.scope
		  ├─ 916 sudo -niHu circleci -- /tmp/circleci-machine-agent2515740742/circleci-agent _internal agent-runner --config -
		  ├─ 925 /tmp/circleci-machine-agent2515740742/circleci-agent _internal agent-runner --config -
		  ├─3732 podman
		  ├─4772 /bin/bash -eo pipefail -c "loginctl session-status | cat"
		  ├─5095 loginctl session-status
		  └─5096 cat

Nov 16 22:30:27 ip-172-28-2-234 sudo[1289]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1001)
Nov 16 22:30:35 ip-172-28-2-234 sudo[1289]: pam_unix(sudo:session): session closed for user root
Nov 16 22:30:36 ip-172-28-2-234 sudo[2278]: circleci : TTY=pts/0 ; PWD=/home/circleci/project ; USER=root ; COMMAND=/usr/bin/apt-get -y install podman
Nov 16 22:30:36 ip-172-28-2-234 sudo[2278]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1001)
Nov 16 22:30:43 ip-172-28-2-234 useradd[2840]: new user: name=dnsmasq, UID=115, GID=65534, home=/var/lib/misc, shell=/usr/sbin/nologin, from=/dev/pts/2
Nov 16 22:30:43 ip-172-28-2-234 usermod[2847]: change user 'dnsmasq' password
Nov 16 22:30:43 ip-172-28-2-234 chage[2854]: changed password expiry for dnsmasq
Nov 16 22:30:43 ip-172-28-2-234 chfn[2858]: changed user 'dnsmasq' information
Nov 16 22:30:53 ip-172-28-2-234 sudo[2278]: pam_unix(sudo:session): session closed for user root
Nov 16 22:30:54 ip-172-28-2-234 podman[3728]: 2022-11-16 22:30:54.002908492 +0000 UTC m=+0.163240424 system refresh

-- see https://app.circleci.com/pipelines/github/adelton/freeipa-container/98/workflows/9301ab35-a5fd-48a5-9d90-c4090232a2d1/jobs/505. So there is some user session there. Anything specific about it that I should look for?

What latest version do you have in mind? I install podman from download.opensuse.org/repositories/devel:kubic:libcontainers:unstable/ in this case.

Luap99 · 2022-11-17T13:59:45Z

sudo or su will not create a proper systemd user session, see https://github.com/containers/podman/blob/main/troubleshooting.md#31-podman-run-fails-with-erro0000-xdg_runtime_dir-directory-runuser0-is-not-owned-by-the-current-user-or-error-creating-tmpdir-mkdir-runuser1000-permission-denied for alternatives.

I don't know much about the packaging but I assume the kubic repo has a more up to date version so I would try that one.
I think it is very likely that this is already fixed in newer versions. I remember seeing issues like that before.

adelton · 2022-11-17T15:57:50Z

As I already mentioned above:

I also tried systemd-run --scope --user and ssh to self to no avail. It seems like we have session alright, it just needs some permissions somewhere.

I assume CircleCI falls into the category of "badly configured systemd session" at https://github.com/containers/podman/blob/main/troubleshooting.md#31-podman-run-fails-with-erro0000-xdg_runtime_dir-directory-runuser0-is-not-owned-by-the-current-user-or-error-creating-tmpdir-mkdir-runuser1000-permission-denied -- but what specifically should I be looking for? What sd-bus operations does podman do?

What latest version do you have in mind? I install podman from download.opensuse.org/repositories/devel:kubic:libcontainers:unstable/ in this case.

What kubic repo do you have in mind, different from this unstable one I've been trying it with?

Luap99 · 2022-11-17T17:16:45Z

Yes I think this is the correct kubic repo.

Podman doesn't do anything with sd-bus here. It is crun which is failing.

/usr/bin/crun creating container for [/bin/sh -c date]: sd-bus call: Permission denied

/usr/bin/crun creating container for [/bin/sh -c date]: sd-bus call: Interactive authentication required.: Permission denied

cc @giuseppe

@adelton Does a normal podman run work? You could also try installing runc and using this as your runtime for testing.

adelton · 2022-11-17T17:52:18Z

What do you mean exactly by normal podman? I thought that my use of podman was quite normal. :-)

When I installed runc and added --log-level=debug, I see

DEBU[0003] setting image creation date to 2022-11-11 16:00:36.341508704 +0000 UTC 
DEBU[0003] created new image ID "e08717ee1d9523552549e565e6cae5fadee41e089594db802f02dfd1ae115167" 
DEBU[0003] saved image metadata "{\"signatures-sizes\":{\"sha256:7e593e51341bce240b4d0893e9d0e508b810ab84176fedd9027c4e34636e910b\":[]}}" 
DEBU[0003] set names of image "e08717ee1d9523552549e565e6cae5fadee41e089594db802f02dfd1ae115167" to [quay.io/centos/centos:stream9] 
DEBU[0003] Pulled candidate quay.io/centos/centos:stream9 successfully 
DEBU[0003] Looking up image "e08717ee1d9523552549e565e6cae5fadee41e089594db802f02dfd1ae115167" in local containers storage 
DEBU[0003] Trying "e08717ee1d9523552549e565e6cae5fadee41e089594db802f02dfd1ae115167" ... 
DEBU[0003] parsed reference into "[overlay@/home/circleci/.local/share/containers/storage+/run/user/1001/containers]@e08717ee1d9523552549e565e6cae5fadee41e089594db802f02dfd1ae115167" 
DEBU[0003] Found image "e08717ee1d9523552549e565e6cae5fadee41e089594db802f02dfd1ae115167" as "e08717ee1d9523552549e565e6cae5fadee41e089594db802f02dfd1ae115167" in local containers storage 
DEBU[0003] Found image "e08717ee1d9523552549e565e6cae5fadee41e089594db802f02dfd1ae115167" as "e08717ee1d9523552549e565e6cae5fadee41e089594db802f02dfd1ae115167" in local containers storage ([overlay@/home/circleci/.local/share/containers/storage+/run/user/1001/containers]@e08717ee1d9523552549e565e6cae5fadee41e089594db802f02dfd1ae115167) 
DEBU[0003] exporting opaque data as blob "sha256:e08717ee1d9523552549e565e6cae5fadee41e089594db802f02dfd1ae115167" 
DEBU[0003] exporting opaque data as blob "sha256:e08717ee1d9523552549e565e6cae5fadee41e089594db802f02dfd1ae115167" 
DEBU[0003] [graphdriver] trying provided driver "overlay" 
DEBU[0003] cached value indicated that overlay is supported 
DEBU[0003] cached value indicated that metacopy is not being used 
DEBU[0003] backingFs=extfs, projectQuotaSupported=false, useNativeDiff=true, usingMetacopy=false 
DEBU[0003] overlay test mount indicated that volatile is being used 
DEBU[0003] overlay: mount_data=,lowerdir=/home/circleci/.local/share/containers/storage/overlay/l/XWGI7OQW2Y6VRWIMSUNKVYFN4S,upperdir=/home/circleci/.local/share/containers/storage/overlay/de22f85c78cf33119a12c2c8d158584d71c35917b774059a40805a4a3da77047/diff,workdir=/home/circleci/.local/share/containers/storage/overlay/de22f85c78cf33119a12c2c8d158584d71c35917b774059a40805a4a3da77047/work,userxattr,volatile 
DEBU[0003] Container ID: 4b329f72580b21c1afb45e93ad40b591919dd2fe476c3423278ac242468d798a 
DEBU[0003] Parsed Step: {Env:[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin] Command:run Args:[date] Flags:[] Attrs:map[] Message:RUN date Original:RUN date} 
STEP 2/2: RUN date
DEBU[0003] parsed reference into "[overlay@/home/circleci/.local/share/containers/storage+/run/user/1001/containers]@e08717ee1d9523552549e565e6cae5fadee41e089594db802f02dfd1ae115167" 
DEBU[0003] exporting opaque data as blob "sha256:e08717ee1d9523552549e565e6cae5fadee41e089594db802f02dfd1ae115167" 
DEBU[0003] RUN imagebuilder.Run{Shell:true, Args:[]string{"date"}, Mounts:[]string(nil)}, docker.Config{Hostname:"", Domainname:"", User:"", Memory:0, MemorySwap:0, MemoryReservation:0, KernelMemory:0, CPUShares:0, CPUSet:"", PortSpecs:[]string(nil), ExposedPorts:map[docker.Port]struct {}{}, PublishService:"", StopSignal:"", StopTimeout:0, Env:[]string{"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"}, Cmd:[]string{"/bin/bash"}, Shell:[]string{}, Healthcheck:(*docker.HealthConfig)(nil), DNS:[]string(nil), Image:"", Volumes:map[string]struct {}{}, VolumeDriver:"", WorkingDir:"", MacAddress:"", Entrypoint:[]string{}, SecurityOpts:[]string(nil), OnBuild:[]string{}, Mounts:[]docker.Mount(nil), Labels:map[string]string{"io.buildah.version":"1.19.8", "org.label-schema.build-date":"20221111", "org.label-schema.license":"GPLv2", "org.label-schema.name":"CentOS Stream 9 Base Image", "org.label-schema.schema-version":"1.0", "org.label-schema.vendor":"CentOS"}, AttachStdin:false, AttachStdout:false, AttachStderr:false, ArgsEscaped:false, Tty:false, OpenStdin:false, StdinOnce:false, NetworkDisabled:false, VolumesFrom:""} 
DEBU[0003] using "/var/tmp/buildah1094495047" to hold bundle data 
DEBU[0003] Forcing use of an IPC namespace.             
DEBU[0003] Forcing use of a PID namespace.              
DEBU[0003] Forcing use of a user namespace.             
DEBU[0003] Resources: &define.CommonBuildOptions{AddHost:[]string{}, CgroupParent:"", CPUPeriod:0x0, CPUQuota:0, CPUShares:0x0, CPUSetCPUs:"", CPUSetMems:"", HTTPProxy:true, Memory:0, DNSSearch:[]string{}, DNSServers:[]string{}, DNSOptions:[]string{}, MemorySwap:0, LabelOpts:[]string(nil), OmitTimestamp:false, SeccompProfilePath:"/usr/share/containers/seccomp.json", ApparmorProfile:"", ShmSize:"65536k", Ulimit:[]string{}, Volumes:[]string{}, Secrets:[]string{}, SSHSources:[]string{}} 
DEBU[0004] /etc/system-fips does not exist on host, not mounting FIPS mode subscription 
DEBU[0000] bind mounted "/home/circleci/.local/share/containers/storage/overlay/de22f85c78cf33119a12c2c8d158584d71c35917b774059a40805a4a3da77047/merged" to "/var/tmp/buildah1094495047/mnt/rootfs" 
DEBU[0000] config = {"ociVersion":"1.0.2-dev","process":{"user":{"uid":0,"gid":0},"args":["/bin/sh","-c","date"],"env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin","HOSTNAME=4b329f72580b"],"cwd":"/","capabilities":{"bounding":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FOWNER","CAP_FSETID","CAP_KILL","CAP_NET_BIND_SERVICE","CAP_SETFCAP","CAP_SETGID","CAP_SETPCAP","CAP_SETUID","CAP_SYS_CHROOT"],"effective":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FOWNER","CAP_FSETID","CAP_KILL","CAP_NET_BIND_SERVICE","CAP_SETFCAP","CAP_SETGID","CAP_SETPCAP","CAP_SETUID","CAP_SYS_CHROOT"],"inheritable":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FOWNER","CAP_FSETID","CAP_KILL","CAP_NET_BIND_SERVICE","CAP_SETFCAP","CAP_SETGID","CAP_SETPCAP","CAP_SETUID","CAP_SYS_CHROOT"],"permitted":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FOWNER","CAP_FSETID","CAP_KILL","CAP_NET_BIND_SERVICE","CAP_SETFCAP","CAP_SETGID","CAP_SETPCAP","CAP_SETUID","CAP_SYS_CHROOT"],"ambient":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_FOWNER","CAP_FSETID","CAP_KILL","CAP_NET_BIND_SERVICE","CAP_SETFCAP","CAP_SETGID","CAP_SETPCAP","CAP_SETUID","CAP_SYS_CHROOT"]},"rlimits":[{"type":"RLIMIT_NOFILE","hard":1024,"soft":1024}]},"root":{"path":"/var/tmp/buildah1094495047/mnt/rootfs"},"hostname":"4b329f72580b","mounts":[{"destination":"/dev","type":"tmpfs","source":"/dev","options":["private","strictatime","noexec","nosuid","mode=755","size=65536k"]},{"destination":"/dev/mqueue","type":"mqueue","source":"mqueue","options":["private","nodev","noexec","nosuid"]},{"destination":"/dev/pts","type":"devpts","source":"pts","options":["private","noexec","nosuid","newinstance","ptmxmode=0666","mode=0620"]},{"destination":"/dev/shm","type":"tmpfs","source":"shm","options":["private","nodev","noexec","nosuid","mode=1777","size=65536k"]},{"destination":"/proc","type":"proc","source":"/proc","options":["private","nodev","noexec","nosuid"]},{"destination":"/sys","type":"bind","source":"/sys","options":["rbind","private","nodev","noexec","nosuid","ro"]},{"destination":"/etc/hosts","type":"bind","source":"/var/tmp/buildah1094495047/hosts","options":["rbind"]},{"destination":"/etc/resolv.conf","type":"bind","source":"/var/tmp/buildah1094495047/resolv.conf","options":["rbind"]},{"destination":"/run/.containerenv","type":"bind","source":"/var/tmp/buildah1094495047/run/.containerenv","options":["rbind"]}],"linux":{"uidMappings":[{"containerID":0,"hostID":0,"size":1},{"containerID":1,"hostID":1,"size":65536}],"gidMappings":[{"containerID":0,"hostID":0,"size":1},{"containerID":1,"hostID":1,"size":65536}],"namespaces":[{"type":"pid"},{"type":"ipc"},{"type":"uts"},{"type":"mount"},{"type":"user"}],"seccomp":{"defaultAction":"SCMP_ACT_ERRNO","defaultErrnoRet":38,"architectures":["SCMP_ARCH_AARCH64","SCMP_ARCH_ARM"],"syscalls":[{"names":["bdflush","io_pgetevents","kexec_file_load","kexec_load","migrate_pages","move_pages","nfsservctl","nice","oldfstat","oldlstat","oldolduname","oldstat","olduname","pciconfig_iobase","pciconfig_read","pciconfig_write","sgetmask","ssetmask","swapcontext","swapoff","swapon","sysfs","uselib","userfaultfd","ustat","vm86","vm86old","vmsplice"],"action":"SCMP_ACT_ERRNO","errnoRet":1},{"names":["_llseek","_newselect","accept","accept4","access","adjtimex","alarm","bind","brk","capget","capset","chdir","chmod","chown","chown32","clock_adjtime","clock_adjtime64","clock_getres","clock_getres_time64","clock_gettime","clock_gettime64","clock_nanosleep","clock_nanosleep_time64","clone","clone3","close","close_range","connect","copy_file_range","creat","dup","dup2","dup3","epoll_create","epoll_create1","epoll_ctl","epoll_ctl_old","epoll_pwait","epoll_pwait2","epoll_wait","epoll_wait_old","eventfd","eventfd2","execve","execveat","exit","exit_group","faccessat","faccessat2","fadvise64","fadvise64_64","fallocate","fanotify_mark","fchdir","fchmod","fchmodat","fchown","fchown32","fchownat","fcntl","fcntl64","fdatasync","fgetxattr","flistxattr","flock","fork","fremovexattr","fsconfig","fsetxattr","fsmount","fsopen","fspick","fstat","fstat64","fstatat64","fstatfs","fstatfs64","fsync","ftruncate","ftruncate64","futex","futex_time64","futimesat","get_robust_list","get_thread_area","getcpu","getcwd","getdents","getdents64","getegid","getegid32","geteuid","geteuid32","getgid","getgid32","getgroups","getgroups32","getitimer","get_mempolicy","getpeername","getpgid","getpgrp","getpid","getppid","getpriority","getrandom","getresgid","getresgid32","getresuid","getresuid32","getrlimit","getrusage","getsid","getsockname","getsockopt","gettid","gettimeofday","getuid","getuid32","getxattr","inotify_add_watch","inotify_init","inotify_init1","inotify_rm_watch","io_cancel","io_destroy","io_getevents","io_setup","io_submit","ioctl","ioprio_get","ioprio_set","ipc","keyctl","kill","lchown","lchown32","lgetxattr","link","linkat","listen","listxattr","llistxattr","lremovexattr","lseek","lsetxattr","lstat","lstat64","madvise","mbind","memfd_create","memfd_secret","mincore","mkdir","mkdirat","mknod","mknodat","mlock","mlock2","mlockall","mmap","mmap2","mount","move_mount","mprotect","mq_getsetattr","mq_notify","mq_open","mq_timedreceive","mq_timedreceive_time64","mq_timedsend","mq_timedsend_time64","mq_unlink","mremap","msgctl","msgget","msgrcv","msgsnd","msync","munlock","munlockall","munmap","name_to_handle_at","nanosleep","newfstatat","open","openat","openat2","open_tree","pause","pidfd_getfd","pidfd_open","pidfd_send_signal","pipe","pipe2","pivot_root","pkey_alloc","pkey_free","pkey_mprotect","poll","ppoll","ppoll_time64","prctl","pread64","preadv","preadv2","prlimit64","pselect6","pselect6_time64","pwrite64","pwritev","pwritev2","read","readahead","readdir","readlink","readlinkat","readv","reboot","recv","recvfrom","recvmmsg","recvmmsg_time64","recvmsg","remap_file_pages","removexattr","rename","renameat","renameat2","restart_syscall","rmdir","rseq","rt_sigaction","rt_sigpending","rt_sigprocmask","rt_sigqueueinfo","rt_sigreturn","rt_sigsuspend","rt_sigtimedwait","rt_sigtimedwait_time64","rt_tgsigqueueinfo","sched_get_priority_max","sched_get_priority_min","sched_getaffinity","sched_getattr","sched_getparam","sched_getscheduler","sched_rr_get_interval","sched_rr_get_interval_time64","sched_setaffinity","sched_setattr","sched_setparam","sched_setscheduler","sched_yield","seccomp","select","semctl","semget","semop","semtimedop","semtimedop_time64","send","sendfile","sendfile64","sendmmsg","sendmsg","sendto","setns","set_mempolicy","set_robust_list","set_thread_area","set_tid_address","setfsgid","setfsgid32","setfsuid","setfsuid32","setgid","setgid32","setgroups","setgroups32","setitimer","setpgid","setpriority","setregid","setregid32","setresgid","setresgid32","setresuid","setresuid32","setreuid","setreuid32","setrlimit","setsid","setsockopt","setuid","setuid32","setxattr","shmat","shmctl","shmdt","shmget","shutdown","sigaltstack","signalfd","signalfd4","sigreturn","socketcall","socketpair","splice","stat","stat64","statfs","statfs64","statx","symlink","symlinkat","sync","sync_file_range","syncfs","sysinfo","syslog","tee","tgkill","time","timer_create","timer_delete","timer_getoverrun","timer_gettime","timer_gettime64","timer_settime","timer_settime64","timerfd_create","timerfd_gettime","timerfd_gettime64","timerfd_settime","timerfd_settime64","times","tkill","truncate","truncate64","ugetrlimit","umask","umount","umount2","uname","unlink","unlinkat","unshare","utime","utimensat","utimensat_time64","utimes","vfork","wait4","waitid","waitpid","write","writev"],"action":"SCMP_ACT_ALLOW"},{"names":["personality"],"action":"SCMP_ACT_ALLOW","args":[{"index":0,"value":0,"op":"SCMP_CMP_EQ"}]},{"names":["personality"],"action":"SCMP_ACT_ALLOW","args":[{"index":0,"value":8,"op":"SCMP_CMP_EQ"}]},{"names":["personality"],"action":"SCMP_ACT_ALLOW","args":[{"index":0,"value":131072,"op":"SCMP_CMP_EQ"}]},{"names":["personality"],"action":"SCMP_ACT_ALLOW","args":[{"index":0,"value":131080,"op":"SCMP_CMP_EQ"}]},{"names":["personality"],"action":"SCMP_ACT_ALLOW","args":[{"index":0,"value":4294967295,"op":"SCMP_CMP_EQ"}]},{"names":["arm_fadvise64_64","arm_sync_file_range","sync_file_range2","breakpoint","cacheflush","set_tls"],"action":"SCMP_ACT_ALLOW"},{"names":["open_by_handle_at"],"action":"SCMP_ACT_ERRNO","errnoRet":1},{"names":["bpf","fanotify_init","lookup_dcookie","perf_event_open","quotactl","setdomainname","sethostname","setns"],"action":"SCMP_ACT_ERRNO","errnoRet":1},{"names":["chroot"],"action":"SCMP_ACT_ALLOW"},{"names":["delete_module","init_module","finit_module","query_module"],"action":"SCMP_ACT_ERRNO","errnoRet":1},{"names":["acct"],"action":"SCMP_ACT_ERRNO","errnoRet":1},{"names":["kcmp","process_madvise","process_vm_readv","process_vm_writev","ptrace"],"action":"SCMP_ACT_ERRNO","errnoRet":1},{"names":["iopl","ioperm"],"action":"SCMP_ACT_ERRNO","errnoRet":1},{"names":["settimeofday","stime","clock_settime","clock_settime64"],"action":"SCMP_ACT_ERRNO","errnoRet":1},{"names":["vhangup"],"action":"SCMP_ACT_ERRNO","errnoRet":1},{"names":["socket"],"action":"SCMP_ACT_ERRNO","errnoRet":22,"args":[{"index":0,"value":16,"op":"SCMP_CMP_EQ"},{"index":2,"value":9,"op":"SCMP_CMP_EQ"}]},{"names":["socket"],"action":"SCMP_ACT_ALLOW","args":[{"index":2,"value":9,"op":"SCMP_CMP_NE"}]},{"names":["socket"],"action":"SCMP_ACT_ALLOW","args":[{"index":0,"value":16,"op":"SCMP_CMP_NE"}]},{"names":["socket"],"action":"SCMP_ACT_ALLOW","args":[{"index":2,"value":9,"op":"SCMP_CMP_NE"}]}]},"maskedPaths":["/proc/acpi","/proc/kcore","/proc/keys","/proc/latency_stats","/proc/timer_list","/proc/timer_stats","/proc/sched_debug","/proc/scsi","/sys/firmware","/sys/fs/selinux","/sys/dev","/sys/fs/cgroup"],"readonlyPaths":["/proc/asound","/proc/bus","/proc/fs","/proc/irq","/proc/sys","/proc/sysrq-trigger"]}} 
DEBU[0000] Running ["/usr/sbin/runc" "--systemd-cgroup" "create" "--bundle" "/var/tmp/buildah1094495047" "--pid-file" "/var/tmp/buildah1094495047/pid" "--no-new-keyring" "buildah-buildah1094495047"] 
DEBU[0000] "/var/tmp/buildah1094495047/mnt/rootfs" is apparently not really mounted, skipping 
DEBU[0000] "/var/tmp/buildah1094495047/mnt" is apparently not really mounted, skipping 
error running container: error from /usr/sbin/runc creating container for [/bin/sh -c date]: time="2022-11-17T17:40:47Z" level=warning msg="unable to get oom kill count" error="openat2 /sys/fs/cgroup/system.slice/runc-buildah-buildah1094495047.scope/memory.events: no such file or directory"
time="2022-11-17T17:40:47Z" level=error msg="runc create failed: unable to start container process: unable to apply cgroup configuration: unable to start unit \"runc-buildah-buildah1094495047.scope\" (properties [{Name:Description Value:\"libcontainer container buildah-buildah1094495047\"} {Name:Slice Value:\"system.slice\"} {Name:Delegate Value:true} {Name:PIDs Value:@au [7555]} {Name:MemoryAccounting Value:true} {Name:CPUAccounting Value:true} {Name:IOAccounting Value:true} {Name:TasksAccounting Value:true} {Name:DefaultDependencies Value:false}]): Interactive authentication required."
: exit status 1
DEBU[0004] error building at step {Env:[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin] Command:run Args:[date] Flags:[] Attrs:map[] Message:RUN date Original:RUN date}: error while running runtime: exit status 1 
Error: error building at STEP "RUN date": error while running runtime: exit status 1
CircleCI received exit code 0

Does it indicate something useful?

giuseppe · 2022-11-18T10:09:19Z

runc is failing for the same reason.

sudo doesn't create a user session, could you try with machinectl shell $USER@?

giuseppe · 2022-11-18T10:13:49Z

alternatively, you could just use --cgroup-manager cgroupfs and you'll avoid using systemd to create the cgroup

adelton · 2022-11-19T08:13:14Z

While sudo might not create a user session, that sudo runs

sudo -niHu circleci -- /tmp/circleci-machine-agent3174475418/circleci-agent _internal agent-runner --config -

which I assume might do something about the session.

After all, loginctl session-status shows

c1 - circleci (1001)
	   Since: Sat 2022-11-19 06:25:37 UTC; 46s ago
	  Leader: 906 (sudo)
	  Remote: user root
	 Service: sudo-i; type unspecified; class background
	   State: active
	    Unit: session-c1.scope
		  ├─ 906 sudo -niHu circleci -- /tmp/circleci-machine-agent3174475418/circleci-agent _internal agent-runner --config -
		  ├─ 915 /tmp/circleci-machine-agent3174475418/circleci-agent _internal agent-runner --config -
		  ├─3745 podman
		  ├─4786 /bin/bash -eo pipefail -c "loginctl session-status | cat"
		  ├─5109 loginctl session-status
		  └─5110 cat

showing a session for uid 1001, not for root.

adelton · 2022-11-19T08:15:58Z

Using --cgroup-manager cgroupfs worked, thanks for that pointer: https://app.circleci.com/pipelines/github/adelton/freeipa-container/113/workflows/d0c21c49-b7a1-4020-910a-d48a7bbcffc9/jobs/519

github-actions · 2022-12-20T00:06:21Z

A friendly reminder that this issue had no activity for 30 days.

rhatdan · 2023-07-29T13:48:04Z

I don't believe this is still and issue and you have a workaround closing.

carlosrodfern · 2023-08-08T17:39:22Z

@rhatdan ,
I'm running into this same issue in Fedora 38. I'm logged in as a normal user, and when I try to build something simple, it shows the same problem. This was not a problem before. XDG_RUNTIME_DIR and DBUS_SESSION_BUS_ADDRESS are both set.

$ cat Containerfile 
FROM quay.io/centos/centos:stream9

RUN dnf install -y bind-utils

podman build -t test:latest .
STEP 1/2: FROM quay.io/centos/centos:stream9
STEP 2/2: RUN dnf install -y bind-utils
error running container: from /usr/bin/crun creating container for [/bin/sh -c dnf install -y bind-utils]: sd-bus call: Interactive authentication required.: Permission denied
: exit status 1
ERRO[0000] did not get container create message from subprocess: EOF 
Error: building at STEP "RUN dnf install -y bind-utils": while running runtime: exit status 1

When I use the workaround, it works: podman --cgroup-manager cgroupfs build --log-level=warning -t test:latest ., however, having to add that flag every time for a desktop user that it is logged in, as their first experience, rather shows there is a bug somewhere when using systemd as the cgroup manager.

Is this a podman issue or a systemd issue or some misconfiguration in f38? Is there anything I should look into?

Thank you.

rhatdan · 2023-08-08T18:51:04Z

Please open a new issue, unless you are running under CircleCI?

carlosrodfern · 2023-08-08T19:35:05Z

Will do. Thank you @rhatdan

openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label Nov 16, 2022

github-actions bot added the stale-issue label Dec 20, 2022

maleadt mentioned this issue Feb 7, 2023

crun as systemd service errors containers/crun#1134

Closed

heyakyra mentioned this issue Feb 13, 2023

How to pass "--cgroup-manager=cgroupfs" in "podman-compose build"? containers/podman-compose#209

Closed

rhatdan removed the stale-issue label Jul 29, 2023

rhatdan closed this as completed Jul 29, 2023

github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Nov 7, 2023

github-actions bot locked as resolved and limited conversation to collaborators Nov 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rootless podman on CircleCI fails to even build with error running container: from /usr/bin/crun: sd-bus call: Permission denied #16529

Rootless podman on CircleCI fails to even build with error running container: from /usr/bin/crun: sd-bus call: Permission denied #16529

adelton commented Nov 16, 2022

Luap99 commented Nov 16, 2022

adelton commented Nov 16, 2022

Luap99 commented Nov 17, 2022

adelton commented Nov 17, 2022

Luap99 commented Nov 17, 2022

adelton commented Nov 17, 2022

giuseppe commented Nov 18, 2022

giuseppe commented Nov 18, 2022

adelton commented Nov 19, 2022

adelton commented Nov 19, 2022

github-actions bot commented Dec 20, 2022

rhatdan commented Jul 29, 2023

carlosrodfern commented Aug 8, 2023

rhatdan commented Aug 8, 2023

carlosrodfern commented Aug 8, 2023

Rootless podman on CircleCI fails to even build with error running container: from /usr/bin/crun: sd-bus call: Permission denied #16529

Rootless podman on CircleCI fails to even build with error running container: from /usr/bin/crun: sd-bus call: Permission denied #16529

Comments

adelton commented Nov 16, 2022

Luap99 commented Nov 16, 2022

adelton commented Nov 16, 2022

Luap99 commented Nov 17, 2022

adelton commented Nov 17, 2022

Luap99 commented Nov 17, 2022

adelton commented Nov 17, 2022

giuseppe commented Nov 18, 2022

giuseppe commented Nov 18, 2022

adelton commented Nov 19, 2022

adelton commented Nov 19, 2022

github-actions bot commented Dec 20, 2022

rhatdan commented Jul 29, 2023

carlosrodfern commented Aug 8, 2023

rhatdan commented Aug 8, 2023

carlosrodfern commented Aug 8, 2023