failed to execvp -podmansh: No such file or directory

[nlgranger]

Issue Description

Opening the session of a user with podmansh shell fails with the following error: failed to execvp -podmansh: No such file or directory

The - before the executable name is suspicious but I failed to track it down in the code.

Steps to reproduce the issue

Steps to reproduce the issue

I have followed the man page (https://man.archlinux.org/man/extra/podman/podmansh.1.en#Setup)

1. Create user with podmansh as shell

grep foobar /etc/passwd
foobar:x:1001:1001::/home/foobar:/usr/bin/podmansh

3. Create quadlet file:

[Unit]
Description=The Podmansh container
After=local-fs.target
[Container]
Image=docker.io/alpine
ContainerName=podmansh
RemapUsers=keep-id
RunInit=yes
DropCapability=all
NoNewPrivileges=true
Exec=sleep infinity
[Install]
RequiredBy=default.target

4. login:

su --login foobar
failed to execvp -podmansh: No such file or directory
WARN[0000] Failed to add pause process to systemd sandbox cgroup: dbus: couldn't determine address of session bus

Describe the results you received

The session exits immediately with the error message:

failed to execvp -podmansh: No such file or directory

In the system log, it reads:

mai 11 19:00:56 portablenico su[6377]: pam_unix(su-l:session): session opened for user foobar(uid=1001) by granger(uid=0)
mai 11 19:00:56 portablenico su[6377]: pam_unix(su-l:session): session closed for user foobar

Describe the results you expected

The session opens.

podman info output

host:
  arch: amd64
  buildahVersion: 1.35.3
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - rdma
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: /usr/bin/conmon appartient à conmon 1:2.1.11-1
    path: /usr/bin/conmon
    version: 'conmon version 2.1.10, commit: e21e7c85b7637e622f21c57675bf1154fc8b1866'
  cpuUtilization:
    idlePercent: 85.39
    systemPercent: 3.77
    userPercent: 10.85
  cpus: 4
  databaseBackend: boltdb
  distribution:
    distribution: arch
    version: unknown
  eventLogger: journald
  freeLocks: 2048
  hostname: portablenico
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 6.8.9-arch1-2
  linkmode: dynamic
  logDriver: journald
  memFree: 11273428992
  memTotal: 16151777280
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: Unknown
    package: /usr/lib/podman/netavark appartient à netavark 1.10.3-1
    path: /usr/lib/podman/netavark
    version: netavark 1.10.3
  ociRuntime:
    name: crun
    package: /usr/bin/crun appartient à crun 1.15-1
    path: /usr/bin/crun
    version: |-
      crun version 1.15
      commit: e6eacaf4034e84185fd8780ac9262bbf57082278
      rundir: /run/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: /usr/bin/pasta appartient à passt 2024_04_26.d03c4e2-1
    version: |
      pasta 2024_04_26.d03c4e2
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: false
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /etc/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 0
  swapTotal: 0
  uptime: 0h 29m 3.00s
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries: {}
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 43920654336
  graphRootUsed: 34083332096
  graphStatus:
    Backing Filesystem: f2fs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Supports shifting: "true"
    Supports volatile: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 1
  runRoot: /var/run/containers/storage
  transientStore: false
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 5.0.2
  Built: 1713438799
  BuiltTime: Thu Apr 18 13:13:19 2024
  GitCommit: 3304dd95b8978a8346b96b7d43134990609b3b29-dirty
  GoVersion: go1.22.2
  Os: linux
  OsArch: linux/amd64
  Version: 5.0.2

Podman in a container

No

Privileged Or Rootless

None

Upstream Latest Release

No

Additional environment details

OS: archlinux

Additional information

Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting

[rhatdan]

@lsm5 PTAL

[rhatdan]

@bachradsusi PTAL

[edsantiago]

A leading dash is traditionally an indicator that the process is a login shell, and this seems consistent with the su --login reproducer, but I've never actually seen leading-dash in anything other than a shell. Is podmansh in your /etc/shells? (I'm just curious).

Regardless of the cause, the proper solution is for podman to strip this leading dash.

[nlgranger]

I have tried again with podmansh in /etc/shells, the output remains identical.

[rhatdan]

Could you try to login via ssh or /bin/login?

One thing you also try is with machinectl login

[nlgranger]

It actually did not occur to me to test theses.

• ssh: OK
• machinectl shell --uid foobar: OK
• linux console: OK

[bachradsusi]

The difference happens in https://github.com/containers/podman/blob/main/pkg/domain/infra/abi/system_linux.go#L39

• ssh podmanuser@localhost -> unshare.HasCapSysAdmin() == true
• su --login podmanuser -> unshare.HasCapSysAdmin() == false

[Luap99]

Well first never use su/sudo to "login" into other rootless users. They do not create a proper systemd session and thus the resulting environment is less than ideal for podman, i.e.
WARN[0000] Failed to add pause process to systemd sandbox cgroup: dbus: couldn't determine address of session bus
In your case here this would not even start the quadlet so there is not container for podmansh which would make it fail even if it can do the exec.

The issue here is that the first podman command you run after a boot has do do a re-exec of itself to create and join the userns. And since the quadlet unit is not started podmansh (just a wrapper for podman exec) is now the first podman command and that re-exec is failing likely because it cannot deal with the -podmansh argv0 from the login sh

[Luap99]

This can be reproduced by using

$ podman system migrate # kills the existing pause process so the next command must do the re-exec
$ bash -c "exec -a -podmansh podmansh"

The problem is the the re-exec code uses argv0 as command for the exec call which will not exist, I work on a fix.

[Luap99]

#22695 should fix this error condition but using su will still not create a proper env and not spawn the systemd session so using ssh/machinectl or a normal terminal/gui login is the way to go.

also see https://github.com/containers/podman/blob/main/troubleshooting.md#31-podman-run-fails-with-erro0000-xdg_runtime_dir-directory-runuser0-is-not-owned-by-the-current-user-or-error-creating-tmpdir-mkdir-runuser1000-permission-denied

[nlgranger]

Thank you for the quick response and fix!