Rootless podman error: could not get runtime: open /proc/31678/ns/user: no such file or directory

[faulesocke]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

Running any podman command (for example podman info or podman version) as non-root only gives me the following output:

% podman info
Error: could not get runtime: open /proc/31678/ns/user: no such file or directory

It works when running as root.

Steps to reproduce the issue:

1. Run podman info

Describe the results you received:

Error: could not get runtime: open /proc/31678/ns/user: no such file or directory

Describe the results you expected:
The actual command output...

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

% sudo podman version
Version:            1.8.0
RemoteAPI Version:  1
Go Version:         go1.13.7
OS/Arch:            linux/amd64

Output of podman info --debug:

UNOBTAINABLE

Package info (e.g. output of rpm -q podman or apt list podman):

% xbps-query podman
architecture: x86_64
filename-sha256: 514f725a15bc57ec717606103eaedd77f603d62c3b29007166ef9d235b308ac2
filename-size: 18MB
homepage: https://podman.io/
install-date: 2020-02-24 09:11 CET
installed_size: 54MB
license: Apache-2.0
maintainer: Cameron Nemo <camerontnorman@gmail.com>
metafile-sha256: 347e1471b2966a2350df5661216db7a1f60103544ebbe406d8e4a48dde169782
pkgver: podman-1.8.0_1
repository: https://alpha.de.repo.voidlinux.org/current
shlib-requires:
	libpthread.so.0
	libgpgme.so.11
	libassuan.so.0
	libgpg-error.so.0
	libseccomp.so.2
	librt.so.1
	libdevmapper.so.1.02
	libc.so.6
short_desc: Simple management tool for containers and images
source-revisions: podman:4351b38206
state: installed

Additional environment details (AWS, VirtualBox, physical, etc.):
Physical box running Void Linux.

[faulesocke]

Oh I just noticed one interesting and probably important thing: The PID in the path (/proc/...) is always the same value: 31678. Of course that PID can not exist. That is probably why the thing is failing.

[rhatdan]

Could you check if you have a podman running as the user? If yes, could you kill it and then see if the problem goes away?

[faulesocke]

No podman running, also tried restarting the box.

[rhatdan]

What OS are you running podman on? Could this be your kernel does not support User Namespace?
ls -l /proc/self/ns

[faulesocke]

@rhatdan I'm running Void Linux. The Kernel supports user namespaces. The issue is probably not the namespace but podman figuring out some bogus PID when building the path.

% uname -srvmpio
Linux 5.4.21_1 #1 SMP PREEMPT Thu Feb 20 09:09:55 UTC 2020 x86_64 unknown unknown GNU/Linux

% cat /etc/os-release 
NAME="void"
ID="void"
DISTRIB_ID="void"
PRETTY_NAME="void"

% ls -l /proc/self/ns 
total 0
lrwxrwxrwx 1 socke socke 0 Feb 24 13:42 cgroup -> 'cgroup:[4026531835]'
lrwxrwxrwx 1 socke socke 0 Feb 24 13:42 ipc -> 'ipc:[4026531839]'
lrwxrwxrwx 1 socke socke 0 Feb 24 13:42 mnt -> 'mnt:[4026531840]'
lrwxrwxrwx 1 socke socke 0 Feb 24 13:42 net -> 'net:[4026531992]'
lrwxrwxrwx 1 socke socke 0 Feb 24 13:42 pid -> 'pid:[4026531836]'
lrwxrwxrwx 1 socke socke 0 Feb 24 13:42 pid_for_children -> 'pid:[4026531836]'
lrwxrwxrwx 1 socke socke 0 Feb 24 13:42 user -> 'user:[4026531837]'
lrwxrwxrwx 1 socke socke 0 Feb 24 13:42 uts -> 'uts:[4026531838]'

Could this issue be ulimits related?

[giuseppe]

could be fixed by: #5237

[faulesocke]

@giuseppe Nope. Installed 1.8.1-rc1 and the error is still there.

[faulesocke]

I have downgraded a little bit. 1.7.0 is also affected but 1.6.5 works. So this got broken somewhere between 1.6.5 and 1.7.0.

[faulesocke]

Okay, this is odd. After I've downgraded it yesterday, i've now upgraded it back to 1.8.0. Then it worked. Then I rebooted to make sure no processes from the old version are running. And it still works. Don't know what went wrong there.

[giuseppe]

thanks for the update. Let's close the issue for now.

If it still happens, feel free to re-open it with more details

[tunip]

Have this issue too after each reboot of the host system. Only a podman system migrate resolves this temporary (until next boot). Can't downgrade/upgrade the package, as this is a new void linux installation and only podman 1.8.0 packages is available.

[rhatdan]

Is the runtime directory /run not on a tmpfs? We rely on the /run directory being cleaned on a reboot.