`podman info` fails without sudo #3207

adel-mamin · 2019-05-28T12:42:50Z

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

adel@adel-pc:~  
> podman info
cannot clone: Invalid argument
panic: calling Unlock on unlocked lock

goroutine 1 [running]:
github.com/containers/libpod/vendor/github.com/containers/storage.(*lockfile).Unlock(0xc0000cdcc0)
        /build/podman/src/src/github.com/containers/libpod/vendor/github.com/containers/storage/lockfile_unix.go:144 +0x1ed
github.com/containers/libpod/libpod.makeRuntime.func2(0xc0006f4abe, 0x55b5908f2c00, 0xc0000cdcc0)
        /build/podman/src/src/github.com/containers/libpod/libpod/runtime.go:884 +0x47
github.com/containers/libpod/libpod.makeRuntime(0x55b5908e1ce0, 0xc0000c2048, 0xc0005b60f0, 0x55b5908b05a0, 0xc000814100)
        /build/podman/src/src/github.com/containers/libpod/libpod/runtime.go:897 +0x1a2f
github.com/containers/libpod/libpod.newRuntimeFromConfig(0x55b5908e1ce0, 0xc0000c2048, 0x0, 0x0, 0xc0000c47e0, 0x1, 0x1, 0xc0005b60f0, 0x0, 0x0)
        /build/podman/src/src/github.com/containers/libpod/libpod/runtime.go:564 +0x6a1
github.com/containers/libpod/libpod.NewRuntime(...)
        /build/podman/src/src/github.com/containers/libpod/libpod/runtime.go:355
github.com/containers/libpod/cmd/podman/libpodruntime.getRuntime(0x55b5908e1ce0, 0xc0000c2048, 0xc0006f5bb0, 0x0, 0x3e8, 0x0, 0x1e)
        /build/podman/src/src/github.com/containers/libpod/cmd/podman/libpodruntime/runtime.go:149 +0xbf9
github.com/containers/libpod/cmd/podman/libpodruntime.GetRuntime(...)
        /build/podman/src/src/github.com/containers/libpod/cmd/podman/libpodruntime/runtime.go:26
main.setupRootless(0x55b59179fa00, 0x55b591823320, 0x0, 0x0, 0x0, 0x0)
        /build/podman/src/src/github.com/containers/libpod/cmd/podman/main_local.go:116 +0x144
main.before(0x55b59179fa00, 0x55b591823320, 0x0, 0x0, 0x1, 0xc0000c4610)
        /build/podman/src/src/github.com/containers/libpod/cmd/podman/main.go:106 +0x6e
main.glob..func68(0x55b59179fa00, 0x55b591823320, 0x0, 0x0, 0x0, 0x0)
        /build/podman/src/src/github.com/containers/libpod/cmd/podman/main.go:75 +0x4b
github.com/containers/libpod/vendor/github.com/spf13/cobra.(*Command).execute(0x55b59179fa00, 0xc0000ba030, 0x0, 0x0, 0x55b59179fa00, 0xc0000ba030)
        /build/podman/src/src/github.com/containers/libpod/vendor/github.com/spf13/cobra/command.go:741 +0x572
github.com/containers/libpod/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x55b5917a11c0, 0xc0000b0cc0, 0x7ffed30ea503, 0x6)
        /build/podman/src/src/github.com/containers/libpod/vendor/github.com/spf13/cobra/command.go:852 +0x2ee
github.com/containers/libpod/vendor/github.com/spf13/cobra.(*Command).Execute(...)
        /build/podman/src/src/github.com/containers/libpod/vendor/github.com/spf13/cobra/command.go:800
main.main()
        /build/podman/src/src/github.com/containers/libpod/cmd/podman/main.go:142 +0x8a

Output of podman version:

Version:            1.3.0
RemoteAPI Version:  1
Go Version:         go1.12.4
Built:              Fri May 10 11:46:12 2019
OS/Arch:            linux/amd64

Output of podman info --debug:

cannot clone: Invalid argument
panic: calling Unlock on unlocked lock

goroutine 1 [running]:
github.com/containers/libpod/vendor/github.com/containers/storage.(*lockfile).Unlock(0xc0000cfcc0)
        /build/podman/src/src/github.com/containers/libpod/vendor/github.com/containers/storage/lockfile_unix.go:144 +0x1ed
github.com/containers/libpod/libpod.makeRuntime.func2(0xc00070eabe, 0x555644731c00, 0xc0000cfcc0)
        /build/podman/src/src/github.com/containers/libpod/libpod/runtime.go:884 +0x47
github.com/containers/libpod/libpod.makeRuntime(0x555644720ce0, 0xc0000c4048, 0xc0005e80f0, 0x5556446ef5a0, 0xc000820080)
        /build/podman/src/src/github.com/containers/libpod/libpod/runtime.go:897 +0x1a2f
github.com/containers/libpod/libpod.newRuntimeFromConfig(0x555644720ce0, 0xc0000c4048, 0x0, 0x0, 0xc000010750, 0x1, 0x1, 0xc0005e80f0, 0x0, 0x0)
        /build/podman/src/src/github.com/containers/libpod/libpod/runtime.go:564 +0x6a1
github.com/containers/libpod/libpod.NewRuntime(...)
        /build/podman/src/src/github.com/containers/libpod/libpod/runtime.go:355
github.com/containers/libpod/cmd/podman/libpodruntime.getRuntime(0x555644720ce0, 0xc0000c4048, 0xc00070fbb0, 0x0, 0x3e8, 0x0, 0x1e)
        /build/podman/src/src/github.com/containers/libpod/cmd/podman/libpodruntime/runtime.go:149 +0xbf9
github.com/containers/libpod/cmd/podman/libpodruntime.GetRuntime(...)
        /build/podman/src/src/github.com/containers/libpod/cmd/podman/libpodruntime/runtime.go:26
main.setupRootless(0x5556455dea00, 0xc00032c430, 0x0, 0x1, 0x0, 0x0)
        /build/podman/src/src/github.com/containers/libpod/cmd/podman/main_local.go:116 +0x144
main.before(0x5556455dea00, 0xc00032c430, 0x0, 0x1, 0x1, 0xc000010578)
        /build/podman/src/src/github.com/containers/libpod/cmd/podman/main.go:106 +0x6e
main.glob..func68(0x5556455dea00, 0xc00032c430, 0x0, 0x1, 0x0, 0x0)
        /build/podman/src/src/github.com/containers/libpod/cmd/podman/main.go:75 +0x4b
github.com/containers/libpod/vendor/github.com/spf13/cobra.(*Command).execute(0x5556455dea00, 0xc0000b2170, 0x1, 0x1, 0x5556455dea00, 0xc0000b2170)
        /build/podman/src/src/github.com/containers/libpod/vendor/github.com/spf13/cobra/command.go:741 +0x572
github.com/containers/libpod/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x5556455e01c0, 0xc0000b2cf0, 0x7ffc40dd54fb, 0x6)
        /build/podman/src/src/github.com/containers/libpod/vendor/github.com/spf13/cobra/command.go:852 +0x2ee
github.com/containers/libpod/vendor/github.com/spf13/cobra.(*Command).Execute(...)
        /build/podman/src/src/github.com/containers/libpod/vendor/github.com/spf13/cobra/command.go:800
main.main()
        /build/podman/src/src/github.com/containers/libpod/cmd/podman/main.go:142 +0x8a

Additional environment details (AWS, VirtualBox, physical, etc.):

> uname -a
Linux adel-pc 4.9.178-1-MANJARO #1 SMP PREEMPT Tue May 21 19:31:19 UTC 2019 x86_64 GNU/Linux

With sudo the command sudo podman info --debug works and outputs:

adel@adel-pc:~            
> sudo podman info --debug
debug:                    
  compiler: gc                          
  git commit: ""
  go version: go1.12.4       
  podman version: 1.3.0        
host:                      
  BuildahVersion: 1.8.2    
  Conmon:    
    package: Unknown
    path: /usr/bin/conmon             
    version: 'conmon version , commit: 8fba206232c249a8fc4e2fac1469fb2fddbf5cf7'
  Distribution:
    distribution: manjaro
    version: unknown
  MemFree: 3711102976
  MemTotal: 7782268928
  OCIRuntime:
    package: Unknown
    path: /usr/bin/runc
    version: |-
      runc version 1.0.0-rc8
      commit: 425e105d5a03fabd737a126ad93d62a9eeede87f
      spec: 1.0.1-dev
  SwapFree: 9448923136
  SwapTotal: 9448923136
  arch: amd64
  cpus: 4
  hostname: adel-pc
  kernel: 4.9.178-1-MANJARO
  os: linux
  rootless: false
  uptime: 42m 34.67s
registries:
  blocked: null
  insecure: null
  search:
  - docker.io
  - registry.fedoraproject.org
  - quay.io
  - registry.access.redhat.com
  - registry.centos.org
store:
  ConfigFile: /etc/containers/storage.conf
  ContainerStore:
    number: 0
  GraphDriverName: overlay
  GraphOptions:
  - overlay.mountopt=nodev
  GraphRoot: /var/lib/containers/storage
  GraphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  ImageStore:
    number: 0
  RunRoot: /var/run/containers/storage
  VolumePath: /var/lib/containers/storage/volumes

The text was updated successfully, but these errors were encountered:

vrothberg · 2019-05-28T12:50:16Z

Hi @adel-mamin, thanks a lot for opening the issue. Which kind of filesystem is your home directory on?

adel-mamin · 2019-05-28T12:52:18Z

I believe it is ext4

vrothberg · 2019-05-28T12:57:49Z

@adel-mamin, is it on NFS? I suspect it's on NFS since the filelocks are not working there, which is inherently important for the storage. If that's the case, we can edit ~/.config/containers/storage.conf and make the graph_root point to a non-NFS directory.

adel-mamin · 2019-05-28T13:09:38Z

It is not on NFS. It is on my local disk:

> stat -f -L -c %T /home/adel
ext2/ext3

> stat -f -L -c %T /home/adel/.local/
ext2/ext3

vrothberg · 2019-05-28T13:36:43Z

Thanks for clarifying. I checked the code and believe there's a bug.

Unlock the alive lock only once in the deferred func call. Fixes: containers#3207 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>

vrothberg · 2019-05-28T13:41:22Z

@adel-mamin, if possible, could you compile #3208 and test if that fixes your issue?

adel-mamin · 2019-05-28T13:45:19Z

@vrothberg I can try, if you point me to the build instructions.

Unlock the alive lock only once in the deferred func call. Fixes: containers#3207 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>

vrothberg · 2019-05-28T13:53:50Z

@adel-mamin, you can checkout the pull request in your local git-tree via:

git fetch -f -u $remote pull/3208/head:pull-request-3208

Note that $remote is the git-remote pointing to github.com/containers/libpod which is origin if you just clone this repository. Once that's done, you can follow https://github.com/containers/libpod/blob/master/install.md#building-from-scratch for building podman.

Unlock the alive lock only once in the deferred func call. Fixes: containers#3207 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>

adel-mamin · 2019-05-28T17:51:27Z

FWIW, I have tried building and installing podman from master SHA1 bc7afd6d.
Here is what I get:

adel@adel-pc:~/src/libpod  
> podman --version
podman version 1.3.2-dev

adel@adel-pc:~/src/libpod  
> podman info
cannot clone: Invalid argument
Error: could not get runtime: cannot re-exec process

adel@adel-pc:~/src/libpod  
> sudo podman info
host:
  BuildahVersion: 1.9.0-dev
  Conmon:
    package: Unknown
    path: /usr/bin/conmon
    version: 'conmon version , commit: 8fba206232c249a8fc4e2fac1469fb2fddbf5cf7'
  Distribution:
    distribution: manjaro
    version: unknown
  MemFree: 4104130560
  MemTotal: 7782268928
  OCIRuntime:
    package: Unknown
    path: /usr/bin/runc
    version: |-
      runc version 1.0.0-rc8
      commit: 425e105d5a03fabd737a126ad93d62a9eeede87f
      spec: 1.0.1-dev
  SwapFree: 9448923136
  SwapTotal: 9448923136
  arch: amd64
  cpus: 4
  hostname: adel-pc
  kernel: 4.9.178-1-MANJARO
  os: linux
  rootless: false
  uptime: 23m 30.31s
registries:
  blocked: null
  insecure: null
  search:
  - docker.io
  - registry.fedoraproject.org
  - quay.io
  - registry.access.redhat.com
  - registry.centos.org
store:
  ConfigFile: /etc/containers/storage.conf
  ContainerStore:
    number: 0
  GraphDriverName: overlay
  GraphOptions:
  - overlay.mountopt=nodev
  GraphRoot: /var/lib/containers/storage
  GraphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  ImageStore:
    number: 0
  RunRoot: /var/run/containers/storage
  VolumePath: /var/lib/containers/storage/volumes

Here is the build log:

adel@adel-pc:~/src/libpod  
> make -j4
mkdir -p "/home/adel/src/libpod/_output/src/github.com/containers"
go generate ./cmd/podman/varlink/...
ln -sf "/home/adel/src/libpod" "/home/adel/src/libpod/_output/src/github.com/containers"
ln -sf "/home/adel/src/libpod/vendor/github.com/varlink" "/home/adel/src/libpod/_output/src/github.com/varlink"
touch .gopathok
go build -ldflags ' -X github.com/containers/libpod/libpod.gitCommit=bc7afd6d71da4173e4894ff352667a25987fa2ea -X github.com/containers/libpod/libpod.buildInfo=1559065847' -tags "   ostree selinux systemd exclude_graphdriver_devicemapper seccomp varlink" -o bin/podman github.com/containers/libpod/cmd/podman
go build -ldflags ' -X github.com/containers/libpod/libpod.gitCommit=bc7afd6d71da4173e4894ff352667a25987fa2ea -X github.com/containers/libpod/libpod.buildInfo=1559065847' -tags "   ostree selinux systemd exclude_graphdriver_devicemapper seccomp varlink remoteclient" -o bin/podman-remote github.com/containers/libpod/cmd/podman

vrothberg · 2019-05-28T18:06:53Z

FWIW, I have tried building and installing podman from master SHA1 bc7afd6d.
Here is what I get:

Thanks a ton for checking. I'll reopen. Note that I can't reproduce this at all.

adel@adel-pc:~/src/libpod  
> podman --version
podman version 1.3.2-dev

adel@adel-pc:~/src/libpod  
> podman info
cannot clone: Invalid argument
Error: could not get runtime: cannot re-exec process

@giuseppe, do you have a suspicion what could have gone south?

giuseppe · 2019-05-28T20:54:46Z

Error: could not get runtime: cannot re-exec process:

I fear there is no support for user namespaces in the kernel.

What do you see with ls /proc/self/ns; cat /proc/self/uid_map ?

adel-mamin · 2019-05-29T06:41:13Z

> ls /proc/self/ns; cat /proc/self/uid_map
cgroup  ipc  mnt  net  pid  uts
cat: /proc/self/uid_map: No such file or directory

vrothberg · 2019-05-29T06:50:41Z

@giuseppe's suspicion is correct, the system does not have user namespace support. I guess that every distro configures that a bit different, but here's a link how to enable it in Manjaro: https://forum.manjaro.org/t/how-to-permanently-enable-user-namespaces-in-manjaro-linux/71614/2

adel-mamin · 2019-05-29T07:51:46Z

I updated the Linux kernel to 4.19.45-1-MANJARO and enabled the user namespaces by
sysctl kernel.unprivileged_userns_clone=1
Now podman info works as expected.
I am closing the issue.
Thank you!

vrothberg · 2019-05-29T08:21:03Z

Awesome, thanks a lot!

rhatdan · 2019-05-29T12:13:42Z

@adel-mamin Would you mind opening a PR for adding information on MANJARO on install.md. So future users would not hit the issues you have had.

adel-mamin · 2019-05-31T06:32:48Z

@rhatdan I've added #3242. Please take a look.

Unlock the alive lock only once in the deferred func call. Fixes: containers#3207 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>

openshift-ci-robot added the kind/bug Categorizes issue or PR as related to a bug. label May 28, 2019

vrothberg added a commit to vrothberg/libpod that referenced this issue May 28, 2019

runtime: unlock the alive lock only once

238fe6f

Unlock the alive lock only once in the deferred func call. Fixes: containers#3207 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>

vrothberg mentioned this issue May 28, 2019

runtime: unlock the alive lock only once #3208

Merged

vrothberg added a commit to vrothberg/libpod that referenced this issue May 28, 2019

runtime: unlock the alive lock only once

cd7806d

Unlock the alive lock only once in the deferred func call. Fixes: containers#3207 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>

vrothberg added a commit to vrothberg/libpod that referenced this issue May 28, 2019

runtime: unlock the alive lock only once

6ddf47c

Unlock the alive lock only once in the deferred func call. Fixes: containers#3207 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>

openshift-merge-robot closed this as completed in #3208 May 28, 2019

vrothberg reopened this May 28, 2019

adel-mamin closed this as completed May 29, 2019

adel-mamin mentioned this issue May 31, 2019

Updated install.md for Manjaro Linux #3242

Merged

samc24 pushed a commit to samc24/libpod that referenced this issue Jul 16, 2019

runtime: unlock the alive lock only once

f7b8207

Unlock the alive lock only once in the deferred func call. Fixes: containers#3207 Signed-off-by: Valentin Rothberg <rothberg@redhat.com>

github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 24, 2023

github-actions bot locked as resolved and limited conversation to collaborators Sep 24, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`podman info` fails without sudo #3207

`podman info` fails without sudo #3207

adel-mamin commented May 28, 2019

vrothberg commented May 28, 2019

adel-mamin commented May 28, 2019

vrothberg commented May 28, 2019

adel-mamin commented May 28, 2019 •

edited

vrothberg commented May 28, 2019

vrothberg commented May 28, 2019

adel-mamin commented May 28, 2019

vrothberg commented May 28, 2019

adel-mamin commented May 28, 2019

vrothberg commented May 28, 2019 •

edited

giuseppe commented May 28, 2019

adel-mamin commented May 29, 2019

vrothberg commented May 29, 2019

adel-mamin commented May 29, 2019

vrothberg commented May 29, 2019

rhatdan commented May 29, 2019

adel-mamin commented May 31, 2019

podman info fails without sudo #3207

podman info fails without sudo #3207

Comments

adel-mamin commented May 28, 2019

vrothberg commented May 28, 2019

adel-mamin commented May 28, 2019

vrothberg commented May 28, 2019

adel-mamin commented May 28, 2019 • edited

vrothberg commented May 28, 2019

vrothberg commented May 28, 2019

adel-mamin commented May 28, 2019

vrothberg commented May 28, 2019

adel-mamin commented May 28, 2019

vrothberg commented May 28, 2019 • edited

giuseppe commented May 28, 2019

adel-mamin commented May 29, 2019

vrothberg commented May 29, 2019

adel-mamin commented May 29, 2019

vrothberg commented May 29, 2019

rhatdan commented May 29, 2019

adel-mamin commented May 31, 2019

`podman info` fails without sudo #3207

`podman info` fails without sudo #3207

adel-mamin commented May 28, 2019 •

edited

vrothberg commented May 28, 2019 •

edited