Error: cannot setup namespace using newuidmap: exit status 1 #12637

Lunarequest · 2021-12-17T02:28:48Z

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description
attempting to create any container rootlessly results in Error: cannot setup namespace using newuidmap: exit status 1

Steps to reproduce the issue:

1.run podman run docker.io/hello-world

Describe the results you received:

Error: cannot setup namespace using newuidmap: exit status 1

Describe the results you expected:

runs the docker-hello world container

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

`Error: cannot setup namespace using newuidmap: exit status 1`

running podman --version outputs

podman version 3.4.4

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.23.1
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - rdma
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: Unknown
    path: /nix/store/6r9syknl9xza7jzwd25v0ym41rz84r1m-conmon-2.0.31/bin/conmon
    version: 'conmon version 2.0.31, commit: '
  cpus: 4
  distribution:
    codename: quokka
    distribution: nixos
    version: "22.05"
  eventLogger: journald
  hostname: nixpro
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.10.84
  linkmode: dynamic
  logDriver: journald
  memFree: 4878995456
  memTotal: 8215441408
  ociRuntime:
    name: crun
    package: Unknown
    path: /nix/store/9m18pl5di2p0cin7b4wa1f55jk5fy5ka-crun-1.3/bin/crun
    version: |-
      crun version 1.3
      commit: 1.3
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_AUDIT_WRITE,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_MKNOD,CAP_NET_BIND_SERVICE,CAP_NET_RAW,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: ""
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /nix/store/69rx44glqh00035mji5iw1dif6nxmlkv-slirp4netns-1.1.12/bin/slirp4netns
    package: Unknown
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.2
  swapFree: 4107718656
  swapTotal: 4107718656
  uptime: 13m 38.89s
plugins:
  log:
  - k8s-file
  - none
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - docker.io
  - quay.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 0
    stopped: 1
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 6
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 3.4.4
  Built: 315532800
  BuiltTime: Tue Jan  1 05:30:00 1980
  GitCommit: ""
  GoVersion: go1.16.10
  OsArch: linux/amd64
  Version: 3.4.4

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

nixos unstable

The text was updated successfully, but these errors were encountered:

giuseppe · 2021-12-17T08:20:59Z

can you check if newuidmap has the setuid bit set or has file capabilities?

Please show the output for: getfattr -d -m- /usr/bin/newuidmap and stat /usr/bin/newuidmap.

Lunarequest · 2021-12-17T11:55:58Z

stat returns

  File: /run/wrappers/bin/newuidmap
  Size: 17760           Blocks: 40         IO Block: 4096   regular file
Device: 0,24    Inode: 21          Links: 1
Access: (4511/-r-s--x--x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-12-17 13:52:04.965171853 +0530
Modify: 2021-12-17 13:52:04.965171853 +0530
Change: 2021-12-17 13:52:04.969171853 +0530
 Birth: -

getfattr -d -m- /run/wrappers/bin/newuidmap returns nothing

giuseppe · 2021-12-17T12:32:42Z

thanks.

Can you also show the content of the /etc/subuid and /etc/subgid files?

How is /run mounted? The output grep /run /proc/self/mountinfo could help to detect whether it is mounted with nosuid that would prevent the setuid bit to have any effect

Lunarequest · 2021-12-17T12:37:57Z

/etc/subuid/

nullrequest:100000:65536
nullrequest:100000:65536

/etc/subgid

nullrequest:100000:65536
nullrequest:100000:65536

/run seems to be mounted by nixos with nosuid
however /run/wrappers is mounted with rw,nodev,relatime shared:13 - tmpfs tmpfs rw,mode=755 unlike /run which is mounted /run rw,nosuid,nodev shared:11 - tmpfs tmpfs rw,size=2005724k,mode=755

giuseppe · 2021-12-17T13:17:15Z

ok, thanks.

Can you try the following commands?

$ unshare -U sleep 100 &
$ newuidmap $! 0 100000 65536
$ newgidmap $! 0 100000 65536

Do they work fine?

Lunarequest · 2021-12-17T13:18:31Z

yeah they exit with 0 no issues

giuseppe · 2021-12-17T13:21:17Z

does podman --log-level debug ... give more info on what is going wrong?

Lunarequest · 2021-12-17T13:22:33Z

INFO[0000] /nix/store/b2rb9jllz97s2gq1x4n9l9c9gxk4y9ra-podman-3.4.4/bin/podman filtering at log level debug 
DEBU[0000] Called run.PersistentPreRunE(/nix/store/b2rb9jllz97s2gq1x4n9l9c9gxk4y9ra-podman-3.4.4/bin/podman --log-level debug run hello-world) 
DEBU[0000] Found default OCI runtime /nix/store/9m18pl5di2p0cin7b4wa1f55jk5fy5ka-crun-1.3/bin/crun path via PATH environment variable 
DEBU[0000] Merged system config "/etc/containers/containers.conf" 
DEBU[0000] Using conmon from $PATH: "/nix/store/6r9syknl9xza7jzwd25v0ym41rz84r1m-conmon-2.0.31/bin/conmon" 
DEBU[0000] Initializing boltdb state at /home/nullrequest/.local/share/containers/storage/libpod/bolt_state.db 
DEBU[0000] Using graph driver overlay                   
DEBU[0000] Using graph root /home/nullrequest/.local/share/containers/storage 
DEBU[0000] Using run root /run/user/1000/containers     
DEBU[0000] Using static dir /home/nullrequest/.local/share/containers/storage/libpod 
DEBU[0000] Using tmp dir /run/user/1000/libpod/tmp      
DEBU[0000] Using volume path /home/nullrequest/.local/share/containers/storage/volumes 
DEBU[0000] Set libpod namespace to ""                   
DEBU[0000] Not configuring container store              
DEBU[0000] Initializing event backend journald          
DEBU[0000] using runtime "runc" from $PATH: "/nix/store/7lbhmz69b1x5244h2sm1sddc80agnd0x-runc-1.0.3/bin/runc" 
DEBU[0000] configured OCI runtime kata initialization failed: no valid executable found for OCI runtime kata: invalid argument 
DEBU[0000] configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argument 
DEBU[0000] using runtime "crun" from $PATH: "/nix/store/9m18pl5di2p0cin7b4wa1f55jk5fy5ka-crun-1.3/bin/crun" 
DEBU[0000] Using OCI runtime "/nix/store/9m18pl5di2p0cin7b4wa1f55jk5fy5ka-crun-1.3/bin/crun" 
INFO[0000] Found CNI network podman (type=bridge) at /home/nullrequest/.config/cni/net.d/87-podman.conflist 
DEBU[0000] Default CNI network name podman is unchangeable 
DEBU[0000] error from newuidmap: newuidmap: write to uid_map failed: Invalid argument 
Error: cannot setup namespace using newuidmap: exit status 1

giuseppe · 2021-12-17T13:28:14Z

error from newuidmap: newuidmap: write to uid_map failed: Invalid argument means the wrong mapping was provided.

What is your user id (output from id)?

Lunarequest · 2021-12-17T13:29:01Z

uid=1000(nullrequest) gid=100(users) groups=100(users),1(wheel)

giuseppe · 2021-12-17T13:37:54Z

thanks, still I've no clue why it doesn't work.

Would it be possible for you to apply the patch here: #12641 and see if the error message gives us a better understanding?

Lunarequest · 2021-12-17T13:49:44Z

INFO[0000] /nix/store/b2rb9jllz97s2gq1x4n9l9c9gxk4y9ra-podman-3.4.4/bin/podman filtering at log level debug 
DEBU[0000] Called run.PersistentPreRunE(/nix/store/b2rb9jllz97s2gq1x4n9l9c9gxk4y9ra-podman-3.4.4/bin/podman --log-level debug run hello-world) 
DEBU[0000] Found default OCI runtime /nix/store/9m18pl5di2p0cin7b4wa1f55jk5fy5ka-crun-1.3/bin/crun path via PATH environment variable 
DEBU[0000] Using conmon from $PATH: "/nix/store/6r9syknl9xza7jzwd25v0ym41rz84r1m-conmon-2.0.31/bin/conmon" 
DEBU[0000] Initializing boltdb state at /home/nullrequest/.local/share/containers/storage/libpod/bolt_state.db 
DEBU[0000] Using graph driver overlay                   
DEBU[0000] Using graph root /home/nullrequest/.local/share/containers/storage 
DEBU[0000] Using run root /run/user/1000/containers     
DEBU[0000] Using static dir /home/nullrequest/.local/share/containers/storage/libpod 
DEBU[0000] Using tmp dir /run/user/1000/libpod/tmp      
DEBU[0000] Using volume path /home/nullrequest/.local/share/containers/storage/volumes 
DEBU[0000] Set libpod namespace to ""                   
DEBU[0000] Not configuring container store              
DEBU[0000] Initializing event backend journald          
DEBU[0000] using runtime "crun" from $PATH: "/nix/store/9m18pl5di2p0cin7b4wa1f55jk5fy5ka-crun-1.3/bin/crun" 
DEBU[0000] using runtime "runc" from $PATH: "/nix/store/7lbhmz69b1x5244h2sm1sddc80agnd0x-runc-1.0.3/bin/runc" 
DEBU[0000] configured OCI runtime kata initialization failed: no valid executable found for OCI runtime kata: invalid argument 
DEBU[0000] configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argument 
DEBU[0000] Using OCI runtime "/nix/store/9m18pl5di2p0cin7b4wa1f55jk5fy5ka-crun-1.3/bin/crun" 
DEBU[0000] Default CNI network name podman is unchangeable 
DEBU[0000] error from newuidmap: newuidmap: write to uid_map failed: Invalid argument 
Error: cannot setup namespace using newuidmap: exit status 1

giuseppe · 2021-12-17T13:59:16Z

DEBU[0000] error from newuidmap: newuidmap: write to uid_map failed: Invalid argument
Error: cannot setup namespace using newuidmap: exit status 1

it looks like the patch was not applied

Lunarequest · 2021-12-17T14:25:48Z

yeah my bad I'm trying to rebuild it from source its taking a while, I'll update when its done

Lunarequest · 2021-12-19T05:42:57Z

I'm running into issues building on nixos

Lunarequest · 2021-12-19T05:45:26Z

also another thing to note, running with root everything works. its only in rootless mode where there are issues

giuseppe · 2021-12-20T08:16:24Z

also another thing to note, running with root everything works. its only in rootless mode where there are issues

root doesn't need newuidmap and newgidmap because it is already running with capabilities

giuseppe · 2021-12-20T08:19:23Z

why do you have your user listed twice with the same range in the /etc/subuid and /etc/subgid files?

Can you drop one line from each file and make sure there are no overlapping mappings?

like:

/etc/subuid/
nullrequest:100000:65536

/etc/subgid
nullrequest:100000:65536

Lunarequest · 2021-12-20T08:59:33Z

yeah that was the issue, dropping the extra lines fixed the issue

r3h0 · 2022-02-25T21:11:32Z

why do you have your user listed twice with the same range in the /etc/subuid and /etc/subgid files?

@giuseppe, FWIW that's the default in the latest stable podman image, quay.io/podman/stable:v3.4.4.

[podman@podman /]$ cat /etc/subuid
podman:1:999
podman:1001:64535
[podman@podman /]$ cat /etc/subgid
podman:1:999
podman:1001:64535

markstos · 2022-11-30T20:14:46Z

I ran into this error on Podman 4.2 on Arch Linux. I resolved it after following the "Rootless Podman" setup steps on the Arch Wiki:

https://wiki.archlinux.org/title/Podman#Rootless_Podman

I'm not sure exactly what fixed it, but it seemed to start working after removing my /etc/subuid and /etc/subgid files and running the usermod command there. The contents appear the same, except the range was extended was from 10,000 to 100,000 and there's an extra blank line. Also, before the files were symlinked into my home dir and now they are not.

dzmitry-lahoda · 2023-03-12T22:56:39Z

I am on POPOS with nix home-manager. I installed shadow and podman via home-manager flake. Podman was not working, deleted. Used #12715 and found that shadown also is kind of not working. Deleted too.

openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label Dec 17, 2021

Lunarequest closed this as completed Dec 20, 2021

jplatte mentioned this issue Apr 17, 2023

Error validating CNI config file #13396

Closed

github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Aug 30, 2023

github-actions bot locked as resolved and limited conversation to collaborators Aug 30, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error: cannot setup namespace using newuidmap: exit status 1 #12637

Error: cannot setup namespace using newuidmap: exit status 1 #12637

Lunarequest commented Dec 17, 2021

giuseppe commented Dec 17, 2021

Lunarequest commented Dec 17, 2021 •

edited

giuseppe commented Dec 17, 2021

Lunarequest commented Dec 17, 2021 •

edited

giuseppe commented Dec 17, 2021

Lunarequest commented Dec 17, 2021

giuseppe commented Dec 17, 2021

Lunarequest commented Dec 17, 2021

giuseppe commented Dec 17, 2021

Lunarequest commented Dec 17, 2021

giuseppe commented Dec 17, 2021

Lunarequest commented Dec 17, 2021

giuseppe commented Dec 17, 2021

Lunarequest commented Dec 17, 2021

Lunarequest commented Dec 19, 2021

Lunarequest commented Dec 19, 2021

giuseppe commented Dec 20, 2021

giuseppe commented Dec 20, 2021

Lunarequest commented Dec 20, 2021

r3h0 commented Feb 25, 2022

markstos commented Nov 30, 2022

dzmitry-lahoda commented Mar 12, 2023

Error: cannot setup namespace using newuidmap: exit status 1 #12637

Error: cannot setup namespace using newuidmap: exit status 1 #12637

Comments

Lunarequest commented Dec 17, 2021

giuseppe commented Dec 17, 2021

Lunarequest commented Dec 17, 2021 • edited

giuseppe commented Dec 17, 2021

Lunarequest commented Dec 17, 2021 • edited

giuseppe commented Dec 17, 2021

Lunarequest commented Dec 17, 2021

giuseppe commented Dec 17, 2021

Lunarequest commented Dec 17, 2021

giuseppe commented Dec 17, 2021

Lunarequest commented Dec 17, 2021

giuseppe commented Dec 17, 2021

Lunarequest commented Dec 17, 2021

giuseppe commented Dec 17, 2021

Lunarequest commented Dec 17, 2021

Lunarequest commented Dec 19, 2021

Lunarequest commented Dec 19, 2021

giuseppe commented Dec 20, 2021

giuseppe commented Dec 20, 2021

Lunarequest commented Dec 20, 2021

r3h0 commented Feb 25, 2022

markstos commented Nov 30, 2022

dzmitry-lahoda commented Mar 12, 2023

Lunarequest commented Dec 17, 2021 •

edited

Lunarequest commented Dec 17, 2021 •

edited