cannot join user namespace of restored container

[gitsvm]

Below is the config.json of the restored container. I made multiple changes to resolve the error but still couldn't do it.

{
	"ociVersion": "1.0.2-dev",
	"process": {
		"terminal": false,
		"user": {
			"uid": 0,
			"gid": 0
		},
		"args": [
			"sh",
			"./script.sh"
		],
		"env": [
			"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
			"TERM=xterm"
		],
		"cwd": "/",
		"capabilities": {
			"bounding": [
				"CAP_AUDIT_WRITE",
				"CAP_KILL",
				"CAP_NET_BIND_SERVICE",
				"CAP_SYS_ADMIN"				
			],
			"effective": [
				"CAP_AUDIT_WRITE",
				"CAP_KILL",
				"CAP_NET_BIND_SERVICE",
				"CAP_SYS_ADMIN"				
			],
			"inheritable": [
				"CAP_AUDIT_WRITE",
				"CAP_KILL",
				"CAP_NET_BIND_SERVICE",
				"CAP_SYS_ADMIN"				
			],
			"permitted": [
				"CAP_AUDIT_WRITE",
				"CAP_KILL",
				"CAP_NET_BIND_SERVICE",
				"CAP_SYS_ADMIN"
			],
			"ambient": [
				"CAP_AUDIT_WRITE",
				"CAP_KILL",
				"CAP_NET_BIND_SERVICE",
				"CAP_SYS_ADMIN"
			]
		},
		"rlimits": [
			{
				"type": "RLIMIT_NOFILE",
				"hard": 1024,
				"soft": 1024
			}
		],
		"noNewPrivileges": true
	},
	"root": {
		"path": "rootfs",
		"readonly": false
	},
	"hostname": "runc",
	"mounts": [
		{
			"destination": "/proc",
			"type": "proc",
			"source": "proc",
			"options": [
				"rw"
			]			
		},
		{
			"destination": "/dev",
			"type": "tmpfs",
			"source": "tmpfs",
			"options": [
				"nosuid",
				"strictatime",
				"mode=755",
				"size=65536k",
				"rw"
			]
		},
		{
			"destination": "/dev/pts",
			"type": "devpts",
			"source": "devpts",
			"options": [
				"nosuid",
				"exec",
				"newinstance",
				"ptmxmode=0666",
				"mode=0620",
				"gid=5",
				"rw"
			]
		},
		{
			"destination": "/dev/shm",
			"type": "tmpfs",
			"source": "shm",
			"options": [
				"nosuid",
				"exec",
				"nodev",
				"mode=1777",
				"size=65536k",
				"rw"
			]
		},
		{
			"destination": "/dev/mqueue",
			"type": "mqueue",
			"source": "mqueue",
			"options": [
				"nosuid",
				"exec",
				"nodev",
				"rw"
			]
		},
		{
			"destination": "/sys",
			"type": "sysfs",
			"source": "sysfs",
			"options": [
				"nosuid",
				"exec",
				"nodev",
				"rw"
			]
		},
		{
			"destination": "/sys/fs/cgroup",
			"type": "cgroup",
			"source": "cgroup",
			"options": [
				"nosuid",
				"exec",
				"nodev",
				"relatime",
				"rw"
			]
		}
	],
	"linux": {
		"resources": {
			"devices": [
				{
					"allow": true,
					"access": "rwm"
				}
			]
		},
        "uidMappings": [
            {
                "hostID": 1000,
                "containerID": 0,
                "size": 1
            }
        ],
        "gidMappings": [
            {
                "hostID": 1000,
                "containerID": 0,
                "size": 1
            }
        ],		
		"namespaces": [
			{
				"type": "pid"
			},
			{
				"type": "network"
			},
			{
				"type": "ipc"
			},
			{
				"type": "uts"
			},
			{
				"type": "mount"
			},
			{
				"type": "user"
			}
		],
		"maskedPaths": [
			"/proc/acpi",
			"/proc/asound",
			"/proc/kcore",
			"/proc/keys",
			"/proc/latency_stats",
			"/proc/timer_list",
			"/proc/timer_stats",
			"/proc/sched_debug",
			"/sys/firmware",
			"/proc/scsi"
		],
		"readonlyPaths": [
			"/proc/sysrq-trigger"
		]
	}
}

an external process takes <pid> of the restored container and does setns on the namespaces file descriptors. It is able to join all other namespaces except the user namespace, for that it shows Invalid Argument. I'm stuck here for long and unable to figure out what is the issue here?

[adrianreber]

I have never seen the combination of runc with user namespaces and CRIU working. I have never really tried it but this might not been implemented yet.

[rst0git]

@gitsvm What is your runc and kernel versions?

There was a fix for checkpoint/restore of runc containers with user namespace in CRIU v3.14, but I think the Invalid Argument error may be related to opencontainers/runc#1343.

[gitsvm]

@adrianreber
Then how does criu ensures restoring all the namespaces of checkpointed runc container? and If it's not possible is there any other way of joining the container namespace from outside and still be able to do priviledge operations inside the container (i.e. joining process has priviledge access similar to criu).

I have never seen the combination of runc with user namespaces and CRIU working. I have never really tried it but this might not been implemented yet.

@rst0git
runc version:

runc version 1.0.0-rc10+dev
commit: 3c8da9dae0f3e673839d0fe3f0d862b9b2325f1c
spec: 1.0.2-dev
runc version 1.0.0-rc10+dev
commit: 3c8da9dae0f3e673839d0fe3f0d862b9b2325f1c
spec: 1.0.2-dev

Kernel version : 5.3.0-64-generic

@gitsvm What is your runc and kernel versions?

There was a fix for checkpoint/restore of runc containers with user namespace in CRIU v3.14, but I think the Invalid Argument error may be related to opencontainers/runc#1343.

[adrianreber]

Then how does criu ensures restoring all the namespaces of checkpointed runc container? and If it's not possible is there any other way of joining the container namespace from outside and still be able to do priviledge operations inside the container (i.e. joining process has priviledge access similar to criu).

I do not know the state of user namespaces, runc and CRIU. @rst0git seems to be more familiar with it, but if you are running a container as root (with Podman for example), then user namespace is just not used. I see the same running a container with cri-o. So if I am right and user namespaces, runc and CRIU does not work, then it is just not a problem because, until now, they are not used for containers running as root.