Skip to content

Conversation

galal-hussein
Copy link
Contributor

The PR fixes nestybox/sysbox#958 and adds support to containerd > 2.0, after investigation I found out that containerd was failing to run sysbox-runc features and failing to register the runtime as a runtime that supports user namesapces:

can't set `spec.hostUsers: false`, RuntimeClass handler "sysbox-runc" does not support user namespaces

After enabling the debug log in containerd, the issue was more clear:

level=debug msg="failed to introspect features of runtime \"sysbox-runc\"" error="failed to unmarshal Features (*anypb.Any): type with url : not found"

The PR adds the features command to sysbox-runc hence enabling contaienrd to work with sysbox, this was tested on k3s setup with no modification to containerd except for adding runtime to the config:

/var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl

...
[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.sysbox-runc]
  runtime_type = "io.containerd.runc.v2"

[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.sysbox-runc.options]
  SystemdCgroup = false
  BinaryName="/usr/bin/sysbox-runc"
...

And running pod with the runtime class for sysbox-runc and hostUsers: false:

# cat pod.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: ubuntu
spec:
  runtimeClassName: sysbox-runc
  hostUsers: false
  containers:
  - name: ubuntu2204
    image: ubuntu:22.04
    command: ["sleep", "40000000000"]
  restartPolicy: Never

I was able to run k3s within that pod successfully.

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>
@ctalledo
Copy link
Member

Hi @galal-hussein, thanks for the contribution, will take a look ASAP.

I was able to run k3s within that pod successfully.

Can you do cat /proc/self/uid_map within the pod and post the result, just to double check the user-namespace mapping looks good?

Also, does each pod get a different user-ns mapping?

@ctalledo ctalledo assigned ctalledo and unassigned ctalledo Aug 29, 2025
@ctalledo ctalledo self-requested a review August 29, 2025 22:16
@galal-hussein
Copy link
Contributor Author

Sure, here are the results of two different pods running the sysbox-runc runtime:

➜  k3k git:(chart-0.3.4-rc3) ✗ kubectl exec -it  k3k-virtualcluster-server-0 -n test-x -- sh
~ # cat /proc/self/uid_map 
         0  178192384      65536


➜  k3k git:(chart-0.3.4-rc3) ✗ kubectl exec -it  k3k-virtualcluster-server-1 -n test-x -- sh
~ # cat /proc/self/uid_map 
         0  685834240      65536

The user namespace is intact, and each pod gets a different mapping.

Copy link
Member

@ctalledo ctalledo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @galal-hussein for the contribution!

Mostly looks good, just a few comments.

return nil, err
}
if len(unknownCaps) > 0 {
logrus.Warn("ignoring unknown or unavailable capabilities: ", mapKeys(unknownCaps))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any particular reason for not using slices.Sorted(maps.Keys(unknownCaps)) (and therefore avoid the mapKeys() function)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no not really, slices.Sorted seems more efficient, will fix

var exists bool
if config.RootPropagation, exists = mountPropagationMapping[spec.Linux.RootfsPropagation]; !exists {
return nil, fmt.Errorf("rootfsPropagation=%v is not supported", spec.Linux.RootfsPropagation)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove empty line please.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure will do

AnnotationRuncVersion = "org.opencontainers.runc.version"

// AnnotationRuncCommit corresponds to the output of `git describe --dirty --long --always` in the runc repo.
// Third party implementations such as crun and runsc SHOULD NOT use this annotation, as their repo is different from the runc repo.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like we should not use this annotation per the comment?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah the features annotation were copied from runc implementation of runc features command, I will remove this particular annotation tho per their instruction.

Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>
@galal-hussein galal-hussein requested a review from ctalledo August 29, 2025 23:15
Copy link
Member

@ctalledo ctalledo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@ctalledo ctalledo merged commit 1e3b5a2 into nestybox:master Aug 29, 2025
1 check passed
@ctalledo
Copy link
Member

ctalledo commented Aug 29, 2025

Thanks @galal-hussein again for the contribution!

I've merged the PR and will test it with containerd user-namespaces. Thanks!

@galal-hussein
Copy link
Contributor Author

@ctalledo Thank you very much!

@mueckinger
Copy link

mueckinger commented Sep 6, 2025

Hi, I have built latest Commit 1e3b5a2 of sysbox-runc, added the sysbox config as mentioned above to /etc/containerd/config.toml and added hostUsers: false to my manifest but Kubernetes still says:
Failed to create pod sandbox: can't set spec.hostUsers: false, RuntimeClass handler "sysbox-runc" does not support user namespaces

It seems that I had to build the -static version. Now it works in the following Environment:

  • containerd 2.1.4
  • Kubernetes 1.33.4
  • Ubuntu 24.04.3 LTS (GNU/Linux 6.8.0-78-generic x86_64)

🎉

@bindrad
Copy link

bindrad commented Sep 16, 2025

Hello, can you share the exact steps to run sysbox with containerd > 2.0 in k3s cluster?
I updated /var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl file as mentioned above but I get this error when creating pod.

Error from server (Forbidden): error when creating "STDIN": pods "ubuntu" is forbidden: pod rejected: RuntimeClass "sysbox-runc" not found

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

How to run Sysbox in containerd > 2.0
4 participants