Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update command causes 'Operation not permitted' #2204

Closed
mYmNeo opened this issue Jan 13, 2020 · 1 comment · Fixed by #2391
Closed

Update command causes 'Operation not permitted' #2204

mYmNeo opened this issue Jan 13, 2020 · 1 comment · Fixed by #2391
Labels

Comments

@mYmNeo
Copy link

mYmNeo commented Jan 13, 2020

It's first seen in a kubernetes cluster with docker as container runtime. Our users reported that in some situation their bash script failed with message can't create /dev/null: Operation not permitted. But /dev/null is default device with permission rwm, After digging some logs, we found that it can be reproduced in runc by following steps.

  1. Run a runc container like "busybox". Suppose this container is called A
  2. run while true;do echo >/dev/null;done in container
  3. runc update --cpu-share 1024 A
  4. You will see sh: can't create /dev/null: Operation not permitted

The problem is located at https://github.com/opencontainers/runc/blob/master/libcontainer/cgroups/fs/devices.go#L28. By default, if runc wants to allow some devices, it'll first deny all devices then allow specific devices. It's OK when launching a new container, but if you update the resource of a container, it'll make a change that allowed devices becomes denied first, then allowed, during this period, user app will get xxx Operation not permitted error message.

@ltalirz
Copy link

ltalirz commented Dec 19, 2022

The fix #2391 by @cyphar mentions

A similar issue affects the cgroupv2 devices setup, but that is a topic for another time (as the solution is drastically different).

May I ask whether a solution for cgroupv2 is already available?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
3 participants