-
Notifications
You must be signed in to change notification settings - Fork 161
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Migrate to CGroupv2 #78
Comments
For tracking purposes, dumping more docs: https://systemd.io/CGROUP_DELEGATION https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html#Delegate= |
Apparently this is a WONTFIX and can only be resolved by moving to cgroupsv2: https://lists.freedesktop.org/archives/systemd-devel/2019-May/042558.html |
I don't know where this issue fits in the roadmap of isolate, but I think it should be prioritized a bit. In fact Arch Linux disabled by default cgroups v1 and therefore isolate stopped working. They can still be enabled manually by adding May you update us on it? Thanks! |
This is currently the top item on my TODO list. |
Rudimentary implementation of the move to cgroup v2 is in the First of all, I had to solve integration with systemd. It can delegate some types of cgroups to other managers, but apparently the only way how to make the cgroup persistent is to keep a process in it. I therefore wrote a simple daemon called Isolate's config file now contains the path to a master cgroup, under which cgroups for individual sandboxes will be created. If you are using systemd, this is the cgroup maintained by The good news is that the switch to cgroup v2 simplified isolate a lot. The bad news is that I failed to find a way how to measure maximum memory usage: there is nothing like The code is still almost untested and it has a plenty of rough edges and close to no documentation. However, if you want to get your feet wet, I will be glad for any feedback. |
There is now There's no |
I'm testing the The CMS test suite fails with this new version of isolate, it seems that the This is the work-in-progress PR over at the CMS repo: cms-dev/cms#1222
|
Ok I realized I have to install the systemd configuration files 😅 I'm now getting some more interesting errors. When I try to start the
But if I check the status of
I can find the folder under Maybe this would work without docker? |
Could you please try it without Docker first? |
I consider the cgroup v2 code almost ready now. Among other things, the name of the cgroup is no longer hard-coded in the configuration file. Instead, Also, I implemented proper locking of sandboxes, so different users cannot stomp on each other's sandboxes. It also prevents There is some support for having a system-wide daemon which manages access to sandboxes. The daemon itself is not ready yet, but some rudiments can be found in the I removed the |
In the You will find a sketch of documentation at the top of I will be glad for any feedback. |
Hi @gollux, Does it compatible with CGroup v2? |
The version in the I plan to deprecate v1 and merge |
Thanks @gollux for confirmation. |
Hi @gollux ser,
|
Hi @gollux! First of all, thanks for the great library that has been very useful for us in implementing infrastructure for sandboxed live coding environments. At the moment we're trying to make a decision whether to migrate to cgroup v2 or no. Do you think the v2 is ready for production now? Thanks. |
I'm already running it in production and I plan to release it soon. The only missing thing is a bit of documentation. |
Any update on this? |
hi, anyone was able to run this under docker?, I realize I need to start the service, just not sure how to do that. Here is the simple command I'm trying to run:
and without
Also running the check I get the following:
Are those fail and warnings normal? Seems like I'm doing something wrong. I'm using the branch cg2. UPDATE:If I manually run
Thanks! |
You probably need a privileged container (I'm not sure as I don't use Docker myself). You certainly need |
@gollux , thanks for the quick response. I'm ignoring the check for now, but Running with privilege I can run successfully the following command:
however,
fails with
I think it's related to the service, can't run the service on docker, and running the keeper manually throws:
not sure what that means. I'll keep experimenting and will write an update if I find something useful. Thanks |
You need to have systemd running inside the container. |
Thanks, that won't work in my environment, I thought there may be a way around it, perhaps there still is, gotta investigate more. I'm trying to run an app that would run untrusted user code in AWS, and I thought I could spin it up as a microservice in fargate, but I don't have much control over how docker spins up, though technically they do support cgroups v2, just can't run the keeper as a service. Worst case I can deploy it to a virtual machine, but that's painful to maintain for a single man operation hehe. Is there a reason why you set up a new process with the keeper and not directly as part of the isolate one? |
Isolate needs its own subtree in the cgroup hierarchy. On systems with systemd, we can ask systemd to delegate such subtree to a service (and there must be a process running in the service to keep the subtree alive ... this is what the keeper process does). If you can obtain a subtree delegation in a different way, you can let Isolate use it by putting the path to the subtree in Isolate's config file. |
Can't isolate use cgroupfs instead of systemd for cgroupv2? |
Hi, make sure you initialized your sandbox with --cg flag before running it with --cg |
Finally merged. |
When running a container with systemd-nspawn, systemd remounts /sys/fs/cgroup in read-only. This prevents isolate from creating its own cgroup inside /sys.
Apparently, this is intended, as isolate shouldn't create its own cgroup in the root, but do it in a subgroup of the one provided by systemd: https://lists.freedesktop.org/archives/systemd-devel/2017-November/039736.html
I'm completely unfamiliar with the cgroup/Delegate API of systemd, so I'm not sure what a proper fix should look like. I'll try to investigate, but if anyone already knows what a good fix would be, don't hesitate to tell me :-P
The text was updated successfully, but these errors were encountered: