New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cpuset disabled #1950

Closed
rcwoolley opened this Issue Apr 7, 2017 · 16 comments

Comments

Projects
None yet
4 participants
@rcwoolley

rcwoolley commented Apr 7, 2017

I noticed that cpuset was disabled when using my Raspberry Pi. Others have also run into this in the forums: https://www.raspberrypi.org/forums/viewtopic.php?t=178981&p=1139830

I reverted this commit and it restored cpuset: 7dbc189

It seems that the commit was actually trying to disable the memory resource controller. I can confirm that "memory" remains enabled when the patch is present. Also using cgroup_enable=cpuset does re-enable
cpuset.

I thought I should report it in case it helps others. Would you be open to reverting the commit?

@popcornmix

This comment has been minimized.

Collaborator

popcornmix commented Apr 7, 2017

Back when this commit was first introduced:
CONFIG_MEMCG and CONFIG_CGROUP_MEM_RES_CTLR
resulted in 8 + 24 bytes of accounting memory per 4K page.

http://cateee.net/lkddb/web-lkddb/MEMCG.html
Note that setting this option increases fixed memory overhead associated with each page of memory in the system. By this, 8(16)bytes/PAGE_SIZE on 32(64)bit system will be occupied by memory usage tracking struct at boot. Total amount of this is printed out at boot.
Only enable when you're ok with these trade offs and really sure you need the memory resource controller. Even when you enable this, you can set "cgroup_disable=memory" at your boot option to disable memory resource controller and you can avoid overheads. (and lose benefits of memory resource controller)
http://cateee.net/lkddb/web-lkddb/CGROUP_MEM_RES_CTLR.html
Provides a memory resource controller that manages both anonymous memory and page cache. (See Documentation/cgroups/memory.txt)
Note that setting this option increases fixed memory overhead associated with each page of memory in the system. By this, 20(40)bytes/PAGE_SIZE on 32(64)bit system will be occupied by memory usage tracking struct at boot. Total amount of this is printed out at boot.
Only enable when you're ok with these trade offs and really sure you need the memory resource controller. Even when you enable this, you can set "cgroup_disable=memory" at your boot option to disable memory resource controller and you can avoid overheads. (and lose benefits of memory resource controller)
This config option also selects MM_OWNER config option, which could in turn add some fork/exit overhead.

So we made the choice a boottime switch which avoided the memory overhead unless you added cgroup_enable=memory to cmdline.txt to enable it (upstream code only provides a parameter to disable group options).

Now, it seems the descriptions of the config options have changed, and I haven't tracked down what the current cost is of enabling the options.

What exactly is your issue? Are you wanting to use memory groups? Does cgroup_enable=memory make them work?

@rcwoolley

This comment has been minimized.

rcwoolley commented Apr 7, 2017

My issue is that I want to use the cpuset functionality in cgroups and this commit is disabling the cpuset subsystem when it means to disable memory.

I suspect that the cgroup subsystems don't get initiated in a strict order, which would cause this line:
+static u16 cgroup_disable_mask __initdata = 1<<0;
to inadvertently disable the cpuset controller instead of the memory controller.

I searched Google for "raspberrypi dmesg cgroup" and saw a number of examples where the printks "Initializing cgroup subsys ****" are printed in different orders. Generally, "Initializing cgroup subsys cpuset" seemed to be listed first, however that wasn't always the case.

@popcornmix

This comment has been minimized.

Collaborator

popcornmix commented Apr 7, 2017

The initial version of the patch (0ce4380) was safer as the disable flags were in separate structures.

Would you be happy if 7dbc189 were reverted and CONFIG_MEMCG was removed?

CONFIG_MEMCG was mainly added for the Epiphany browser, but as that is no longer the recommended browser, it is of less interest to enable it.

@rcwoolley

This comment has been minimized.

rcwoolley commented Apr 7, 2017

I can't recommend disabling CONFIG_MEMCG as I don't know how long it has been enabled for in the official kernels people are using on their RPis. Someone else may depend on it. However, reverting the patch would allow people to use cpuset set again, which is unlikely to negatively affect anyone.

@popcornmix

This comment has been minimized.

Collaborator

popcornmix commented Apr 10, 2017

I've checked and CONFIG_MEMCG enabled seems to cost ~2MB on a freshly booted Pi3.
Disabled:

pi@domnfs:~ $ free -h
             total       used       free     shared    buffers     cached
Mem:          590M        86M       503M       4.8M        52K        46M
-/+ buffers/cache:        39M       550M
Swap:         511M         0B       511M

Enabled:

pi@domnfs:~ $ cat free1 
             total       used       free     shared    buffers     cached
Mem:          590M        88M       501M       4.8M        52K        49M
-/+ buffers/cache:        39M       550M
Swap:         511M         0B       511M
@JamesH65

This comment has been minimized.

Contributor

JamesH65 commented Sep 13, 2017

@rcwolley @popcornmix Will be closing if no further comments are posted.

@rcwoolley

This comment has been minimized.

rcwoolley commented Sep 13, 2017

Hi @JamesH65,

Based on this line (https://github.com/raspberrypi/linux/blob/rpi-4.9.y/kernel/cgroup.c#L5640 ) it looks as if the patch is still present and potentially disabling cpuset by mistake.

It is my suggestion to revert the patch to restore cpuset.

Regards,
Rob

@JamesH65

This comment has been minimized.

Contributor

JamesH65 commented Sep 14, 2017

@popcornmix @pelwell Any thoughts? I have no ideas what this should be.

@pelwell

This comment has been minimized.

Contributor

pelwell commented Sep 14, 2017

It looks like we need an alternative way of implementing an implicit cgroup_disable=memory that copes with different bit assignments. Rather than adding a generic cgroup_enable= parameter (which is only really useful for the the memory subsystem), we could add a flag to enable the memory cgroup which is initially cleared, and then one of the following:

  1. In the for_each_subsys loop in cgroup_init_early, if the flag is not set then compare ss->name with "memory" and set the corresponding mask bit if it matches.

  2. Somewhere near the start of cgroup_init, call cgroup_disable("memory"); if the flag is not set.

If that sounds reasonable then I'll work it up into a patch.

@popcornmix

This comment has been minimized.

Collaborator

popcornmix commented Sep 15, 2017

Sounds fine to me.

@pelwell

This comment has been minimized.

Contributor

pelwell commented Sep 18, 2017

See #2201

@rcwoolley

This comment has been minimized.

rcwoolley commented Sep 18, 2017

Thanks, Phil. This will make it much easier to use cpuset on the official Raspberry foundation kernels.

@pelwell

This comment has been minimized.

Contributor

pelwell commented Sep 18, 2017

Is that confirmation that it works for you?

@rcwoolley

This comment has been minimized.

rcwoolley commented Sep 20, 2017

I'll take the action to compile a fresh kernel and confirm it works later today.

@rcwoolley

This comment has been minimized.

rcwoolley commented Sep 23, 2017

Sorry for the delay. I cloned the latest changed and confirmed with /proc/cgroups that memory was disabled by default and enabled when I set the kernel param.

I did see a warning about not checking the return value of kstrtobool. I don't think it changes the behaviour.

Thanks for making the change!

@rcwoolley rcwoolley closed this Sep 23, 2017

@pelwell

This comment has been minimized.

Contributor

pelwell commented Nov 28, 2017

After some feedback and a rethink I've reworked this code as two new patches. The first makes cgroup_enable=memory work as expected, and the second adds the cgroup_memory= flag as a backwards compatibility patch which may be dropped for 4.14.

pelwell added a commit that referenced this issue Nov 28, 2017

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Dec 1, 2017

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit to raspberrypi/firmware that referenced this issue Dec 1, 2017

kernel: Bump to 4.14.3
kernel: cgroup: Disable cgroup memory by default
See: raspberrypi/linux#1950

firmware: Update to latest master

popcornmix added a commit to Hexxeh/rpi-firmware that referenced this issue Dec 1, 2017

kernel: Bump to 4.14.3
kernel: cgroup: Disable cgroup memory by default
See: raspberrypi/linux#1950

firmware: Update to latest master

Terminus-IMRC referenced this issue in raspberrypi/firmware Dec 2, 2017

kernel: Bump to 4.9.66
kernel: config: Add GPIO_PCF857X=m
See: raspberrypi/linux#2294

kernel: mcp2515: Use DT-supplied interrupt flags
See: raspberrypi/linux#2175

kernel: cgroup: Fix automatic disabling of cgroup memory
See: https://github.com/raspberrypi/issues/1950

firmware: arm_dt: Suppress non-fatal errors
See: #906

firmware: dtoverlay: Create "/aliases" node when needed
See: #906

pelwell added a commit that referenced this issue Dec 5, 2017

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

pelwell added a commit that referenced this issue Dec 5, 2017

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Oct 8, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Oct 15, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Oct 15, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

bluegray pushed a commit to bluegray/ubuntu-bionic that referenced this issue Oct 17, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: raspberrypi/linux#1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Oct 22, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Oct 22, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Nov 4, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Nov 4, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Nov 12, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Nov 12, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Nov 13, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Nov 15, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Nov 15, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

lyakh added a commit to lyakh/linux that referenced this issue Nov 19, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: raspberrypi/linux#1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Nov 19, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Nov 21, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Nov 21, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Nov 23, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Nov 28, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Nov 28, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

lyakh added a commit to lyakh/linux that referenced this issue Nov 30, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: raspberrypi/linux#1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Dec 4, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Dec 4, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Dec 6, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Dec 10, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Dec 10, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

Cyndent pushed a commit to Cyndent/ubuntu-core-4.15 that referenced this issue Dec 11, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: raspberrypi/linux#1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Dec 14, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Dec 17, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>

popcornmix added a commit that referenced this issue Dec 17, 2018

cgroup: Disable cgroup "memory" by default
Some Raspberry Pis have limited RAM and most users won't use the
cgroup memory support so it is disabled by default. Enable with:

    cgroup_enable=memory

See: #1950

Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment