Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Swap limit capabilities failure while trying to set memory limit #847

Closed
flaviamissi opened this issue Jun 6, 2013 · 22 comments

Comments

Projects
None yet
10 participants
@flaviamissi
Copy link

commented Jun 6, 2013

Hi guys,

I'm trying to set memory limit on docker commit using the -run option, e.g.:

docker commit  -run='{"Memory": 128}' 0b5e63a76024 flaviamissi/test

This works as expected, but when I attempt to run a container using the obtained image I get the following:

$ docker run -t -i tsuru/test /bin/bash
WARNING:  Your kernel does not support memory swap capabilities. Limitation discarded.
lxc-start: Device or resource busy - write /sys/fs/cgroup/memory/lxc/5e29ddc69751e982fd1895e5d8b8f3159184ab9859c2bc3747b6c16d17a9e1ff/memory.limit_in_bytes : Device or resource busy
lxc-start: Error setting memory.limit_in_bytes to 128 for lxc/5e29ddc69751e982fd1895e5d8b8f3159184ab9859c2bc3747b6c16d17a9e1ff

lxc-start: failed to setup the cgroups for '5e29ddc69751e982fd1895e5d8b8f3159184ab9859c2bc3747b6c16d17a9e1ff'
lxc-start: failed to spawn '5e29ddc69751e982fd1895e5d8b8f3159184ab9859c2bc3747b6c16d17a9e1ff'
lxc-start: Device or resource busy - failed to remove cgroup '/sys/fs/cgroup/cpuset/lxc/5e29ddc69751e982fd1895e5d8b8f3159184ab9859c2bc3747b6c16d17a9e1ff'
lxc-start: Device or resource busy - failed to remove cgroup '/sys/fs/cgroup/cpu/lxc/5e29ddc69751e982fd1895e5d8b8f3159184ab9859c2bc3747b6c16d17a9e1ff'
lxc-start: Device or resource busy - failed to remove cgroup '/sys/fs/cgroup/cpuacct/lxc/5e29ddc69751e982fd1895e5d8b8f3159184ab9859c2bc3747b6c16d17a9e1ff'
lxc-start: Device or resource busy - failed to remove cgroup '/sys/fs/cgroup/memory/lxc/5e29ddc69751e982fd1895e5d8b8f3159184ab9859c2bc3747b6c16d17a9e1ff'
lxc-start: Device or resource busy - failed to remove cgroup '/sys/fs/cgroup/devices/lxc/5e29ddc69751e982fd1895e5d8b8f3159184ab9859c2bc3747b6c16d17a9e1ff'
lxc-start: Device or resource busy - failed to remove cgroup '/sys/fs/cgroup/freezer/lxc/5e29ddc69751e982fd1895e5d8b8f3159184ab9859c2bc3747b6c16d17a9e1ff'

What is puzzling me is the warning printed by docker:

WARNING:  Your kernel does not support memory swap capabilities. Limitation discarded.

If you pay attention, I'm not setting the MemorySwap option, but the Memory opt.

I've found the code that logs it, and it seems that the check is wrong, see:

if container.Config.Memory > 0 && !container.runtime.capabilities.SwapLimit {
...

What should be really checked here isn't the container.Config.MemorySwap attribute?

Would someone mind to clarify this to me?

Thanks!

@unclejack

This comment has been minimized.

Copy link
Contributor

commented Jun 6, 2013

@flaviamissi The memory limit is in bytes. You'd have to use 134217728 for the memory limit if you want to set it to 128MB.

I'm getting the exact same error you're getting if I use 128 as the memory limit. Passing 134217728 works.

docker run -m 128 ubuntu true
lxc-start: Device or resource busy - write /sys/fs/cgroup/memory/lxc/bbbc00f25479d111449d3d117ca322cfe97b2fc988b26967f5371c1bda674bd7/memory.limit_in_bytes : Device or resource busy
lxc-start: Error setting memory.limit_in_bytes to 128 for lxc/bbbc00f25479d111449d3d117ca322cfe97b2fc988b26967f5371c1bda674bd7

lxc-start: failed to setup the cgroups for 'bbbc00f25479d111449d3d117ca322cfe97b2fc988b26967f5371c1bda674bd7'
lxc-start: failed to spawn 'bbbc00f25479d111449d3d117ca322cfe97b2fc988b26967f5371c1bda674bd7'

# let's set the limit in bytes
docker run -m 134217728 ubuntu true
# execution ends without any issue
@flaviamissi

This comment has been minimized.

Copy link
Author

commented Jun 6, 2013

Yes, I just figured that out, thanks!

But the warning still seems wrong...

@unclejack

This comment has been minimized.

Copy link
Contributor

commented Jun 6, 2013

What distro and what kernel are you running docker on? I didn't get that warning on Ubuntu 13.04 with kernel 3.9 when passing 128 as the memory limit.

@flaviamissi

This comment has been minimized.

Copy link
Author

commented Jun 6, 2013

ubuntu 13.04 with kernel 3.8

@flaviamissi

This comment has been minimized.

Copy link
Author

commented Jun 6, 2013

The 3.9 kernel might have the swap capability, but what I'm trying to say is that the if that checks it is wrong:

@flaviamissi

This comment has been minimized.

Copy link
Author

commented Jun 6, 2013

pressed commend before finishing, I meant this if:

if container.Config.Memory > 0 && !container.runtime.capabilities.SwapLimit {
@flaviamissi

This comment has been minimized.

Copy link
Author

commented Jun 6, 2013

It's probably a typo...

@shykes

This comment has been minimized.

Copy link
Collaborator

commented Jun 6, 2013

Flavia, under a certain memory allocation the system can barely get
anything done, and will fail at a very early stage - in this case it can't
even complete setting up the process group. In theory we should chase down
all cases where an error is caused by insufficient memory. In practice this
seemed like more work than it was worth.

On Thu, Jun 6, 2013 at 11:11 PM, unclejack notifications@github.com wrote:

What distro and what kernel are you running docker on? I didn't get that
warning on Ubuntu 13.04 with kernel 3.9 when passing 128 as the memory
limit.


Reply to this email directly or view it on GitHubhttps://github.com//issues/847#issuecomment-19074434
.

@unclejack

This comment has been minimized.

Copy link
Contributor

commented Jun 6, 2013

That piece of code is correct. It's meant to let us know that we lack swap limiting capabilities when we tell docker that we want to limit memory.

Swap limits are automatically computed based on the memory limits.

@flaviamissi

This comment has been minimized.

Copy link
Author

commented Jun 6, 2013

Thanks, @unclejack, that clarifies the question :)

Since it was not an error, I'll close the issue.

Thanks everybody o/

@flaviamissi flaviamissi closed this Jun 6, 2013

@shykes

This comment has been minimized.

Copy link
Collaborator

commented Jun 6, 2013

Sorry, I missed that part. the '-m' flag actually sets a certain blend of
swap and real memory, assuming a certain preset ratio. Currently you can't
control swap and actual memory limits separately in docker (although you
can in the underlying lxc config).

On Thu, Jun 6, 2013 at 11:16 PM, unclejack notifications@github.com wrote:

That piece of code is correct. It's meant to let us know that we lack swap
limiting capabilities when we tell docker that we want to limit memory.

Swap limits are automatically computed based on the memory limits.


Reply to this email directly or view it on GitHubhttps://github.com//issues/847#issuecomment-19074728
.

@samof76

This comment has been minimized.

Copy link

commented Aug 27, 2013

But so that you are not worried about it any long you need to boot the kernel with these parameters cgroup_enable=memory swapaccount=1.

To do that you would need to edit your /etc/default/grub file and set this

GRUB_CMDLINE_LINUX="cgroup_enable=memory swapaccount=1"
@matthewmueller

This comment has been minimized.

Copy link
Contributor

commented Sep 23, 2013

running into same warning while running 13.04 and with -m 5242880. @samof76 what does that command do?

@samof76

This comment has been minimized.

Copy link

commented Sep 30, 2013

@matthewmueller this is to enable memory and swap accounting. But this also enables more control features for cgroups. Please read more here... https://www.kernel.org/doc/Documentation/cgroups/memory.txt

@petri

This comment has been minimized.

Copy link

commented Oct 22, 2013

Sounds like this should be included in the docker installation docs? Ie. memory limits don't work unless swap limiting capabilities are enabled.

@rca

This comment has been minimized.

Copy link

commented Oct 30, 2013

@samof76 thanks for the GRUB config line!

@thebyrd

This comment has been minimized.

Copy link

commented May 13, 2014

@unclejack
I ran into this error when starting the daemon with sudo docker -d and then the daemon was unresponsive. The server had plenty of resources with no spikes (see graph below). I don't know exactly how many containers were running, but I'd guess around 1000. Restarting the daemon with upstart also failed.

  1. What's the best way to remove the containers without the daemon running?
  2. What's causing this error?

screen shot 2014-05-13 at 12 31 44 pm

@scadgek

This comment has been minimized.

Copy link

commented Jul 15, 2015

Sorry for necroposting, but I think this still should go here.
So I'm on Ubuntu 14.04, I updated GRUB options with GRUB_CMDLINE_LINUX="cgroup_enable=memory swapaccount=1" and GRUB_CMDLINE_LINUX_DEFAULT="cgroup_enable=memory swapaccount=1", done update-grub, rebooted and still receive
WARNING: Your kernel does not support swap limit capabilities. Limitation discarded.
By the way, I also can see these options set in /boot/grub/grub.cfg.
I have no idea what I am missing

@shredding

This comment has been minimized.

Copy link

commented Jan 26, 2016

@scadgek : Did you got it fixed? I fail to enable it as well.

@scadgek

This comment has been minimized.

Copy link

commented Jan 26, 2016

@shredding No, I didn't. The task became not important and I stopped looking for the ways to solve.

@shredding

This comment has been minimized.

Copy link

commented Jan 26, 2016

I tracked it down. I assume you are on a VPS like Digital Oceans Droplets? You'd need to use 15.05 or 15.10 then because 14.04 does not support internal kernel configuration from within a vps.

@scadgek

This comment has been minimized.

Copy link

commented Jan 26, 2016

@shredding No it wasn't Digital Ocean, it was my local machine - not even a virtual one

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.