New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow setting ulimits for containers #9437

Merged
merged 1 commit into from Mar 4, 2015

Conversation

Projects
None yet
@cpuguy83
Contributor

cpuguy83 commented Dec 1, 2014

Closes #4717

Add option to set ulimit settings for containers.
Currently the container inherits from the daemon's ulimit settings.

Support for this was added in libcontainer here docker/libcontainer@7ce34f5

@crosbymichael

This comment has been minimized.

Show comment
Hide comment
@crosbymichael

crosbymichael Dec 1, 2014

Contributor

I like it LGTM

Contributor

crosbymichael commented Dec 1, 2014

I like it LGTM

@SvenDowideit

This comment has been minimized.

Show comment
Hide comment
@SvenDowideit

SvenDowideit Dec 2, 2014

Contributor

Doc LGTM - @fredlf @jamtur01

it might be good to add what happens if you try to set a limit in the container that is larger than that on the host.

Contributor

SvenDowideit commented Dec 2, 2014

Doc LGTM - @fredlf @jamtur01

it might be good to add what happens if you try to set a limit in the container that is larger than that on the host.

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Dec 2, 2014

Contributor

@SvenDowideit I think it doesn't matter since Docker is setting the limit, which has full host access.

Contributor

cpuguy83 commented Dec 2, 2014

@SvenDowideit I think it doesn't matter since Docker is setting the limit, which has full host access.

@cpuguy83 cpuguy83 referenced this pull request Dec 2, 2014

Closed

sysctl tunables #4717

@fredlf

This comment has been minimized.

Show comment
Hide comment
@fredlf

fredlf Dec 2, 2014

Contributor

Docs LGTM, though I would like that one point clarified, ideally. Thanks.

Contributor

fredlf commented Dec 2, 2014

Docs LGTM, though I would like that one point clarified, ideally. Thanks.

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Dec 3, 2014

Contributor

@fredlf Updated.

Contributor

cpuguy83 commented Dec 3, 2014

@fredlf Updated.

@jamtur01

This comment has been minimized.

Show comment
Hide comment
@jamtur01

jamtur01 Dec 9, 2014

Contributor

LGTM

Contributor

jamtur01 commented Dec 9, 2014

LGTM

1 similar comment
@fredlf

This comment has been minimized.

Show comment
Hide comment
@fredlf

fredlf Dec 10, 2014

Contributor

LGTM

Contributor

fredlf commented Dec 10, 2014

LGTM

@fredlf

This comment has been minimized.

Show comment
Hide comment
@fredlf

fredlf Dec 10, 2014

Contributor

@cpuguy83 I'll merge if you can address the conflicts.

Contributor

fredlf commented Dec 10, 2014

@cpuguy83 I'll merge if you can address the conflicts.

@lexinator

This comment has been minimized.

Show comment
Hide comment
@lexinator

lexinator commented Dec 15, 2014

sorry i'm doing a flyby here, @fredlf this may be of interest to you, http://blog.spreedly.com/2014/06/24/merge-pull-request-considered-harmful/#.U6w1vY1dVDs

@SvenDowideit

This comment has been minimized.

Show comment
Hide comment
@SvenDowideit

SvenDowideit Dec 16, 2014

Contributor

@lexinator doesn't really apply, as this project only uses the merge-pull-request-button - and if you're involved enough that its a problem, you can request a change to the project's processes via PR :)

we also have a policy of taking over and updating PR's for contributors - but @cpuguy83 is a pretty core contributor, so :)

Contributor

SvenDowideit commented Dec 16, 2014

@lexinator doesn't really apply, as this project only uses the merge-pull-request-button - and if you're involved enough that its a problem, you can request a change to the project's processes via PR :)

we also have a policy of taking over and updating PR's for contributors - but @cpuguy83 is a pretty core contributor, so :)

@icecrime

This comment has been minimized.

Show comment
Hide comment
@icecrime

icecrime Jan 14, 2015

Contributor

Talked with @crosbymichael: we believe this would get merged if you submitted an implementation. Do you want to close this one and make a new PR when ready, or add new commits here?

Contributor

icecrime commented Jan 14, 2015

Talked with @crosbymichael: we believe this would get merged if you submitted an implementation. Do you want to close this one and make a new PR when ready, or add new commits here?

@jeremyeder

This comment has been minimized.

Show comment
Hide comment
@jeremyeder

jeremyeder Jan 14, 2015

I plan to test this when there's code.

jeremyeder commented Jan 14, 2015

I plan to test this when there's code.

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Jan 14, 2015

Contributor

Working on it. I'll put commits here.

Contributor

cpuguy83 commented Jan 14, 2015

Working on it. I'll put commits here.

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Jan 14, 2015

Contributor

Ok, it's done, tear it up!

Note, I changed the the HostConifg->Ulimits field to be an array instead of map.

Contributor

cpuguy83 commented Jan 14, 2015

Ok, it's done, tear it up!

Note, I changed the the HostConifg->Ulimits field to be an array instead of map.

@cpuguy83 cpuguy83 changed the title from Proposal: Allow setting ulimits for containers to Allow setting ulimits for containers Jan 14, 2015

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah
Member

thaJeztah commented Jan 14, 2015

Ulimtits!

@crosbymichael

This comment has been minimized.

Show comment
Hide comment
@crosbymichael

crosbymichael Jan 15, 2015

Contributor

You are not done yet, still needs a rebase ;)

Contributor

crosbymichael commented Jan 15, 2015

You are not done yet, still needs a rebase ;)

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Jan 15, 2015

Contributor

All because of your --read-only or ;)

Contributor

cpuguy83 commented Jan 15, 2015

All because of your --read-only or ;)

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Jan 15, 2015

Contributor

Rebased and ready.

Contributor

cpuguy83 commented Jan 15, 2015

Rebased and ready.

Show outdated Hide outdated daemon/execdriver/native/create.go Outdated
@philips

This comment has been minimized.

Show comment
Hide comment
@philips

philips Feb 26, 2015

Contributor

lgtm, I couldn't find the default for 'default-ulimit'. I assume it is 1024?

Contributor

philips commented Feb 26, 2015

lgtm, I couldn't find the default for 'default-ulimit'. I assume it is 1024?

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Feb 26, 2015

Contributor

@philips No default-defaults are set, and would still inherit from the docker daemon settings.

Contributor

cpuguy83 commented Feb 26, 2015

@philips No default-defaults are set, and would still inherit from the docker daemon settings.

@philips

This comment has been minimized.

Show comment
Hide comment
@philips

philips Feb 26, 2015

Contributor

@cpuguy83 makes sense, I see it in the docs now, no idea how I missed that. Thanks.

Contributor

philips commented Feb 26, 2015

@cpuguy83 makes sense, I see it in the docs now, no idea how I missed that. Thanks.

@jessfraz

This comment has been minimized.

Show comment
Hide comment
@jessfraz

jessfraz Mar 4, 2015

Contributor

what is this waiting on?

Contributor

jessfraz commented Mar 4, 2015

what is this waiting on?

jessfraz pushed a commit that referenced this pull request Mar 4, 2015

Jessie Frazelle
Merge pull request #9437 from cpuguy83/set_rlimits_in_container
Allow setting ulimits for containers

@jessfraz jessfraz merged commit df7ba57 into moby:master Mar 4, 2015

1 check passed

janky Jenkins build Docker-PRs 1866 has succeeded
Details

@cpuguy83 cpuguy83 deleted the cpuguy83:set_rlimits_in_container branch Apr 16, 2015

@liquid-sky

This comment has been minimized.

Show comment
Hide comment
@liquid-sky

liquid-sky Apr 22, 2015

How can one provide RLIM_INFINITY value as per resource.h or equivalent of providing unlimited value when running ulimit for a resource?

liquid-sky commented Apr 22, 2015

How can one provide RLIM_INFINITY value as per resource.h or equivalent of providing unlimited value when running ulimit for a resource?

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Apr 22, 2015

Contributor

@liquid-sky it is unsupported. There really isn't an unlimited, it's just a really high value.

Contributor

cpuguy83 commented Apr 22, 2015

@liquid-sky it is unsupported. There really isn't an unlimited, it's just a really high value.

@frol

This comment has been minimized.

Show comment
Hide comment
@frol

frol Apr 23, 2015

UPDATE: It seems that ulimit subsystem is not namespaced and applies limits to all processes based only on UID/GID.

Original post:

I cannot figure out why it happens, but it seems that resources that are limited by ulimit are counted across containers. Here is an example:

If I run one container with --ulimit 'nproc=2' it runs fine, but I cannot start the second container with the same --ulimit 'nproc=2':

Terminal #1:

$ docker run -it --rm --ulimit 'nproc=2' --user nobody debian bash
nobody@7005f259a827:/$

Terminal #2:

$ docker run -it --rm --ulimit 'nproc=2' --user nobody debian bash
resource temporarily unavailable
FATA[0000] Error response from daemon: Cannot start container 589299c070779487462393fcb04df05d619d2debe1b1197c41587ee53c2283b8: [8] System error: resource temporarily unavailable

I tested this on Ubuntu 14.04 (kernel 3.16) and Ubuntu 12.04 (kernel 3.13), Docker 1.6.0.

frol commented Apr 23, 2015

UPDATE: It seems that ulimit subsystem is not namespaced and applies limits to all processes based only on UID/GID.

Original post:

I cannot figure out why it happens, but it seems that resources that are limited by ulimit are counted across containers. Here is an example:

If I run one container with --ulimit 'nproc=2' it runs fine, but I cannot start the second container with the same --ulimit 'nproc=2':

Terminal #1:

$ docker run -it --rm --ulimit 'nproc=2' --user nobody debian bash
nobody@7005f259a827:/$

Terminal #2:

$ docker run -it --rm --ulimit 'nproc=2' --user nobody debian bash
resource temporarily unavailable
FATA[0000] Error response from daemon: Cannot start container 589299c070779487462393fcb04df05d619d2debe1b1197c41587ee53c2283b8: [8] System error: resource temporarily unavailable

I tested this on Ubuntu 14.04 (kernel 3.16) and Ubuntu 12.04 (kernel 3.13), Docker 1.6.0.

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Apr 23, 2015

Contributor

After some digging, it seems nproc is special and is per user.

Contributor

cpuguy83 commented Apr 23, 2015

After some digging, it seems nproc is special and is per user.

@frol

This comment has been minimized.

Show comment
Hide comment
@frol

frol Apr 23, 2015

Note: another part of discussion is going in #6479 (Fork bomb prevention) and will probably continue there.

frol commented Apr 23, 2015

Note: another part of discussion is going in #6479 (Fork bomb prevention) and will probably continue there.

@ashish235

This comment has been minimized.

Show comment
Hide comment
@ashish235

ashish235 Aug 3, 2015

So how to set the Ulimit? Can it be done per container wise? I seem to be clueless with the discussions happening in this thread.. Can someone help please.
I need "ulimit -c unlimited" equivalent for docker.

ashish235 commented Aug 3, 2015

So how to set the Ulimit? Can it be done per container wise? I seem to be clueless with the discussions happening in this thread.. Can someone help please.
I need "ulimit -c unlimited" equivalent for docker.

@ashish235

This comment has been minimized.

Show comment
Hide comment
@ashish235

ashish235 Aug 3, 2015

[root@localhost ~]# docker run --ulimit core=unlimited -t ubuntu:12.04 bash -c "ulimit -c"
invalid value "core=unlimited" for flag --ulimit: strconv.ParseInt: parsing "unlimited": invalid syntax
See 'docker run --help'.

Unlimited is not a recognized keyword here.

My docker version is the latest 1.7.

[root@localhost ~]# docker version
Client version: 1.6.2
Client API version: 1.18
Go version (client): go1.4.2
Git commit (client): ba1f6c3/1.6.2
OS/Arch (client): linux/amd64
INFO[0203] GET /v1.18/version
Server version: 1.7.1
Server API version: 1.19
Go version (server): go1.4.2
Git commit (server): 786b29d
OS/Arch (server): linux/amd64

Also setting big values doesn't help here.

[root@localhost ~]# docker run --ulimit core=99999 -t ubuntu:12.04 bash -c "ulimit -c"
97

It sets only 97.

ashish235 commented Aug 3, 2015

[root@localhost ~]# docker run --ulimit core=unlimited -t ubuntu:12.04 bash -c "ulimit -c"
invalid value "core=unlimited" for flag --ulimit: strconv.ParseInt: parsing "unlimited": invalid syntax
See 'docker run --help'.

Unlimited is not a recognized keyword here.

My docker version is the latest 1.7.

[root@localhost ~]# docker version
Client version: 1.6.2
Client API version: 1.18
Go version (client): go1.4.2
Git commit (client): ba1f6c3/1.6.2
OS/Arch (client): linux/amd64
INFO[0203] GET /v1.18/version
Server version: 1.7.1
Server API version: 1.19
Go version (server): go1.4.2
Git commit (server): 786b29d
OS/Arch (server): linux/amd64

Also setting big values doesn't help here.

[root@localhost ~]# docker run --ulimit core=99999 -t ubuntu:12.04 bash -c "ulimit -c"
97

It sets only 97.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Aug 3, 2015

Member

Hi @ashish235 please keep in mind that GitHub is an issue tracker, and not a general support forum.

A number of things I noticed; first of all, although your daemon is 1.7.1, your client is still at 1.6.2, so you might want to update that as well.

There were some fixes to ulimits in docker 1.7.x, but I'm not sure if those fixes were in the client or daemon, so this may be worth checking. (#12515)

W.r.t. "unlimited"; #12515 (comment)

unlimited is purposefully not support since it's not a real value, it's just a constant set to a really high number.

W.r.t. "setting big values doesn't help here"; please note this comment: #12957 (comment)

It sets the limit value as number of bytes, while ulimit reports the number of kb. So 32768 bytes == 32 kb

Converting 99999 to kb; https://www.google.com/search?q=99999%20bytes%20in%20kibibyte I get "97", so this looks right

Member

thaJeztah commented Aug 3, 2015

Hi @ashish235 please keep in mind that GitHub is an issue tracker, and not a general support forum.

A number of things I noticed; first of all, although your daemon is 1.7.1, your client is still at 1.6.2, so you might want to update that as well.

There were some fixes to ulimits in docker 1.7.x, but I'm not sure if those fixes were in the client or daemon, so this may be worth checking. (#12515)

W.r.t. "unlimited"; #12515 (comment)

unlimited is purposefully not support since it's not a real value, it's just a constant set to a really high number.

W.r.t. "setting big values doesn't help here"; please note this comment: #12957 (comment)

It sets the limit value as number of bytes, while ulimit reports the number of kb. So 32768 bytes == 32 kb

Converting 99999 to kb; https://www.google.com/search?q=99999%20bytes%20in%20kibibyte I get "97", so this looks right

shimaore added a commit to shimaore/ccnq4-opensips that referenced this pull request Dec 7, 2015

Add (partial) support for coredumps
First attempt: most probably Docker.io would not honor the values in /etc/security etc. so this would not work:

    # Core Generation: http://www.opensips.org/Documentation/TroubleShooting-Crash
    RUN \
      echo -n -e 'opensips soft core unlimited\nopensips hard core unlimited\n' >> /etc/security/limits.conf \
      && \
      echo -n -e 'fs.suid_dumpable = 1\nkernel.core_uses_pid = 1\n' >> /etc/sysctl.conf

See e.g. moby/moby#9437

@kesor kesor referenced this pull request Jan 26, 2018

Open

rlimit support #3595

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment