New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option to configure cgroup manager (adds --exec-opt) #10427

Merged
merged 2 commits into from Apr 29, 2015

Conversation

Projects
None yet
@jessfraz
Copy link
Contributor

jessfraz commented Jan 28, 2015

This adds a flag --exec-opt on the daemon (much like the current --storage-opt flag.
With the addition of this flag we can configure --exec-opt native.cgroupdriver.

This specifies the management of a container's cgroups. Obviously since I have chosen to add it as a daemon flag it applies to all containers. As of now the only viable options are
cgroupfs and systemd. The option will always fallback to cgroupfs. By default, if no option
is specified, the execdriver will try systemd and fallback to cgroupfs.
Same applies if systemd is passed as the cgroupdriver but is not capable of being used.

@jessfraz jessfraz force-pushed the jessfraz:option-to-disable-systemd-cgroups branch from 4c46fb4 to 94ffb18 Jan 29, 2015

@jessfraz

This comment has been minimized.

Copy link
Contributor

jessfraz commented Jan 29, 2015

pinging the people who commented on the original libcontainer patch that seemed interested: @tianon @ibuildthecloud

@tianon

This comment has been minimized.

Copy link
Member

tianon commented Jan 29, 2015

Heh, you've already covered the part I cared about (ie, consistency with our existing options). 👍

@@ -919,6 +919,7 @@ _docker() {
--registry-mirror
--storage-driver -s
--storage-opt
--exec-opt

This comment has been minimized.

@tianon

tianon Jan 29, 2015

Member

Shouldn't this be alphabetical? 😉

This comment has been minimized.

@crosbymichael

crosbymichael Jan 29, 2015

Member

ok professor

This comment has been minimized.

@tianon

@jessfraz jessfraz force-pushed the jessfraz:option-to-disable-systemd-cgroups branch 4 times, most recently from 4944a01 to ce2f3c3 Jan 29, 2015

@@ -39,7 +39,7 @@ func toggle(state cgroups.FreezerState) error {
return err
}

if systemd.UseSystemd() {
if container.Cgroups.Driver == "systemd" && systemd.UseSystemd() {

This comment has been minimized.

@chr0n1x

chr0n1x Jan 29, 2015

much container.Cgroups.Driver == "systemd". Is there a reason why it shouldn't just be container.Cgroups.IsSystemd()?

This comment has been minimized.

@jessfraz

jessfraz Jan 29, 2015

Contributor

If you look at the patch in libcontainer that's linked from this PR you
will see the circular dependency problem

On Wednesday, January 28, 2015, Kevin Ran notifications@github.com wrote:

In vendor/src/github.com/docker/libcontainer/nsinit/pause.go
#10427 (comment):

@@ -39,7 +39,7 @@ func toggle(state cgroups.FreezerState) error {
return err
}

  • if systemd.UseSystemd() {
  • if container.Cgroups.Driver == "systemd" && systemd.UseSystemd() {

much container.Cgroups.Driver == "systemd". Is there a reason why it
shouldn't just be container.Cgroups.IsSystemd()?


Reply to this email directly or view it on GitHub
https://github.com/docker/docker/pull/10427/files#r23744403.

This comment has been minimized.

@chr0n1x

chr0n1x Jan 29, 2015

womp...womp.


Options to exec backend can be specified with **--exec-opt** flags. The
only backend which currently takes options is *native* (libcontainer). Therefore
use these flags with **-s=**native.

This comment has been minimized.

@SvenDowideit

SvenDowideit Jan 29, 2015

Contributor

mmm, I think you're missing some the's, and elsewhere we call it exec-driver

Options to the exec-driver can be specified using the **--exec-opt** flags. The
only driver accepting options is the *native* (libcontainer) driver.
only backend which currently takes options is *native* (libcontainer). Therefore
use these flags with **-s=**native.
Here is the list of *native* options:

This comment has been minimized.

@SvenDowideit

SvenDowideit Jan 29, 2015

Contributor

And then you list one?

This comment has been minimized.

@jessfraz

jessfraz Jan 31, 2015

Contributor

lol sorry grammar
I probably would not have gotten into college if it wasnt for my math sat score. no joke, my grammar sucks, i will update

@jessfraz jessfraz force-pushed the jessfraz:option-to-disable-systemd-cgroups branch 3 times, most recently from 6d05677 to c5fb186 Jan 31, 2015

@jessfraz

This comment has been minimized.

Copy link
Contributor

jessfraz commented Mar 11, 2015

closing for now

@jessfraz

This comment has been minimized.

Copy link
Contributor

jessfraz commented Apr 6, 2015

reopening, then I will force push

@jessfraz jessfraz reopened this Apr 6, 2015

@jessfraz jessfraz changed the title Proposal: option to disable systemd cgroups Option to configure cgroup manager (adds --exec-opt) Apr 6, 2015

@jessfraz jessfraz force-pushed the jessfraz:option-to-disable-systemd-cgroups branch from c5fb186 to 8301d01 Apr 6, 2015

@jessfraz

This comment has been minimized.

Copy link
Contributor

jessfraz commented Apr 6, 2015

reopened to use the new libcontainer api.

ping @ibuildthecloud, I know you were interested in this one.

@jessfraz

This comment has been minimized.

Copy link
Contributor

jessfraz commented Apr 7, 2015

btw ill fix the code after we decide we like this, i dont like the current implementation

@ibuildthecloud

This comment has been minimized.

Copy link
Contributor

ibuildthecloud commented Apr 9, 2015

I like this. It makes perfect sense to me. I need to test this but ironically I think setting the cgroup manager to cgroupfs and then using cgroup parent you can actually properly supervisor docker containers with systemd.

@jessfraz jessfraz force-pushed the jessfraz:option-to-disable-systemd-cgroups branch from 8301d01 to 6dfce62 Apr 16, 2015

@GordonTheTurtle GordonTheTurtle removed the dco/no label Apr 16, 2015

return nil, fmt.Errorf("Unknown option %s\n", key)
}
}

This comment has been minimized.

@icecrime

icecrime Apr 23, 2015

Contributor

Maybe add a DEBUG or INFO log giving the cgroup manager we picked?

This comment has been minimized.

@jessfraz

jessfraz Apr 23, 2015

Contributor

added!

@@ -1150,6 +1150,7 @@ _docker() {
--dns
--dns-search
--exec-driver -e
--exec-opt

This comment has been minimized.

@icecrime

icecrime Apr 23, 2015

Contributor

I think fish completion needs updating too ;-)

viable options are `cgroupfs` and `systemd`. The option will always
fallback to `cgroupfs`. By default, if no option is specified, the
execdriver will try `systemd` and fallback to `cgroupfs`. Same applies if
`systemd` is passed as the `cgroupdriver` but is not capable of being used.

This comment has been minimized.

@jessfraz

jessfraz Apr 23, 2015

Contributor

so then I should change this last sentence and error out as well?

This comment has been minimized.

@jessfraz

jessfraz Apr 23, 2015

Contributor

aactually i think warning is enough

@jessfraz jessfraz force-pushed the jessfraz:option-to-disable-systemd-cgroups branch from 045044e to c1a4baa Apr 23, 2015

@GordonTheTurtle GordonTheTurtle removed the dco/no label Apr 23, 2015

}
}

logrus.Debugf("Using %v as native.crgoupdriver", cgm)

This comment has been minimized.

@jessfraz

jessfraz Apr 23, 2015

Contributor

much typo ugh

@jessfraz jessfraz force-pushed the jessfraz:option-to-disable-systemd-cgroups branch from c1a4baa to d03a110 Apr 23, 2015

@GordonTheTurtle GordonTheTurtle removed the dco/no label Apr 23, 2015

@calavera

This comment has been minimized.

Copy link
Contributor

calavera commented Apr 24, 2015

code LGTM. Should we move it to docs review?

@jessfraz

This comment has been minimized.

Copy link
Contributor

jessfraz commented Apr 24, 2015

@icecrime

This comment has been minimized.

Copy link
Contributor

icecrime commented Apr 27, 2015

LGTM

@@ -319,6 +322,18 @@ for data and metadata:
--storage-opt dm.metadatadev=/dev/vdc \
--storage-opt dm.basesize=20G

# EXEC DRIVER OPTIONS

This comment has been minimized.

@moxiegirl

moxiegirl Apr 27, 2015

Contributor

# Examples heading with its Client subhead should come after this. "As of now and only viable" are assumed in context.

-->

EXEC DRIVER OPTIONS

Use the --exec-opt flags to specify options to the execdriver. The only
driver that accepts this flag is the native (libcontainer) driver. As a
result, you must also specify **-s=**native for this option to have effect. The following is the only native option:

native.cgroupdriver

Specifies the management of the container's cgroups. You can specify cgroupfs or systemd. If you specify systemd and it is not available, the system uses cgroupfs.

This comment has been minimized.

@jessfraz

jessfraz Apr 27, 2015

Contributor

will do!

Add docs for `--exec-opt` and setting `native.cgroupdriver`.
update man pages.
update bash completion.

Docker-DCO-1.1-Signed-off-by: Jessica Frazelle <jess@docker.com> (github: jfrazelle)

Docker-DCO-1.1-Signed-off-by: Jessie Frazelle <jess@docker.com> (github: jfrazelle)

@jessfraz jessfraz force-pushed the jessfraz:option-to-disable-systemd-cgroups branch from d03a110 to 07f0f61 Apr 28, 2015

@@ -452,6 +452,27 @@ https://linuxcontainers.org/) via the `lxc` execution driver, however, this is
not where the primary development of new functionality is taking place.
Add `-e lxc` to the daemon flags to use the `lxc` execution driver.

#### Exec driver options

This comment has been minimized.

@moxiegirl

moxiegirl Apr 28, 2015

Contributor

I've seen exec driver, exec-driver, and execdriver so I arbitrarily picked execdriver we should be consistent unless one is a proper name or a command. I've seen execdriver in other docker docs and the .go files use execdriver it seems. Anyway, I'd pick one (and I'll update my last comment). Finally, I changed the heading to be more specific --- no sense in getting folks hopes up.

Don't know if that capper sentence about the daemon is necessary. Your call.

-->

Options for the native execdriver

You can configure the native (libcontainer) execdriver using options specified
with the --exec-opt flag. All the flag's options have the native prefix. A
single native.cgroupdriver option is available.

The native.cgroupdriver option specifies the management of the container's cgroups. You can specify
cgroupfs or systemd. If you specify systemd and it is not available, the
system uses cgroupfs. By default, if no option is specified, the execdriver
first tries systemd and falls back to cgroupfs. This example sets the
execdriver to cgroupfs:

 $ sudo docker -d --exec-opt native.cgroupdriver=cgroupfs

Setting this option applies to all containers the daemon launches.

This comment has been minimized.

@jessfraz

jessfraz Apr 28, 2015

Contributor

sounds good to me! updated

option to configure cgroups
Signed-off-by: Jessica Frazelle <jess@docker.com>

@jessfraz jessfraz force-pushed the jessfraz:option-to-disable-systemd-cgroups branch from 07f0f61 to 2afcd10 Apr 28, 2015

@moxiegirl

This comment has been minimized.

Copy link
Contributor

moxiegirl commented Apr 28, 2015

LGTM

ping @docker/docs-owners

@jessfraz

This comment has been minimized.

Copy link
Contributor

jessfraz commented Apr 29, 2015

merge it merge it merge it

jessfraz pushed a commit that referenced this pull request Apr 29, 2015

Jessie Frazelle
Merge pull request #10427 from jfrazelle/option-to-disable-systemd-cg…
…roups

Option to configure cgroup manager (adds --exec-opt)

@jessfraz jessfraz merged commit 53bef64 into moby:master Apr 29, 2015

3 checks passed

docker/dco-signed All commits signed
Details
janky Jenkins build Docker-PRs 6893 has succeeded
Details
windows Jenkins build Windows-PRs 3858 has succeeded
Details

@jessfraz jessfraz deleted the jessfraz:option-to-disable-systemd-cgroups branch Apr 29, 2015

@grossws

This comment has been minimized.

Copy link
Contributor

grossws commented Sep 22, 2015

Why does --exec-opt native.cgroupdriver require -s=native (native storage driver)?

@b-a-t

This comment has been minimized.

Copy link

b-a-t commented Feb 11, 2016

Looks like information about need of -s=native is misleading and, actually not present in the current docs: daemon-storage-driver-option. What you need is native execdriver, but as that's the only one available ATM it is implied anyhow.
What possibly could be nice though if @jfrazelle adds use of the --exec-opt=native.systemd to the distributed docker.service. Or give somewhere a note why it's not there...

@djmattyg007

This comment has been minimized.

Copy link

djmattyg007 commented Mar 28, 2016

Is there any documentation available on how to make use of this in such a way that it makes the systemd-docker wrapper script unnecessary?

@b-a-t

This comment has been minimized.

Copy link

b-a-t commented Apr 7, 2016

I couldn't find any good sample, but, based on the manpage I'm starting my docker as:

 Main PID: 664 (docker)
   CGroup: /system.slice/docker.service
           └─664 /usr/bin/docker daemon -H fd:// --exec-opt native.cgroupdriver=systemd --storage-driver=overlay --graph='/docker' --icc=false --iptables=true

which seems, does the trick.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment