Document new CPU scheduling flags

[mdlinville]

Describe the proposed changes

Documented new CPU-related flags and tried to pull them together into a coherent topic

Unreleased project version

Engine 1.13

Related issue

Fixes #583

Related issue or PR in another project

moby/moby#23430 (review)

Please take a look

@justincormack @thaJeztah @erikstmartin

[mdlinville]

PTAL @cpuguy83

[cpuguy83]

The options Docker provides are really more about being able to configure the resources that the RT scheduler should allot for it, but does not set the scheduler itself.

[mdlinville]

@mlaventure PTAL, Brian nominated you. :)

[mdlinville]

I realized that this needs to wait for #334 is merged into master, master is merged into vnext-engine, and this stuff needs to go into that topic instead of this new topic I've created. Just a note for me, no action for reviewers.

[mdlinville]

OK, this work has now been refactored on top of #334. No extra review is needed because no facts were changed.

[vincentwoo]

I think this documentation should include more information about realtime scheduling and links to places to find out how to enable it for various distros. Docker introduced a pretty useful new feature without a great plan for users on how to actually get it running in prod. For instance, if you want to use --cpus, do you need to cap-add sys_nice to all your containers?

[mdlinville]

@cpuguy83 do you have any info to help answer @vincentwoo's question?

[vincentwoo]

There's a lot more information in the description/comments of the originating PR (moby/moby#23430) but no information in the documentation about how much of this is still needed, nor what their base assumptions are. I think it likely that many users (including myself) have no idea what "real time scheduling" is.

[mdlinville]

I'm not sure how much about real-time scheduling as a computer science concept or a Linux kernel concept that Docker should be covering. I think in general there is an idea that "If you are not sure whether you need RT, you don't". I think we already link to the kernel references, and I'm not sure how far we should be going. I'll wait for @cpuguy83 or others to weigh in.

[vincentwoo]

I completely disagree. The docs present the new cpu options as just-so. As in, simply pass these flags to your containers and they will be throttled accordingly. --cpus=1.5 seems like a great thing, and very easy for anyone to understand! The layout of the documentation makes it seem like you can just turn it on for any container because it appears under "Configure the default scheduler".

Then, the docs go on to describe a bunch of weird options, like realtime, or sys_nice, with no mention of why you might want to turn on those options. Is the reality that you actually need all of that stuff to use --cpus? Because from reading the other PR it seems like the answer is definitely, yes.

[vincentwoo]

See also moby/moby#27958, which introduced --cpus on top of the previous PR. What's also very weird is all this instruction around "If you use Docker 1.13 or higher, use --cpus instead". ALL of these flags were introduced in 1.13, no?

[cpuguy83]

@vincentwoo I think you are mixing things up here.
--cpus configures CFS, and it works just as is.

If someone is using realtime workloads --cpus is not for them. The docs for --cpus is specifically under the CFS scheduler options.

These options seem well separated in the docs.

[mdlinville]

Maybe it would help to see this content in context: https://docs.docker.com/engine/admin/resource_constraints/#/cpu

[vincentwoo]

Oh jesus I finally understand. I apologize for my inability to understand the docs - did not realize that different PRs were talking about runtime vs cfs periods, ugh. apologies to all involved