Downward API proposal for resources (cpu, memory) limits and requests #24051

aveshagarwal · 2016-04-08T17:36:20Z

Proposal to address #9473
This PR proposes three approaches to expose values of resource limits and requests as env vars and volumes.This proposal has details about merits and demerits of each approach, and I am looking for community feedback regarding which one (or may more than one) we would like to go with. Also would like to know if there is any other approach.

This change is

aveshagarwal · 2016-04-08T17:36:35Z

@kubernetes/rh-cluster-infra

aveshagarwal · 2016-04-08T17:38:59Z

For reference: #9473

pmorie · 2016-04-08T18:39:37Z

docs/design/downward_api_resources_limits_requests.md

+
+## Background
+
+Currently downward APIs (via environment variables and volume plugin) only support exposing a Pod's name, namespace, annotations, labels and its IP. This document explains the need and design to extend them to expose resources (e.g. cpu, memory, storage) limits and requests.


We just call it 'the downward API'

Please link to the main downward API doc on kubernetes.io

pmorie · 2016-04-08T20:34:56Z

@aveshagarwal What's your opinion about which option to support? What are the relative merits and drawbacks? A proposal should examine the different options, contrast them on the merits, and pick one. So far this is a good explanation of the options, but I think it could use more comparison/contrast/and opinion.

bgrant0607 · 2016-04-15T06:57:59Z

cc @justinsb @vishh @dchen1107

bgrant0607 · 2016-04-15T06:58:43Z

docs/design/downward_api_resources_limits_requests.md

+| ---- | ------------------- |
+| CPU_LIMIT | .spec.containers[?(@.name=="container-name")].resources.limits.cpu |
+| MEMORY_LIMIT | .spec.containers[?(@.name=="container-name")].resources.limits.memory |
+| STORAGE_LIMIT | .spec.containers[?(@.name=="container-name")].resources.limits.storage |


We don't have storage limits and requests.

vishh · 2016-04-15T21:28:32Z

docs/design/downward_api_resources_limits_requests.md

+
+#### Environment variables
+
+This table shows how selectors can be used for various requests and limits to be exposed as environment variables.


How would this work if we support dynamically updating resource limits in the future?
cc @bgrant0607

IMO, in general, if a user updates some property that's fed into an env var via downward API, then it should cause the container to restart. If the user doesn't want the container to restart, they can use volumes instead or the API.

But it's sort of moot right now.

@bgrant0607 Is it fair to assume that resource limits/requests can change often in the future, if we have vertical autoscaling? If yes, should we even allow exposing these values via env vars?

@vishh We need a way to set Java heap size and GOMAXPROCS on the command line. I'm more worried about making it possible to run common workloads now. I think we're far, far away from enabling vertical autosizing by default. Users who opt in will need to make other changes to their apps in order to adequately take advantage of it. Apps aren't built that way today.

aveshagarwal · 2016-04-19T05:01:17Z

I have updated this proposal with more details and addressed all comments except following 2: pmorie's cgroup output format and bgrant's Examples of how to use these to set the Java heap size and GOMAXPROCS would be helpful. I will incorporate them soon.

pmorie · 2016-05-20T02:13:00Z

docs/design/downward_api_resources_limits_requests.md

+| cpu_request | requests.cpu |
+| memory_request | requests.memory |
+
+Since environment variables are container scoped, the container name must


The container name must be specified because volume are pod scoped rather than container scoped.

yeah will be fixed in next update soon.

aveshagarwal · 2016-05-20T02:15:17Z

fixed tests.

pmorie · 2016-05-20T02:25:41Z

@aveshagarwal The information here is good. My suggestions are purely additive:

There needs to be a passage stating which approach we will implement and why. It would be fine with me to add it at the end, and keep from spending time discussing how to mutate the text of this proposal so we can update the PR.

I think other than that, this LGTM. Thanks a lot for the proposal.

aveshagarwal · 2016-05-20T02:58:34Z

@pmorie updated PR.

pmorie · 2016-05-20T03:10:55Z

@aveshagarwal LGTM

@bgrant0607 PTAL

bparees · 2016-05-20T03:25:04Z

As long as the format is consistent with the divisor, I think its okay. I would want @bparees to weigh in.

agree and also agree that we don't want units by default, i think there are lots of cases where the unit would make the value harder to use in a config.

pmorie · 2016-05-20T03:26:12Z

I do not want units period initially. You know the divisor you used; you know what the unit is.

k8s-bot · 2016-05-20T03:34:23Z

GCE e2e build/test passed for commit 0f3a8bd.

aveshagarwal · 2016-05-20T03:56:45Z

@bparees yeah there are no units in the output now in the proposal with the selected approach.

bgrant0607 · 2016-05-20T06:25:06Z

docs/design/downward_api_resources_limits_requests.md

+| memory_request | requests.memory |
+
+Volumes are pod scoped, the container name must be specified as part of
+`containerSpecFieldRef` with them.


resourceFieldRef

bgrant0607 · 2016-05-20T06:27:39Z

Thank you. One nit, but LGTM. Feel free to reapply lgtm if you fix the nit.

k8s-github-robot · 2016-05-20T10:00:00Z

Automatic merge from submit-queue

pmorie · 2016-05-20T12:08:28Z

@aveshagarwal, send a follow-up PR to fix that nit and I will tag it.
Thanks again.

On Friday, May 20, 2016, k8s-merge-robot notifications@github.com wrote:

Merged #24051 #24051.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#24051 (comment)

aveshagarwal · 2016-05-20T12:23:54Z

@pmorie #25962

…rces-limits-requests-implementation Automatic merge from submit-queue Downward API implementation for resources limits and requests This is an implementation of Downward API for resources limits and requests, and it works with environment variables and volume plugin. This is based on proposal #24051. This implementation follows API with magic keys approach as discussed in the proposal. @kubernetes/rh-cluster-infra  --- This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/24179)

UPSTREAM: 84561: FIPS changes for passing unit tests Origin-commit: 38755aa39d1c483aedd51524ca34963f671bcce9

UPSTREAM: 84561: FIPS changes for passing unit tests

googlebot added the cla: yes label Apr 8, 2016

k8s-github-robot assigned bgrant0607 Apr 8, 2016

k8s-github-robot added kind/design Categorizes issue or PR as related to design. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. release-note-label-needed labels Apr 8, 2016

aveshagarwal force-pushed the master-downward-api-resources-limits-requests branch from 3db2747 to 7561842 Compare April 8, 2016 18:10

pmorie reviewed Apr 8, 2016
View reviewed changes

aveshagarwal mentioned this pull request Apr 13, 2016

Downward API implementation for resources limits and requests #24179

Merged

bgrant0607 reviewed Apr 15, 2016
View reviewed changes

pmorie added the release-note-none Denotes a PR that doesn't merit a release note. label Apr 15, 2016

k8s-github-robot removed the release-note-label-needed label Apr 15, 2016

vishh reviewed Apr 15, 2016
View reviewed changes

aveshagarwal force-pushed the master-downward-api-resources-limits-requests branch from 7561842 to 82b9ccd Compare April 19, 2016 04:58

aveshagarwal force-pushed the master-downward-api-resources-limits-requests branch 3 times, most recently from 155706b to 1d6617d Compare April 19, 2016 13:43

pmorie reviewed May 20, 2016
View reviewed changes

aveshagarwal force-pushed the master-downward-api-resources-limits-requests branch from e150ecd to e8d4a3a Compare May 20, 2016 02:14

k8s-github-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels May 20, 2016

Downward API proposal for resources (cpu, memory) limits and requests

0f3a8bd

aveshagarwal force-pushed the master-downward-api-resources-limits-requests branch from e8d4a3a to 0f3a8bd Compare May 20, 2016 02:57

bgrant0607 reviewed May 20, 2016
View reviewed changes

bgrant0607 added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 20, 2016

k8s-github-robot merged commit 9e23c55 into kubernetes:master May 20, 2016

aveshagarwal deleted the master-downward-api-resources-limits-requests branch May 24, 2016 18:12

p0lyn0mial pushed a commit to p0lyn0mial/kubernetes that referenced this pull request Dec 20, 2019

Merge pull request kubernetes#24051 from rphillips/fixes/fips_tests

5c339c9

UPSTREAM: 84561: FIPS changes for passing unit tests

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Downward API proposal for resources (cpu, memory) limits and requests #24051

Downward API proposal for resources (cpu, memory) limits and requests #24051

aveshagarwal commented Apr 8, 2016 •

edited by mikedanese

aveshagarwal commented Apr 8, 2016

aveshagarwal commented Apr 8, 2016

pmorie Apr 8, 2016

bgrant0607 Apr 15, 2016

pmorie commented Apr 8, 2016

bgrant0607 commented Apr 15, 2016

bgrant0607 Apr 15, 2016

vishh Apr 15, 2016

bgrant0607 Apr 16, 2016

vishh Apr 18, 2016

bgrant0607 Apr 19, 2016

aveshagarwal commented Apr 19, 2016

pmorie May 20, 2016

aveshagarwal May 20, 2016

aveshagarwal commented May 20, 2016

pmorie commented May 20, 2016

aveshagarwal commented May 20, 2016

pmorie commented May 20, 2016

bparees commented May 20, 2016

pmorie commented May 20, 2016 •

edited

k8s-bot commented May 20, 2016

aveshagarwal commented May 20, 2016

bgrant0607 May 20, 2016

bgrant0607 commented May 20, 2016

k8s-github-robot commented May 20, 2016

pmorie commented May 20, 2016

aveshagarwal commented May 20, 2016


		## Background

		Currently downward APIs (via environment variables and volume plugin) only support exposing a Pod's name, namespace, annotations, labels and its IP. This document explains the need and design to extend them to expose resources (e.g. cpu, memory, storage) limits and requests.


		#### Environment variables

		This table shows how selectors can be used for various requests and limits to be exposed as environment variables.

Downward API proposal for resources (cpu, memory) limits and requests #24051

Downward API proposal for resources (cpu, memory) limits and requests #24051

Conversation

aveshagarwal commented Apr 8, 2016 • edited by mikedanese

aveshagarwal commented Apr 8, 2016

aveshagarwal commented Apr 8, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pmorie commented Apr 8, 2016

bgrant0607 commented Apr 15, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aveshagarwal commented Apr 19, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aveshagarwal commented May 20, 2016

pmorie commented May 20, 2016

aveshagarwal commented May 20, 2016

pmorie commented May 20, 2016

bparees commented May 20, 2016

pmorie commented May 20, 2016 • edited

k8s-bot commented May 20, 2016

aveshagarwal commented May 20, 2016

Choose a reason for hiding this comment

bgrant0607 commented May 20, 2016

k8s-github-robot commented May 20, 2016

pmorie commented May 20, 2016

aveshagarwal commented May 20, 2016

aveshagarwal commented Apr 8, 2016 •

edited by mikedanese

pmorie commented May 20, 2016 •

edited