This repository has been archived by the owner on Dec 5, 2017. It is now read-only.
restrict pods to the resource constraints declared in their manifest #68
Assignees
Comments
|
kubelet doesn't currently offer any public API hooks for interacting with newly launched containers: ... although the kubelet does publish events about them, referencing such CT's by handles generated in |
|
xref moby/moby#6791 |
|
custom cgroup parents have landed in docker: |
This was referenced May 28, 2015
|
@jdef k8s now supports setting the parent cgroup for the kublet. Does this open the door to apply resource constraints? |
|
Sure does. It's on our todo list. On Thu, Jul 2, 2015 at 1:43 PM, sushantk notifications@github.com wrote:
|
|
There are no pod resource constraints right now in k8s, only by container. Hence, we can – maybe as a first step – easily sum up the values and put the sum into the |
sttts
added a commit
to mesosphere-backup/kubernetes
that referenced
this issue
Jul 30, 2015
This patch - set limits (0.25 cpu, 64 MB) on containers which are not limited in pod spec (these are also passed to the kubelet such that it uses them for the docker run limits) - sums up the container resource limits for cpu and memory inside a pod, - compares the sums to the offered resources - puts the sums into the Mesos TaskInfo such that Mesos does the accounting for the pod. - parses the static pod spec and adds up the resources - sets the executor resources to 0.25 cpu, 64 MB plus the static pod resources - sets the cgroups in the kubelet for system containers, resource containers and docker to the one of the executor that Mesos assigned - adds scheduler parameters --default-container-cpu-limit and --default-container-mem-limit. The containers themselves are resource limited the Docker resource limit which the kubelet applies when launching them. Fixes mesosphere/kubernetes-mesos#68 and mesosphere/kubernetes-mesos#304
|
resolved by kubernetes/kubernetes#11230 |
sttts
added a commit
to mesosphere-backup/kubernetes
that referenced
this issue
Jul 31, 2015
This patch - set limits (0.25 cpu, 64 MB) on containers which are not limited in pod spec (these are also passed to the kubelet such that it uses them for the docker run limits) - sums up the container resource limits for cpu and memory inside a pod, - compares the sums to the offered resources - puts the sums into the Mesos TaskInfo such that Mesos does the accounting for the pod. - parses the static pod spec and adds up the resources - sets the executor resources to 0.25 cpu, 64 MB plus the static pod resources - sets the cgroups in the kubelet for system containers, resource containers and docker to the one of the executor that Mesos assigned - adds scheduler parameters --default-container-cpu-limit and --default-container-mem-limit. The containers themselves are resource limited the Docker resource limit which the kubelet applies when launching them. Fixes mesosphere/kubernetes-mesos#68 and mesosphere/kubernetes-mesos#304
sttts
added a commit
to mesosphere-backup/kubernetes
that referenced
this issue
Aug 1, 2015
This patch - set limits (0.25 cpu, 64 MB) on containers which are not limited in pod spec (these are also passed to the kubelet such that it uses them for the docker run limits) - sums up the container resource limits for cpu and memory inside a pod, - compares the sums to the offered resources - puts the sums into the Mesos TaskInfo such that Mesos does the accounting for the pod. - parses the static pod spec and adds up the resources - sets the executor resources to 0.25 cpu, 64 MB plus the static pod resources - sets the cgroups in the kubelet for system containers, resource containers and docker to the one of the executor that Mesos assigned - adds scheduler parameters --default-container-cpu-limit and --default-container-mem-limit. The containers themselves are resource limited the Docker resource limit which the kubelet applies when launching them. Fixes mesosphere/kubernetes-mesos#68 and mesosphere/kubernetes-mesos#304
sttts
added a commit
to mesosphere-backup/kubernetes
that referenced
this issue
Aug 13, 2015
This patch - set limits (0.25 cpu, 64 MB) on containers which are not limited in pod spec (these are also passed to the kubelet such that it uses them for the docker run limits) - sums up the container resource limits for cpu and memory inside a pod, - compares the sums to the offered resources - puts the sums into the Mesos TaskInfo such that Mesos does the accounting for the pod. - parses the static pod spec and adds up the resources - sets the executor resources to 0.25 cpu, 64 MB plus the static pod resources - sets the cgroups in the kubelet for system containers, resource containers and docker to the one of the executor that Mesos assigned - adds scheduler parameters --default-container-cpu-limit and --default-container-mem-limit. The containers themselves are resource limited the Docker resource limit which the kubelet applies when launching them. Fixes mesosphere/kubernetes-mesos#68 and mesosphere/kubernetes-mesos#304
sttts
added a commit
to mesosphere-backup/kubernetes
that referenced
this issue
Sep 4, 2015
This patch - set limits (0.25 cpu, 64 MB) on containers which are not limited in pod spec (these are also passed to the kubelet such that it uses them for the docker run limits) - sums up the container resource limits for cpu and memory inside a pod, - compares the sums to the offered resources - puts the sums into the Mesos TaskInfo such that Mesos does the accounting for the pod. - parses the static pod spec and adds up the resources - sets the executor resources to 0.25 cpu, 64 MB plus the static pod resources - sets the cgroups in the kubelet for system containers, resource containers and docker to the one of the executor that Mesos assigned - adds scheduler parameters --default-container-cpu-limit and --default-container-mem-limit. The containers themselves are resource limited the Docker resource limit which the kubelet applies when launching them. Fixes mesosphere/kubernetes-mesos#68 and mesosphere/kubernetes-mesos#304
sttts
added a commit
to mesosphere-backup/kubernetes
that referenced
this issue
Oct 8, 2015
This patch - set limits (0.25 cpu, 64 MB) on containers which are not limited in pod spec (these are also passed to the kubelet such that it uses them for the docker run limits) - sums up the container resource limits for cpu and memory inside a pod, - compares the sums to the offered resources - puts the sums into the Mesos TaskInfo such that Mesos does the accounting for the pod. - parses the static pod spec and adds up the resources - sets the executor resources to 0.25 cpu, 64 MB plus the static pod resources - sets the cgroups in the kubelet for system containers, resource containers and docker to the one of the executor that Mesos assigned - adds scheduler parameters --default-container-cpu-limit and --default-container-mem-limit. The containers themselves are resource limited the Docker resource limit which the kubelet applies when launching them. Fixes mesosphere/kubernetes-mesos#68 and mesosphere/kubernetes-mesos#304
deads2k
pushed a commit
to kubernetes-retired/kube-mesos-framework
that referenced
this issue
Sep 29, 2016
This patch - set limits (0.25 cpu, 64 MB) on containers which are not limited in pod spec (these are also passed to the kubelet such that it uses them for the docker run limits) - sums up the container resource limits for cpu and memory inside a pod, - compares the sums to the offered resources - puts the sums into the Mesos TaskInfo such that Mesos does the accounting for the pod. - parses the static pod spec and adds up the resources - sets the executor resources to 0.25 cpu, 64 MB plus the static pod resources - sets the cgroups in the kubelet for system containers, resource containers and docker to the one of the executor that Mesos assigned - adds scheduler parameters --default-container-cpu-limit and --default-container-mem-limit. The containers themselves are resource limited the Docker resource limit which the kubelet applies when launching them. Fixes mesosphere/kubernetes-mesos#68 and mesosphere/kubernetes-mesos#304
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Currently pods can declare cpu and memory constraints but we don't honor them. We should.
The current design maps a single mesos task to a k8s pod, which in reality represents multiple containers. One approach would be to place all pods managed by an executor into its container, growing and shrinking the resource constraints of the executor container on the fly to accommodate the pods that it's in charge of managing. That implies that we can actually control placement of the docker containers (well, their cgroup placement). Currently this is only possible by manually moving the processes from their default docker-determined cgroup to that of the executor.
There is a docker proposal on the table to allow for customizable cgroup placement: moby/moby#8551 (related prototype tooling here: https://github.com/ibuildthecloud/systemd-docker).
Assuming that we could convince docker to honor custom cgroup placement (at container launch time), then we'd also need to convince/hack the kubelet to apply custom placement rules.
The text was updated successfully, but these errors were encountered: