Repository navigation
Container and pod resource limits #168
Description
Activity
+1.
We have cpu and memory in the container manifest:
// Optional: Defaults to unlimited.
Memory intyaml:"memory,omitempty" json:"memory,omitempty"
// Optional: Defaults to unlimited.
CPU intyaml:"cpu,omitempty" json:"cpu,omitempty"However, AFAICT, we don't do anything with them. Besides, I think we want something more similar to lmctfy's API (request, limit, qos for each resource).
Another consideration: We could make it fairly easy to add new resources. Kubelet needs to understand each individual resource's characteristics, for isolation, QoS, overcommitment, etc. OTOH, the scheduler could deal with resources entirely abstractly. It could get resources and their capacities from the machines. Similarly, we'd need to make it possible to request abstract resources in the container/pod manifest.
What we described internally was that "common" resources like CPU, memory,
disk, etc were described as first-class things. Other resources are
handled essentially as opaque counters. E.g. a node says "I have 5
resources with ID 12345", a client says "I need 2 resources with ID 12345".
The scheduler maps them.On Fri, Jul 11, 2014 at 2:04 PM, bgrant0607 notifications@github.com
wrote:We have cpu and memory in the container manifest:
// Optional: Defaults to unlimited.
Memory int yaml:"memory,omitempty" json:"memory,omitempty"
// Optional: Defaults to unlimited.
CPU int yaml:"cpu,omitempty" json:"cpu,omitempty"However, AFAICT, we don't do anything with them. Besides, I think we want
something more similar to lmctfy's API (request, limit, qos for each
resource).Another consideration: We could make it fairly easy to add new resources.
Kubelet needs to understand each individual resource's characteristics, for
isolation, QoS, overcommitment, etc. OTOH, the scheduler could deal with
resources entirely abstractly. It could get resources and their capacities
from the machines. Similarly, we'd need to make it possible to request
abstract resources in the container/pod manifest.Reply to this email directly or view it on GitHub
#168 (comment)
.Consider that the resource types and units used for pod/container requests could also be used for describing how to subdivide cluster resources (see #442 ). For example, if team A is limited to using 10GB RAM at the cluster level, then team A can run 10 pods x 1GB RAM; or 2 pods x 5GB per pod; or some combination, etc.
+1 to all of this. Mesos has a very similar model, with the scheduler/allocator able to work with any custom resource, but the slave/containerizer needs to know enough details to map it to an isolator. This would also be the appropriate separation for requested resource vs. resource limits.
- modified the milestones: ,
on Sep 26, 2014 /cc @johnwilkes @davidopp @rjnagal @smarterclayton @brendandburns @thockin
The resource model doc has been created. We should align our API with it. v1beta3 leaves resource requests unchanged, though the
ResourceListtype was added in order to represent node capacity. We could either add the new fields in a backwards-compatible way, or replace the existingContainerMemoryandCPUfields in v1beta3 -- if we prefer to do the latter, we should add this issue to #1519.I propose that we add an optional
ResourceSpecstruct containing optionalRequestandLimitResourceListfields to bothPodSpecandContainer.52 remaining items
Thanks! Will container engine be using that image by default in the future?
Yes. That might happen as early as v1.4 on GKE.
- I think this can be closed.…On Wed, Dec 7, 2016 at 3:13 PM Timothy St. Clair ***@***.***> wrote: I think we should move to close this issue, the root topic has been addressed but there are multiple side-threads that are on this issue where I believe they would be better served on other issues. @vishh <https://github.com/vishh> thoughts? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#168 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AF8dbELMdlH4rE1zioGrUTXpVEu6Piwdks5rFxNLgaJpZM4CFzkL> .
- added a commit that references this issue
on Feb 6, 2018 Before we implement QoS tiers (#147), we need to support basic resource limits for containers and pods. All resource values should be integers.
For inspiration, see lmctfy:
https://github.com/google/lmctfy/blob/master/include/lmctfy.protoArguably we should start with pods first, to at least provide isolation between pods. However, that would require the ability to start Docker containers within cgroups. The support we need for individual containers already exists.
We should allow both minimum and maximum resource values to be provided, as lmctfy does. But let's not reuse lmctfy's limit and max_limit terminology. I like "requested" (amount scheduler will use for placement) and "limit" (hard limit beyond which the pod/container is throttled or killed).
Even without limit enforcement, the scheduler could use resource information for placement decisions.
Schedule a Pod as follows:
Name: nginx-kusc00101
Image: nginx
Node selector: disk=ssdapiVersion: v1
kind: Pod
metadata:
name: nginx-kusc00101
spec:
containers:
- name: nginx-kusc00101
image: nginx
nodeSelector:
disk: ssdkubectl create -f nginx-kusc00101.yaml
- added a commit that references this issue
on Jun 10, 2019 - added a commit that references this issue
on Jul 21, 2020 - added a commit that references this issue
on Jan 31, 2022 - added a commit that references this issue
on Jan 18, 2024
Before we implement QoS tiers (#147), we need to support basic resource limits for containers and pods. All resource values should be integers.
For inspiration, see lmctfy:
https://github.com/google/lmctfy/blob/master/include/lmctfy.proto
Arguably we should start with pods first, to at least provide isolation between pods. However, that would require the ability to start Docker containers within cgroups. The support we need for individual containers already exists.
We should allow both minimum and maximum resource values to be provided, as lmctfy does. But let's not reuse lmctfy's limit and max_limit terminology. I like "requested" (amount scheduler will use for placement) and "limit" (hard limit beyond which the pod/container is throttled or killed).
Even without limit enforcement, the scheduler could use resource information for placement decisions.