Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve resource quota doc. #11487

Merged
merged 1 commit into from
Jul 20, 2015
Merged

Improve resource quota doc. #11487

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/admin/etcd.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ Documentation for other releases can be found at
<!-- END STRIP_FOR_RELEASE -->

<!-- END MUNGE: UNVERSIONED_WARNING -->

# etcd

[etcd](https://coreos.com/etcd/docs/2.0.12/) is a highly-available key value
Expand Down
89 changes: 62 additions & 27 deletions docs/admin/resource-quota.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,35 @@ Documentation for other releases can be found at

<!-- END MUNGE: UNVERSIONED_WARNING -->

# Administering Resource Quotas

Kubernetes can limit both the number of objects created in a namespace, and the
total amount of resources requested by pods in a namespace. This facilitates
sharing of a single Kubernetes cluster by several teams or tenants, each in
a namespace.
# Resource Quotas

When several users or teams share a cluster with a fixed number of nodes,
there is a concern that one team could use more than its fair share of resources.

Resource quotas are a tool for administrators to address this concern. Resource quotas
work like this:
- Different teams work in different namespaces. Currently this is voluntary, but
support for making this mandatory via ACLs is planned.
- Users put [compute resource limits](../user-guide/compute-resources.md) on their pods.
- The administrator creates a Resource Quota for each namespace.
- If users exceed the limits specified in the Resource Quota in a namespace, they are blocked
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This implies that they are allowed to exceed, and thereafter are blocked. But more generally, this is really vague. This is in the admin section so we should explain in reasonable detail how this works. The pod gets created but the scheduler is not allowed to schedule it? If so, what prevents the scheduler from scheduling it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It 403 FORBIDDEN and not created. Updated doc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would clarify this as "If a new Pod would exceed the limits specified in the Resource Quota in a namespace, then creation of the Pod is blocked."

from creating more pods in that namespace. The POST of the pod will fail with HTTP status
code `403 FORBIDDEN`.
- If quota is enabled in a namespace and the user does not specify limits on the pod for each
of the resources for which quota is enabled, then the POST of the pod will fail with HTTP
status code `403 FORBIDDEN`. Hint: Use the LimitRange admission controller to force default
values of limits before the quota is checked to avoid this problem.

Examples of policies that could be created using namespaces and quotas are:
- In a cluster with a capacity of 32 GiB RAM, and 16 cores, let team A use 20 Gib and 10 cores,
let B use 10GiB and 4 cores, and hold 2GiB and 2 cores in reserve for future allocation.
- Limit the "testing" namespace to using 1 core and 1GiB RAM. Let the "production" namespace
use any amount.

In the case where the total capacity of the cell is less than the sum of the quotas of the namespaces,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cell -> cluster

there may be contention for resources. This is handled on a first-come-first-served basis.

Neither contention nor changes to quota will affect already-running pods.

## Enabling Resource Quota

Expand All @@ -48,7 +71,24 @@ Resource Quota is enforced in a particular namespace when there is a
`ResourceQuota` object in that namespace. There should be at most one
`ResourceQuota` object in a namespace.

See [ResourceQuota design doc](../design/admission_control_resource_quota.md) for more information.
## Compute Resource Quota

The total sum of [compute resources](../user-guide/compute-resources.md) requested by pods
in a namespace can be limited. The following compute resource types are supported:

| ResourceName | Description |
| ------------ | ----------- |
| cpu | Total cpu limits of containers |
| memory | Total memory limits of containers
| `example.com/customresource` | Total of `resources.limits."example.com/customresource"` of containers |

For example, `cpu` quota sums up the `resources.limits.cpu` fields of every
container of every pod in the namespace, and enforces a maximum on that sum.

Any resource that is not part of core Kubernetes must follow the resource naming convention prescribed by Kubernetes.

This means the resource must have a fully-qualified name (i.e. mycompany.org/shinynewresource)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd remove the paragraph break between this sentence and the previous one.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deleted this line cause it turns out that arbitrary resources are not supported by current code.



## Object Count Quota

Expand All @@ -67,23 +107,9 @@ are supported:
For example, `pods` quota counts and enforces a maximum on the number of `pods`
created in a single namespace.

## Compute Resource Quota

The total number of objects of a given type can be restricted. The following types
are supported:

| ResourceName | Description |
| ------------ | ----------- |
| cpu | Total cpu limits of containers |
| memory | Total memory usage limits of containers
| `example.com/customresource` | Total of `resources.limits."example.com/customresource"` of containers |

For example, `cpu` quota sums up the `resources.limits.cpu` fields of every
container of every pod in the namespace, and enforces a maximum on that sum.

Any resource that is not part of core Kubernetes must follow the resource naming convention prescribed by Kubernetes.

This means the resource must have a fully-qualified name (i.e. mycompany.org/shinynewresource)
You might want to set a pods quota on a namespace
to avoid the case where a user creates many small pods and exhausts the cluster's
supply of Pod IPs.

## Viewing and Setting Quotas

Expand Down Expand Up @@ -128,22 +154,31 @@ services 3 5

## Quota and Cluster Capacity

Resource Quota objects are independent of the Cluster Capacity. They are
expressed in absolute units.
Resource Quota objects are independent of the Cluster Capacity. They are
expressed in absolute units. So, if you add nodes to your cluster, this does *not*
automatically give each namespace the ability to consume more resources.

Sometimes more complex policies may be desired, such as:
- proportionally divide total cluster resources among several teams.
- allow each tenant to grow resource usage as needed, but have a generous
limit to prevent accidental resource exhaustion.
- detect demand from one namespace, add nodes, and increase quota.

Such policies could be implemented using ResourceQuota as a building-block, by
writing a 'controller' which watches the quota usage and adjusts the quota
hard limits of each namespace.
hard limits of each namespace according to other signals.

Note that resource quota divides up aggregate cluster resources, but it creates no
restrictions around nodes: pods from several namespaces may run on the same node.

## Example

See a [detailed example for how to use resource quota](../user-guide/resourcequota/).

## Read More

See [ResourceQuota design doc](../design/admission_control_resource_quota.md) for more information.


<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/admin/resource-quota.md?pixel)]()
Expand Down