Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Resource Metrics API proposal #24253

Merged
merged 1 commit into from
May 5, 2016
Merged

Resource Metrics API proposal #24253

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
181 changes: 181 additions & 0 deletions docs/proposals/resource-metrics-api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->

<!-- BEGIN STRIP_FOR_RELEASE -->

<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
width="25" height="25">
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
width="25" height="25">
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
width="25" height="25">
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
width="25" height="25">
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
width="25" height="25">

<h2>PLEASE NOTE: This document applies to the HEAD of the source tree</h2>

If you are using a released version of Kubernetes, you should
refer to the docs that go with that version.

Documentation for other releases can be found at
[releases.k8s.io](http://releases.k8s.io).
</strong>
--

<!-- END STRIP_FOR_RELEASE -->

<!-- END MUNGE: UNVERSIONED_WARNING -->

# Resource Metrics API

*This proposal is based on and supersedes [compute-resource-metrics-api.md](compute-resource-metrics-api.md).*
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, remove the old proposal?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do it once we will merge this one.


This document describes API part of MVP version of Resource Metrics API effort in Kubernetes.
Once the agreement will be made the document will be extended to also cover implementation details.
The shape of the effort may be also a subject of changes once we will have more well-defined use cases.

## Goal

The goal for the effort is to provide resource usage metrics for pods and nodes through the API server.
This will be a stable, versioned API which core Kubernetes components can rely on.
In the first version only the well-defined use cases will be handled,
although the API should be easily extensible for potential future use cases.

## Main use cases

This section describes well-defined use cases which should be handled in the first version.
Use cases which are not listed below are out of the scope of MVP version of Resource Metrics API.

#### Horizontal Pod Autoscaler

HPA uses the latest value of cpu usage as an average aggregated across 1 minute
(the window may change in the future). The data for a given set of pods
(defined either by pod list or label selector) should be accesible in one request
due to performance issues.

#### Scheduler

Scheduler in order to schedule best-effort pods requires node level resource usage metrics
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add some comment that this is something the scheduler will do in the future, but not yet.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

as an average aggreated across 1 minute (the window may change in the future).
The metrics should be available for all resources supported in the scheduler.
Currently the scheduler does not need this information, because it schedules best-effort pods
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I plan to add a new node condition as part of #18724 that could be used as a signal to the scheduler for it to not place additional best effort pods on a node. If a machine's available memory falls below the specified threshold, the node will report a MemoryPressure node condition of true, and it will block all incoming best effort pods from running in kubelet admission flow.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation.

without considering node usage. But having the metrics available in the API server is a blocker
for adding the ability to take node usage into account when scheduling best-effort pods.

## Other considered use cases

This section describes the other considered use cases and explains why they are out
of the scope of the MVP version.

#### Custom metrics in HPA

HPA requires the latest value of application level metrics.

The design of the pipeline for collecting application level metrics should
be revisited and it's not clear whether application level metrics should be
available in API server so the use case initially won't be supported.

#### Ubernetes
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rewrite this whole section as "Ubernetes might want to consider cluster-level usage (in addition to cluster-level request) of running pods when choosing where to schedule new pods. Although Ubernetes is still in design, we expect the metrics API described here to be sufficient. Cluster-level usage can be obtained by summing over usage of all nodes in the cluster."

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


Ubernetes might want to consider cluster-level usage (in addition to cluster-level request)
of running pods when choosing where to schedule new pods. Although Ubernetes is still in design,
we expect the metrics API described here to be sufficient. Cluster-level usage can be
obtained by summing over usage of all nodes in the cluster.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpicking: for cluster-wide rank statistics such as 95th and 99th percentile, and people will want them, you can't compute those from pre-aggregated data. Think of the case where you look at the max over a window of one hour and a large RC/RS moves from one set of machines to another: it will be counted twice. (You also need to decide whether it's cluster-wide, but computed at the container, pod or node level... most likely people will want the last two.) One way to approximate this would be computing something like Q-digests or t-digests from data streamed from PodMetrics (not node data). Unless you only want to support instantaneous usage, the last sentence is misleading.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. In the future we may want to introduce also aggregations on a cluster level though as for now there is no use case for it. I added it to Further improvements section.


#### kubectl top

This feature is not yet specified/implemented although it seems reasonable to provide users information
about resource usage on pod/node level.

Since this feature has not been fully specified yet it will be not supported initally in the API although
it will be probably possible to provide a reasonable implementation of the feature anyway.

#### Kubernetes dashboard

[Kubernetes dashboard](https://github.com/kubernetes/dashboard) in order to draw graphs requires resource usage
in timeseries format from relatively long period of time. The aggreations should be also possible on various levels
including replication controllers, deployments, services, etc.

Since the use case is complicated it will not be supported initally in the API and they will query Heapster
directly using some custom API there.

## Proposed API

Initially the metrics API will be in a separate [API group](api-group.md) called ```metrics```.
Later if we decided to have Node and Pod in different API groups also
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would you have them in different API groups instead of using the paths you listed at the end of the doc?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is explained by @erictune in #18824 (comment)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Yeah, I don't think it's necessary to put them in separate groups now.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what I meant here.

NodeMetrics and PodMetrics should be in different API groups.

#### Schema

The proposed schema is as follow. Each top-level object has `TypeMeta` and `ObjectMeta` fields
to be compatible with Kubernetes API standards.

```go
type NodeMetrics struct {
unversioned.TypeMeta
ObjectMeta

// The following fields define time interval from which metrics were
// collected in the following format [Timestamp-Window, Timestamp].
Timestamp unversioned.Time
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not wrap these fields in a sub-structure that makes it easy to add more windows in the future?

type Sample struct {
Timestamp unversioned.Time
Window    unversioned.Duration
Usage map[ResourceType]Resource.Quantity
}

type NodeMetrics struct {
   unversioned.TypeMeta
   ObjectMeta
   Samples []Sample
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current proposal is to have window as a parameter so by definition you won't be able to return metrics from multiple windows in one response.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh makes sense.. In that case, what is the purpose of including this field?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make response self-descriptive. I won't fight for having it though I found it usable. See also #24253 (comment)

Window unversioned.Duration
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is window free to vary from response to response for the same node or the same pod, over time?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default the windows will be set to 1minute. In the future once we will add support for window parameter this will have various values depending on the request. This field is somehow redundant and I'm ok with removing it though I think it's better user experience when the response is self-descriptive especially in the default case. See also #24253 (comment)


// The memory usage is the memory working set.
Usage v1.ResourceList
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where will you define exactly how Usage is measured for each resource? For example, is memory usage working-set or something else? It seems like this definition should be part of the API (at least as comments).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As for now it's total memory in use. I agree it should be documented.

Also I think it should be consistent with what Kubelet uses right now (or at least is planning to use in the future) to account resource usage. Especially user experience should be as follow:

  • a user specifies memory request for a container to 200MB
  • in Metrics API the container usage is 190MB
  • the container started using a little bit more memory
  • Kubelet under high memory pressure decided to kill the container due to OOM

@dchen1107, @timstclair could you please explain which kind of memory usage Kubelet uses?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what you mean by "which kind of memory usage Kubelet uses". Admission is based on request, and currently we don't do any eviction (the proposal doesn't define what memory usage is). The stats API has multiple fields describing memory.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For eviction, the proposal is using memory.available which I recently added to the stats API.

It's defined as memory limit - workingSetBytes , so for purposes of node eviction, its memory.available = memory.capacity - workingSetBytes

We will block best effort pods in admission if the eviction threshold for memory.available has fallen below a operator specified value, we will update a node condition like MemoryPressure when the node is in that state.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, then Usage should be workingSetBytes? If we all agree on that, let's document it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The stats objects are more complete. Why not just use them instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@timstclair thanks for the info. What I meant here is exactly what is not defined in the proposal, but as @derekwaynecarr wrote it's working set.

@erictune it seems we all agree here, so I'll add an appropriate comment.

@Q-Lee I agree that we can provide more info. The question is whether we want to do it. As for now it's out of the scope of the MVP and all more specific metrics are available in Heapster/InfluxDB

}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If pod metrics is an aggregate of all of its container metrics, Is it possible to describe similarly what are the different constituents of a node metrics? Or how is computed just as a whole or by taking aggregation of various different metrics, if the later what are those?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It definitely has to to be clarified. The question is what does scheduler expect to get. @davidopp for clarifying the scheduler needs.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually it might make sense to have both, though we don't have a use case for both yet.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC the question is: is there any difference between the sum over all ContainerMetrics on a node, and the value reported by NodeMetrics? And if so, should we expose the constituents of that difference explicitly? (I actually don't know the answer to the question, if that is the question. But I assume docker daemon and kubelet usage count in NodeMetrics but aren't represented in ContainerMetrics.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@davidopp yes, you understand the problem correctly. What we are exposing currently as a node usage is containers usage + system overhead + docker + kubelet + kube-proxy. The question is whether we should change it to just containers usage, which might be in line with what scheduler expects, especially in context we want to support Allocatable feature.

@dchen1107 any thoughts?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in a separate email, @piosz asked:

How should we account CPU usage on node level?

AFAIU the code, currently in stats summary API we export just the cpu usage of the whole node. > On the other hand from scheduler perspective it might make sense to use the sum of cpu usage of all pods running on the machine especially assuming we will start supporting Allocatable feature.

Technically as for now there is no difference. There is aggregating logic already implemented in Heapster so adding yet another metric to aggregate is a 2-lines change.

If the scheduler is just using a "least loaded" heuristic for spreading work, then whole node cpu usage is sufficient. It is just a rough heuristic, so we should do the simplest thing, I think.

Regarding unaccounted cpu usage: If some node has a bunch of CPU usage that is not accounted to any container, (e.g. kernel threads or processes in root cpu cgroup), we still want to avoid that node.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to what @erictune said. NodeMetrics should report overall for node, including stuff that isn't in ContainerMetrics.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM, thanks for the clarification.

type PodMetrics struct {
unversioned.TypeMeta
ObjectMeta

// The following fields define time interval from which metrics were
// collected in the following format [Timestamp-Window, Timestamp].
Timestamp unversioned.Time
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think putting Timestamp on the higher level objects is too constraining, and it should be attached to individual metrics. Reasons include:

  • We may collect metrics at varying intervals (e.g. more expensive metrics are collected less frequently). We already do this with some disk metrics and network metrics.
  • Containers have independent scrapers which scrape at different times.
  • If CustomMetrics were to make it into this API, they would certainly need different timestamps

Copy link
Member Author

@piosz piosz Apr 14, 2016 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I strongly disagree with having Timestamp per metric. I can imagine it might make sense to have it per in ContainerMetrics (example needed) but in general I think we should have a consistent information about pod from some point in time. Especially if we want to make some aggregation on pod level (which HPA does) it totally doesn't make sense to have different timestamps per container.

Containers have independent scrapers which scrape at different times.

IMO this is a bug not a feature.

Also the goal for this effort is to have some basic overview of resource usage. Not every metric has to included here, so hopefully we are able to align timestamps for the metrics supported here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HPA does not aggregate across metrics right? As long as metrics are aligned for every resource, why does it matter to HPA?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we decide to have a top-level timestamp, we should at least make sure it's well-defined. Is it:

  • The time the response was generated at
  • The timestamp of the newest metric
  • The timestamp of the oldest metric
  • The average timestamp

Containers have independent scrapers which scrape at different times.

IMO this is a bug not a feature.

It's to avoid usage spikes at each housekeeping interval, the same reason Heapster polls each node at a different time.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with Tim, at least you should add a comment explaining what timestamp and window mean.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added comment.

It's to avoid usage spikes at each housekeeping interval, the same reason Heapster polls each node at a different time.

I know about it and I'm fine with it, although it's rather implementation imperfection and metrics inaccuracy that the feature itself.

Window unversioned.Duration
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this field indicate? It sounds like it's specified as a request parameter, so why have it on every metric object rather than with the top-level response (or not at all)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the first version there won't be a support for the parameter. It will be added if we decide we need to support various window sizes. PodMetrics is actually a top level resource (this will be returned for request to /namespaces/{namespace}/pods/{pod}). I'm fine with not having this field however I think it's better user experience to have self-descriptive response especially when window param is not supported.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be implying that all containers of the same Pod are measured at the same Timestamp and Window? Is that correct?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. If we want to make any reasonable aggregation on pod level (which for example HPA does) it's hard requirement.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree. Document in the comments of the type.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


// Metrics for all containers are collected within the same time window.
Containers []ContainerMetrics
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Containers sounds confusing, how about containerUsageList or containerMetricsList or may be something else.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@erictune WDYT?

}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to have a collective usage for pod too.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this would make users' life easier. The question is how compatible with pod definition we want to be. Especially you don't request resource per pod but rather per container. @erictune WDYT?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would you do usage per pod? Usage per pod is always sum of usage per container.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What @aveshagarwal means is to have it precomputed for easier usage, though I'm rather against here.

Copy link
Member

@erictune erictune Apr 25, 2016 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we later add volume resources, then it pod metrics would be sum of container and volume metrics for the pod. So, if we plan to add volume later, it might be better to add Pod metrics here, so that when we add volume accounting, users who use Pod metrics immediately see this new usage. OTOH, it might be breaking for users to change the Pod accounting. So maybe better to not add it until we are sure what it will contain.

Also, if we later add 95th as an aggregation, the user cannot compute this herself. The kubelet has to do that. So, if 95th, or other non-user-computable aggregations are needed, then I agree we will need a PodMetrics. However, I am fine with leaving those out of the MVP until we have a stronger use case.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean by adding "95th as an aggregation"? I'm unsure why the users can't compute that. And if we do want to start offering aggregated stats, then we should serve a distribution (e.g., f(x)) rather than a specific value (e.g., f(0.95)).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is one of possibilities to to provide a distribution, however this is out of the scope of MVP so let's do not discuss it right now.


type ContainerMetrics struct {
// Container name corresponding to the one from v1.Pod.Spec.Containers.
Name string
// The memory usage is the memory working set.
Usage v1.ResourceList
Copy link
Member

@erictune erictune Apr 25, 2016 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What keys can I expect to find in this map, or are there no guarantees?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either 'everything we have' (if we collected only memory, we will return only memory) or 'all or nothing' (if we collected only memory, we will respond with 404). I don't have strong opinion which one is better. WDYT?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure.

}
```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar is already part of pkg/.../types.go:

type ResourceName string
const (
        // CPU, in cores. (500m = .5 cores)
        ResourceCPU ResourceName = "cpu"
        // Memory, in bytes. (500Gi = 500GiB = 500 * 1024 * 1024 * 1024)
        ResourceMemory ResourceName = "memory"
        // Volume size, in bytes (e,g. 5Gi = 5GiB = 5 * 1024 * 1024 * 1024)
        ResourceStorage ResourceName = "storage"
        // Number of Pods that may be running on this Node: see ResourcePods
)

Copy link
Member Author

@piosz piosz Apr 14, 2016 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know and I also like this type, however someone (@davidopp, @fgrzadkowski, @mwielgus?) has objections that maybe some day we would like to have a different set of keys here. Is this still current?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was just pointing to that similar constants already exist, and you could reuse them.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reusing just constants doesn't make sense here. If we decide to reuse something we should reuse ResourceList.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We generally do not reuse constants - we type them and keep them distinct
for better API descriptiveness.

On Apr 14, 2016, at 2:35 PM, Avesh Agarwal notifications@github.com wrote:

In docs/proposals/resource-metrics-api.md
#24253 (comment):

  • Timestamp unversioned.Time
  • Window unversioned.Duration
  • Containers []ContainerMetrics
    +}

+type ContainerMetrics struct {

  • Name string
  • Usage map[ResourceType]Resource.Quantity
    +}

+type ResourceType string
+const CpuResource ResourceType = "cpu"
+const MemoryResource ResourceType = "memory"
+const StorageResource ResourceType = "storage"
+```

I was just pointing to that similar constants already exist, and you could
reuse them.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/kubernetes/kubernetes/pull/24253/files/9f124139f4c4e9fe149075b1bef55824c7a2092f#r59768572


By default `Usage` is the mean from samples collected within the returned time window.
The default time window is 1 minute.

#### Endpoints

All endpoints are GET endpoints, rooted at `/apis/metrics/v1alpha1/`.
There won't be support for the other REST methods.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will watch be ever supported? Polling requires clients to poll with a period lesser than the minimum resolution in-order to not miss metrics.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Support for watch is definitely a good idea and I think eventually we will add it, although it doesn't add much value (metrics will change very frequently) so there won't be a support for the watch in the first version.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. Would it make sense to mention the future plans in the proposal?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely yes. I'll add a paragraph.


The list of supported endpoints:
- `/nodes` - all node metrics; type `[]NodeMetrics`
- `/nodes/{node}` - metrics for a specified node; type `NodeMetrics`
- `/namespaces/{namespace}/pods` - all pod metrics within namespace with support for `all-namespaces`; type `[]PodMetrics`
- `/namespaces/{namespace}/pods/{pod}` - metrics for a specified pod; type `PodMetrics`

The following query parameters are supported:
- `labelSelector` - restrict the list of returned objects by labels (list endpoints only)

In the future we may want to introduce the following params:
`aggreator` (`max`, `min`, `95th`, etc.) and `window` (`1h`, `1d`, `1w`, etc.)
which will allow to get the other aggregates over the custom time window.

## Further improvements

Depending on the further requirements the following features may be added:
- support for more metrics
- support for application level metrics
- watch for metrics
- possibility to query for window sizes and aggreation functions (though single window size/aggregation function per request)
- cluster level metrics

<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/proposals/resource-metrics-api.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->