Support resource request tuning for cost optimistation

[alexec]

Summary

Enhance the node view with information highlighting pods that are using much more or less that they request.

Motivation

I want to cost optimise and quickly. Nothing I've yet used makes that possible. But Argo CD is well place to make this easy.

Proposal

1. This view only appears to surface total requests. What about per-pod requests.
2. Combine this with metrics server data to surface the actual usage.
3. Highlight diffs.

[keithchong]

Hi @AlexC, @rbreeze , @alexmt, could you please elaborate on points 2 and 3?

So for point 1, do you want to show the actual CPU/Mem usage in the tooltip of each pod, something like this, or something else?

Looks like there might be server and app controller changes

[alexec]

If a pod is using less resources that it requests (in resource requests) then it is costing more than it needs to.

Metrics Server stores the resource usage of pods.

https://github.com/kubernetes-sigs/metrics-server#use-cases

You can then calculate "(requested resources - resource usage) / requested resources" which would tell you how much more a resource is costing to run than it needs to.

You can then adjust your resource requests to pack you pods more efficiently and save $£¥.

[keithchong]

Yeah, it looks like in the app controller, we are totalling up the capacity and resource requests for the entire node. I think we need to keep track of the individual pod stats, right @rbreeze ?

[keithchong]

This will be for point 1 in the enhancement description. Using the sync-waves example:

Here are the requests for the frontend pod:

Here is the memory request for the backend pod (No specific CPU request, probably should not show it):

Here is the total memory request, which adds up:

Here is the total CPU request:

[keithchong]

Updated with usage stats

[keithchong]

Hi @alexec, @rbreeze, @alexmt, I'm making progress with this. What do you think about the above? What's left is the actual usage percentage (the API returns different units.)

[alexec]

Maybe take a look at Kubecost.

They surface these details, but as a user, I want to know where I can save the most money. I don't want to tweak a resource from 110m to 100m, but I would if it was from 1000m to 500m.

[keithchong]

Hi Alex, perhaps integrating this with Kubecost could be a follow-on change? Here we have the individual pod stats.

[keithchong]

Latest screenshots with PR, with memory units fixed:

microk8s kubectl top pod backend-j2qq6 --namespace=default
NAME            CPU(cores)   MEMORY(bytes)   
backend-j2qq6   0m           2Mi     

microk8s kubectl top pod frontend-9t5tn --namespace=default
NAME             CPU(cores)   MEMORY(bytes)   
frontend-9t5tn   0m           2Mi        

CPU usage > 0m example

When metrics is not enabled: