Define requests & limits for all placeos services

[viv-4]

• Find & define reasonable defaults for dev/poc
• Find & define reasonable defaults for prod
• Guaranteed class for core & frontends (limits == requests)
• Burstable class with limits for api, auth, staff (limits > requests)
• Burstable class without limits for rubbersoul, triggers, dispatch, source (only requests)

notes:

• when there is a resource shortage on a node, BestEffort pods are killed first, then Burstable
• if a Burstable pod exceeds it's memory limit the kubernetes scheduler will stop/recreate it with OOMKilled status
• a pods requests must fit in a nodes available Allocatable Capacity to be scheduled to it

[viv-4]

Currently monitoring following resource definitions on 3 node cluster - 2cpu 8gb nodes,
api, auth, core, staff - 2 pods | elastic 2 data, 3 master pod cluster | rethinkdb 3 pod cluster
lens metrics (prometheus etc.) deployed
Fairly low driver usage cluster - More active cluster required more overall resources and higher request/limit values on api & core memory

• core

            limits:
              cpu: 1
              memory: 1536Mi
            requests:
              cpu: 1
              memory: 1536Mi

• api

            limits:
              cpu: 100m
              memory: 100Mi
            requests:
              cpu: 10m
              memory: 50Mi

• auth

            limits:
              cpu: 500m
              memory: 500Mi
            requests:
              cpu: 10m
              memory: 400Mi

• staff

            limits:
              cpu: 25m
              memory: 250Mi
            requests:
              cpu: 10m
              memory: 50Mi

• frontends - frontends

            limits:
              cpu: 500m
              memory: 100Mi
            requests:
              cpu: 500m
              memory: 100Mi

• frontends - nginx

            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi

• search-ingest

            requests:
              cpu: 10m
              memory: 50Mi

• triggers

            requests:
              cpu: 10m
              memory: 50Mi

• dispatch (no active deployment to reference - 5Mi covers it running without activity)

            requests:
              cpu: 10m
              memory: 5Mi

• source

            requests:
              cpu: 10m
              memory: 50Mi

third-party charts using their defaults except elastic-data:

requests:
  cpu: 25m
  memory: 1536Mi

These requests are suitable for dev/poc cluster with pod redundancy and low usage

[viv-4]

Above committed to: 100eac4
To be merged to master
Prod values to go in new prod branch