Pre-requisites

Install Kubernetes (i.e. kops, minikube, etc)

Install Helm

Install Prometheus

Install Grafana

Import Dashboards

• Prometheus for system metrics - Load, CPU, RAM, network, process (mod of 159)
• Nodes - Overview of the nodes utilisation (mod of 3140)
• Kubernetes Resource CPU + Memory Requests - Resource requests vs allocatable in the cluster (mod of 3149)
  • Kubernetes Resource CPU Requests - CPU Resource requests vs allocatable in the cluster
  • Kubernetes Resource Memory Requests - Memory Resource requests vs allocatable in the cluster
• Pod Metrics - Basic pod and container information

Useful dashboards

• Nginx Ingress Stats
  • Update Prometheus helm chart values.yaml:
    • Set controller.stats.enabled: true in your Prometheus helm chart
    • Add the following to controller.service.annotation:

      prometheus.io/port: '10254'
      prometheus.io/scrape: 'true'
      

    • Upgrade Prometheus release with the new values
  • Update Grafana helm chart values.yaml:
    • Add grafana-worldmap-panel to server.installPlugins
    • Upgrade Grafana release with the new values
  • Download the json file, replace [1m] with [2m], and import the updated json file
• Kubernetes Cluster Monitoring (via Prometheus) - excellent dashboard that requires Kubernetes 1.7.3+ and cAdvisor to be installed
  • Update Prometheus helm chart values.yaml:
    • Update the scrape_configs per the dashboard instructions
    • Upgrade Prometheus release with the new values
  • Download the json file, replace ^/dev/[sv]da9$ with ^/dev/.*$, and import the updated json file
• Kubernetes Deployment Metrics - requires Kubernetes 1.7.3+ and cAdvisor to be installed

Note: The https://prometheus-server/targets page provides health stats of your Prometheus Scrape Targets. Additional scrape_configs settings are available here