Skip to content

Latest commit

 

History

History
314 lines (227 loc) · 7.84 KB

README.md

File metadata and controls

314 lines (227 loc) · 7.84 KB
Raw

Requirements for this POC

This POC will use Multipass to create instances to install Grafana, Prometheus and Microk8s, we will install Metrics Server, Kube-State-Metrics and Istio + Basic Prometheus and then we will make metrics used by our main instance with Prometheus.

Install Multipass

sudo snap install multipass --classic

MicroK8s

MicroK8s is the smallest, fastest, fully-conformant Kubernetes that tracks upstream releases and makes clustering trivial. MicroK8s is great for offline development, prototyping, and testing.

Create a MicroK8s template

echo '#cloud-config
runcmd:
 - apt update --fix-missing
 - snap refresh
 - snap install microk8s --classic
 - microk8s status --wait-ready
 - microk8s enable dns ingress' > cloud-config-microk8s.yaml

Create a MicroK8s instance

multipass launch focal -n microk8s -c 2 -m 2G -d 10G --cloud-init cloud-config-microk8s.yaml

Export the current MicroK8s configuration for use with Kubectl

Create the folder to store the configuration of the Kubernetes cluster in the test instance

sudo mkdir -p $HOME/.kube/configs

Export the MicroK8s configuration in the test instance to the created folder

multipass exec microk8s sudo microk8s config > $HOME/.kube/configs/config-microk8s

Use in your session the configuration exported as default for use with Kubectl

export KUBECONFIG=$HOME/.kube/configs/config-microk8s

Install Kubectl

sudo snap install kubectl --classic
kubectl get no
NAME       STATUS   ROLES    AGE   VERSION
microk8s   Ready    <none>   1m    v1.21.3-3+90fd5f3d2aea0a

Add an IP alias of the test instance to teste.info

multipass info microk8s | grep IPv4 | cut -f 2 -d ":" | tr -d [:blank:] | sed 's/$/     microk8s.info/' | sudo tee -a /etc/hosts
multipass info microk8s | grep IPv4 | cut -f 2 -d ":" | tr -d [:blank:] | sed 's/$/     teste.info/' | sudo tee -a /etc/hosts

Deploy a test application

kubectl apply -f https://raw.githubusercontent.com/smashse/playbook/master/HOWTO/KUBERNETES/COMBO/example_combo_full.yaml

Grafana

Create a Grafana template

echo '#cloud-config
runcmd:
 - echo "deb https://packages.grafana.com/oss/deb stable main" > /etc/apt/sources.list.d/grafana.list
 - wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
 - echo "DEBIAN_FRONTEND=noninteractive" >> /etc/environment
 - source /etc/environment && source /etc/environment
 - apt update --fix-missing
 - apt -y install grafana
 - systemctl daemon-reload
 - systemctl start grafana-server
 - systemctl status grafana-server
 - systemctl enable grafana-server.service' > cloud-config-grafana.yaml

Create a Grafana instance

multipass launch focal -n grafana -c 2 -m 1G -d 10G --cloud-init cloud-config-grafana.yaml

Add an IP alias of the test instance to grafana.info

multipass info grafana | grep IPv4 | cut -f 2 -d ":" | tr -d [:blank:] | sed 's/$/     grafana.info/' | sudo tee -a /etc/hosts

Add a Data Source

To add the Data Source to Prometheus with the URL "http://prometheus.info:9090", just follow the steps as per the documentation below:

https://grafana.com/docs/grafana/latest/datasources/add-a-data-source/

Prometheus

Create a Prometheus template

echo '#cloud-config
runcmd:
 - echo "deb https://packages.grafana.com/oss/deb stable main" > /etc/apt/sources.list.d/grafana.list
 - wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
 - echo "DEBIAN_FRONTEND=noninteractive" >> /etc/environment
 - source /etc/environment && source /etc/environment
 - apt update --fix-missing
 - apt -y install prometheus
 - systemctl daemon-reload
 - systemctl start prometheus
 - systemctl status prometheus
 - systemctl enable prometheus.service' > cloud-config-prometheus.yaml

Create a Prometheus instance

multipass launch focal -n prometheus -c 2 -m 1G -d 10G --cloud-init cloud-config-prometheus.yaml

Add an IP alias of the test instance to prometheus.info

multipass info prometheus | grep IPv4 | cut -f 2 -d ":" | tr -d [:blank:] | sed 's/$/     prometheus.info/' | sudo tee -a /etc/hosts

Deploy the Metrics Server

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

OR

wget -c https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
sed -i '/- --secure-port=443/a\        - --kubelet-insecure-tls' components.yaml
kubectl apply -f components.yaml

OR

helm repo add metrics-server https://kubernetes-sigs.github.io/metrics-server/
helm repo update
helm install metrics-server metrics-server/metrics-server --set 'args={--kubelet-insecure-tls}' --namespace kube-system --create-namespace
kubectl get deployment metrics-server -n kube-system

Deploy the Kube-State-Metrics

kubectl apply -f https://github.com/kubernetes/kube-state-metrics/tree/master/examples/standard --recursive

OR

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm install kube-state-metrics prometheus-community/kube-state-metrics --namespace monitoring --create-namespace
kubectl get deployment kube-state-metrics -n monitoring

Istio + Basic Prometheus

Istio provides a basic sample installation to quickly get Prometheus up and running:

kubectl create ns istio-system
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.11/samples/addons/prometheus.yaml
echo 'apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: prometheus-ingress
  namespace: istio-system
spec:
  rules:
  - host: prometheus.istio.info
    http:
      paths:
      - backend:
          service:
            name: prometheus
            port:
              number: 9090
        path: /
        pathType: Prefix' > prometheus_ingress.yaml
kubectl apply -f prometheus_ingress.yaml
multipass info microk8s | grep IPv4 | cut -f 2 -d ":" | tr -d [:blank:] | sed 's/$/     prometheus.istio.info/' | sudo tee -a /etc/hosts

Allow a Prometheus server(prometheus.info) to scrape selected time series from Prometheus server running in Microk8s(prometheus.istio.info).

Access the Prometheus instance

multipass exec prometheus bash

Configuring federation

sudo tee -a /etc/prometheus/prometheus.yml <<EOF
  - job_name: 'federate'
    scrape_interval: 15s

    honor_labels: true
    metrics_path: '/federate'

    params:
      'match[]':
        - '{job="prometheus"}'
        - '{__name__=~"job:.*"}'

    static_configs:
      - targets:
        - 'prometheus.istio.info'
EOF

Restart Prometheus

sudo systemctl restart prometheus

Create test deployments

echo 'africa
america
asia
europa
oceania' > lista.txt
wget -c https://raw.githubusercontent.com/smashse/playbook/master/HOWTO/KUBERNETES/COMBO/example_combo_full.yaml
for i in `cat lista.txt` ; do sed "s/teste/$i/" example_combo_full.yaml > app-$i.yaml ; kubectl apply -f app-$i.yaml ; done
for i in `cat lista.txt` ; do multipass info microk8s | grep IPv4 | cut -f 2 -d ":" | tr -d [:blank:] | sed "s/$/     $i.info/" | sudo tee -a /etc/hosts ; done

Sources:

https://istio.io/latest/docs/ops/integrations/prometheus/

https://github.com/kubernetes-sigs/metrics-server

https://github.com/kubernetes/kube-state-metrics

https://github.com/helm/charts/tree/master/stable/prometheus-operator/templates/grafana/dashboards-1.14

https://prometheus.io/docs/prometheus/latest/federation/

https://grafana.com/grafana/dashboards/10000

https://grafana.com/grafana/dashboards/10856

https://grafana.com/grafana/dashboards/10858

https://grafana.com/grafana/dashboards/8588