charts,salt,tools: Add Thanos querier in front of Prometheus

[TeddyAndrieux]

As part of kube-prometheus-stack chart, deploy Thanos sidecar as part
of Prometheus Pod and the Thanos sidecar service used by Thanos querier
to discover the Thanos sidecars IPs.

Re-render the chart using:

./charts/render.py prometheus-operator \
  charts/kube-prometheus-stack.yaml \
  charts/kube-prometheus-stack/ \
  --namespace metalk8s-monitoring \
  --service-config grafana \
  metalk8s-grafana-config \
  metalk8s/addons/prometheus-operator/config/grafana.yaml \
  metalk8s-monitoring \
  --service-config prometheus \
  metalk8s-prometheus-config \
  metalk8s/addons/prometheus-operator/config/prometheus.yaml \
  metalk8s-monitoring \
  --service-config alertmanager \
  metalk8s-alertmanager-config \
  metalk8s/addons/prometheus-operator/config/alertmanager.yaml \
  metalk8s-monitoring \
  --service-config dex \
  metalk8s-dex-config \
  metalk8s/addons/dex/config/dex.yaml.j2 metalk8s-auth \
  --drop-prometheus-rules charts/drop-prometheus-rules.yaml \
  > salt/metalk8s/addons/prometheus-operator/deployed/chart.sls

Import Thanos helm chart from banzaicloud:

helm repo add banzaicloud-stable https://kubernetes-charts.banzaicloud.com
helm repo update
helm fetch -d charts --untar banzaicloud-stable/thanos

Note that we only deploy the Thanos querier and not all other
components.
We also bump Thanos image since the one set in the helm chart is a bit
old and do not support all Prometheus endpoint we need.

Render the Thanos helm chart using

./charts/render.py thanos \
  charts/thanos.yaml charts/thanos/ \
  --namespace metalk8s-monitoring \
  > salt/metalk8s/addons/prometheus-operator/deployed/thanos-chart.sls

Then we replace the Prometheus datasource for Grafana in order to use
this Thanos querier. We also replace the proxy ingress used by MetalK8s
UI to use this Thanos querier as well.

Since we now reach Thanos to retrieve Prometheus information, the order
for the Prometheus rules changed a bit, so they get extracted again
using

./tools/rule_extractor/rule_extractor.py \
  -i <control-plane-ip> -p 8443 -t rules

NOTE: Test not get updated a lot, just added sanity check to ensure
Thanos querier Pod is running, since we already have some tests about
Prometheus and for all those tests we now reach Thanos so we implicitly
test Thanos querier

[bert-e]

Hello teddyandrieux,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Status report is not available.

[bert-e]

Waiting for approval

The following approvals are needed before I can proceed with the merge:

• the author

• one peer

Peer approvals must include at least 1 approval from the following list:

• @alexandre-allard-scality

• @thomasdanan

• @gdemonet

• @ChengYanJin

• @NicolasT

• @JBWatenbergScality

• @TeddyAndrieux

• @lucieleonard

[bert-e]

Waiting for approval

The following approvals are needed before I can proceed with the merge:

• the author

• one peer

Peer approvals must include at least 1 approval from the following list:

• @ChengYanJin

• @NicolasT

• @lucieleonard

• @TeddyAndrieux

• @JBWatenbergScality

• @alexandre-allard-scality

• @thomasdanan

• @gdemonet

[gdemonet]

LGTM except for the small upgrade detail, will test this tomorrow!

[bert-e]

Conflict

There is a conflict between your branch improvement/deploy-thanos-querier and the
destination branch development/2.11.

Please resolve the conflict on the feature branch (improvement/deploy-thanos-querier).

git fetch && \
git checkout origin/improvement/deploy-thanos-querier && \
git merge origin/development/2.11

Resolve merge conflicts and commit

git push origin HEAD:improvement/deploy-thanos-querier

[bert-e]

Waiting for approval

The following approvals are needed before I can proceed with the merge:

• the author

• one peer

Peer approvals must include at least 1 approval from the following list:

• @alexandre-allard-scality

• @gdemonet

• @thomasdanan

• @ChengYanJin

• @lucieleonard

• @TeddyAndrieux

• @JBWatenbergScality

• @NicolasT

[TeddyAndrieux]

/approve

[bert-e]

In the queue

The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.

The changeset will be merged in:

• ✔️ development/2.11

The following branches will NOT be impacted:

• development/2.0
• development/2.1
• development/2.10
• development/2.2
• development/2.3
• development/2.4
• development/2.5
• development/2.6
• development/2.7
• development/2.8
• development/2.9

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of the admin team will
contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

• Any commit you add on the source branch will trigger a new cycle after the
  current queue is merged.
• Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of the admin team now.

The following options are set: approve

[bert-e]

I have successfully merged the changeset of this pull request
into targetted development branches:

• ✔️ development/2.11

The following branches have NOT changed:

• development/2.0
• development/2.1
• development/2.10
• development/2.2
• development/2.3
• development/2.4
• development/2.5
• development/2.6
• development/2.7
• development/2.8
• development/2.9

Please check the status of the associated issue None.

Goodbye teddyandrieux.