Skip to content

Add an option to expose prometheus metrics securely via kube-rbac-proxy#1545

Merged
fedepaol merged 9 commits into
metallb:mainfrom
fedepaol:metrics/addsecuremetrics
Aug 3, 2022
Merged

Add an option to expose prometheus metrics securely via kube-rbac-proxy#1545
fedepaol merged 9 commits into
metallb:mainfrom
fedepaol:metrics/addsecuremetrics

Conversation

@fedepaol
Copy link
Copy Markdown
Member

Here we adjust the manifests and the helm chart to optionally set a kube-rbac-proxy sidecar to interact with prometheus, in order to expose the metrics securely.

At the same time, we change CI to deploy the prometheus k8s stack and validate the metrics against the real prometheus instance (as opposed to validating that the pods are exposing the metrics). This will give us more confidence in the fact that the integration with prometheus is really working.

@fedepaol fedepaol force-pushed the metrics/addsecuremetrics branch 5 times, most recently from 11c044a to b27fa01 Compare July 29, 2022 21:08
Comment thread config/frr/speaker-patch.yaml
Comment thread tasks.py
Comment thread config/manifests/metallb-frr.yaml
template:
metadata:
annotations:
prometheus.io/port: "7472"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

weird, that shouldn't have been here? remember removing it on the initial frr exporter pr for the reason in the previous comment - when no port is defined it should look for all monitoring ports 😄

Comment thread config/manifests/metallb-frr-prometheus.yaml Outdated
Comment thread e2etest/pkg/metrics/prometheus.go Outdated
Comment thread e2etest/pkg/metrics/prometheus.go
Comment thread e2etest/pkg/metrics/prometheus.go
Comment thread e2etest/pkg/metrics/prometheus.go
Comment thread e2etest/l2tests/l2.go Outdated
@fedepaol fedepaol force-pushed the metrics/addsecuremetrics branch 6 times, most recently from bc62af6 to 616c8ad Compare August 1, 2022 13:21
Copy link
Copy Markdown
Member

@oribon oribon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@fedepaol fedepaol force-pushed the metrics/addsecuremetrics branch 9 times, most recently from f352c16 to 5e1fbea Compare August 2, 2022 15:11
Changing the overlays to add a layer for exposing metrics to the
prometheus operator via servicemonitors.

Signed-off-by: Federico Paolinelli <fpaoline@redhat.com>
Adding options to deploy prometheus and run metrics tests against it
passing the namespace it's deployed to.

We add a local configuration of prometheus stripped down of unnecessary
items, deploying only the operator and prometheus itself.

Signed-off-by: Federico Paolinelli <fpaoline@redhat.com>
Testing the metrics exposed by the pods doesn't cover the full e2e
solution, and eventual issues in the integration with prometheus won't
appear. Here we add the option to the e2e tests to check the metrics on
prometheus too, if deployed on the cluster.

Signed-off-by: Federico Paolinelli <fpaoline@redhat.com>
@fedepaol fedepaol force-pushed the metrics/addsecuremetrics branch 2 times, most recently from 4374eee to 65ee24e Compare August 2, 2022 21:29
Adding an option to add kube-rbac-proxy to the pods, exposing the
metrics via a secure channel, and adding an option to use
servicemonitors in order to consume those securely exposed metrics.

Signed-off-by: Federico Paolinelli <fpaoline@redhat.com>
Regenerating the manifests and adding two variants for prometheus
enabled.

Signed-off-by: Federico Paolinelli <fpaoline@redhat.com>
Deploying prometheus to be able to run prometheus tests.
Also, add lanes with no prometheus.  We want to make sure the
deployment works also with no prometheus, so add a few helm / manifest
lanes without it.

Signed-off-by: Federico Paolinelli <fpaoline@redhat.com>
Extending the docs accordingly.

Signed-off-by: Federico Paolinelli <fpaoline@redhat.com>
The prometheus tests add test duration time. Raising the timeout to
accomodate them.

Signed-off-by: Federico Paolinelli <fpaoline@redhat.com>
@fedepaol fedepaol force-pushed the metrics/addsecuremetrics branch from ed73248 to e075f57 Compare August 3, 2022 11:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants