Fix openvsx-proxy metrics

[vulkoingim]

Description

This adds the kube-rbac-proxy to the openvxs-proxy statefulset, exposing the metrics endpoint through it, so prometheus is able to scrape it correctly.

How to test

Before and after updating the address

Release Notes

NONE

Werft options:

• /werft with-preview

[aledbf]

@vulkoingim Instead of this change, I would suggest to add the kube-rbac-proxy to the statefulset *common.KubeRBACProxyContainer(ctx)

Edit: the role bindings are already there

[andreafalzetti]

Just to understand how this works, we didn't have this param specified here before. How did this impact? Prometheus wasn't able to scrape the metrics at all? and why?

[vulkoingim]

Heya, there was a bit of a longer PR description (guess you can still see it in the history), but trimmed it, since I changed the solution on advise of Alejandro.

Long-story short is that there was a breaking change introduced in: 14095f4 It changed the prometheus listen address from any (0.0.0.0:9500) to 127.0.0.1:9500. Listening on 127.0.0.1 doesn't work because it is not accessible from outside of the container.

The rest of the services in the cluster use kube-rbac-proxy to proxy the metrics endpoint, which this PR adds. The *common.KubeRBACProxyContainer(ctx) is just a function that generates the container spec and we add it to the list of containers of the StatefulSet.

There's a longer discussion on Slack about that, if you are interested: https://gitpod.slack.com/archives/C01KGM9EBD4/p1658907591252089