previews: Remote-write metrics to core-dev #10737
Conversation
ArthurSens
force-pushed
the
as/preview-remote-write
branch
from
00991a8
to
d111c09
Compare
.werft/build.yaml
Outdated
| - name: PROM_REMOTE_WRITE_USER | ||
| valueFrom: | ||
| secretKeyRef: | ||
| name: prometheus-remote-write-auth | ||
| key: user | ||
| - name: PROM_REMOTE_WRITE_USER | ||
| valueFrom: | ||
| secretKeyRef: | ||
| name: prometheus-remote-write-auth | ||
| key: password |
There was a problem hiding this comment.
@liam-j-bennett just to let you know, the auth problem is that I'm repeating PROM_REMOTE_WRITE_USER here 🤦
One of those should be PROM_REMOTE_WRITE_PASSWORD, I'm fixing and then it should be good to merge
ArthurSens
force-pushed
the
as/preview-remote-write
branch
from
d111c09
to
674fd56
Compare
| urls: ['https://prometheus.gitpod-dev.com/api/v1/write'], | ||
| writeRelabelConfigs: [{ | ||
| sourceLabels: ['__name__'], | ||
| regex: 'probe_.*|rest_client_requests_total', |
There was a problem hiding this comment.
I've just noticed that this regex drops the up metric, which is required by SLI 2.
The up metric is built from ALL targets, so I need to figure out a good regex that will only give us the up from probers...
Signed-off-by: ArthurSens <arthursens2005@gmail.com>
ArthurSens
force-pushed
the
as/preview-remote-write
branch
from
674fd56
to
c9f9378
Compare
|
started the job as gitpod-build-as-preview-remote-write.9 because the annotations in the pull request description changed |
|
@ArthurSens Is this ready for review? I wasn't sure if the network problems were resolved |
|
Yes, they've been resolved :) |
Signed-off-by: ArthurSens arthursens2005@gmail.com
Description
Configure Prometheus in Preview Environments to remote-write metrics to core-dev, but only the ones needed for our SLIs.
Related Issue(s)
Fixes https://github.com/gitpod-io/ops/issues/2764
Fixes https://github.com/gitpod-io/ops/issues/2766
How to test
Unfortunately, we're still having some network issues between previews and core-dev, but once that is solved we should be able to query metrics from the core-dev Prometheus with the following query:
Release Notes
Werft options: