Use honorLabels instead of labelDrop

[erkanerol]

Thanks to labeldrop, we can label metrics belong to
workloads correctly but as a side effect we lose
namespace labels for metrics belong to control plane.
In the past, honorLabels were not recommended by
openshift monitoring team because of security concerns but now
it is recommended.

Signed-off-by: Erkan Erol eerol@redhat.com

Necessary to fix #5383

Release note:

Use honorLabels instead of labelDrop for namespace label on metrics

[rmohr]

/lgtm
/approve

Great that we can finally use honorLabels.

/hold

@erkanerol feel free to unhold once verified on your target platforms.
@marceloamaral maybe you can think of a way on how to test this since you added the prometheus operator monitoring stack. Maybe we can add a test which gets run on a dedicated cluster?

[kubevirt-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rmohr

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [rmohr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[erkanerol]

@simonpasquier Could you please confirm that honorLabels is safe to use?

[erkanerol]

/unhold

[erkanerol]

Note that this PR will fix missing namespace labels on "up" metrics but there is another issue as stated in #5383

The other problem is here that the expr checks whether the result is zero or not. When the result is empty, it is false and the alert is not triggered. -> We need to fix this as well. I will open another PR to fix alert rules for stale data. Fortunately, we have a unit test infra for that part. It will be verified with automated tests.

[simonpasquier]

yes I think that it's reasonable to use honor_labels here.

[erkanerol]

/retest-required

[kubevirt-commenter-bot]

/retest
This bot automatically retries jobs that failed/flaked on approved PRs.
Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

[kubevirt-commenter-bot]

/retest
This bot automatically retries jobs that failed/flaked on approved PRs.
Silence the bot with an /lgtm cancel or /hold comment for consistent failures.