Metrics collection profiles

[JoaoBraveCoding]

This PR proposes the addition of the metrics collection profiles feature for the cluster-monitoring-operator

Issue https://issues.redhat.com/browse/MON-3043

[JoaoBraveCoding]

/assign @simonpasquier @jan--f

Some sections still need to be finished and I still want to polish the text in some parts but consider it a v1 ;)

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[jan--f]

Nice work, I left a few comments to start us off.

[simonpasquier]

Very nice start!

[jan--f]

Just a few more comments, but I think this looks pretty good. I think the draft status can be removed soon so we can gather feedback from others.

[openshift-bot]

Inactive enhancement proposals go stale after 28d of inactivity.

See https://github.com/openshift/enhancements#life-cycle for details.

Mark the proposal as fresh by commenting /remove-lifecycle stale.
Stale proposals rot after an additional 7d of inactivity and eventually close.
Exclude this proposal from closing by commenting /lifecycle frozen.

If this proposal is safe to close now please do so with /close.

/lifecycle stale

[JoaoBraveCoding]

/remove-lifecycle stale

[openshift-bot]

Inactive enhancement proposals go stale after 28d of inactivity.

See https://github.com/openshift/enhancements#life-cycle for details.

Mark the proposal as fresh by commenting /remove-lifecycle stale.
Stale proposals rot after an additional 7d of inactivity and eventually close.
Exclude this proposal from closing by commenting /lifecycle frozen.

If this proposal is safe to close now please do so with /close.

/lifecycle stale

[openshift-bot]

Stale enhancement proposals rot after 7d of inactivity.

See https://github.com/openshift/enhancements#life-cycle for details.

Mark the proposal as fresh by commenting /remove-lifecycle rotten.
Rotten proposals close after an additional 7d of inactivity.
Exclude this proposal from closing by commenting /lifecycle frozen.

If this proposal is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[rexagod]

/remove-lifecycle rotten

Some thoughts on merging/layering the Collection Profiles featureset with the EP in question.

The primary setback I can observe is the fact that the two differ w.r.t. the degree of dynamism they offer. Users are allowed to switch back and forth between profiles, but w.r.t. the EP, they will need to "commit" to one of the full or minimal profiles for the cluster lifecycle because (2) we plan to support turning a component on, but not turning it off. This is in line with other superseding use-cases to ours, such as (5) eliminating or replacing cluster profiles which has been categorized as a non-goal.

Furthermore, since we cannot embed logic into the installer itself ((6) encoding logic in the installer itself about which components should be disabled under specific circumstances ), at the point where it's initialized, there would be little to no components that actually define profiles to validate in the first place. The installer will ideally need to check for the profiles defined by these OLM components' exporters down the line, and validate them every now and then, which seems fall in the EP non-goals ((4) allowing components to be disabled post-install).

Finally, every dependent resource that's a part the Collection Profiles featureset, such as rules, alerts, SMs, etc., that components deploy may or may not have the capability annotation with it, but may define a collection profile no less, which can cause them to be left out, and may not be the robust assumption we'd like to have (since we'd be subject to assuming that every component respects the capability annotation).

Consequently, there are constraints on embedding logic to handle specific cases such as updating service monitors with the appropriate regex for the currently set capability.openshift.io/name=monitoring_collection_profiles_{full,minimal} annotation. If dynamism is something we can tolerate to skip, the EP would still delegate such operations back to CMO, since it has no intention of doing that itself (no embedded logic), while including this EP as a dependency into the currently established workflow, which we'll have to build on top of.

All that being said, one way partially to go about this (while respecting the terms imposed) is by defining a subset capability (minimal), while the superset (full) can be optionally enabled down the line, if need be, since ((2) admins can enable a previously excluded optional component, at runtime), although it's easy to see this will not be of much use owing to the constraints that shifts control from components to the central stack. Additionally, it's worth explicitly mentioning here that this procedure goes forward, is irreversible, and can only happen once.

This is what leads me to believe it's better to keep the two decoupled to harness the most out of each featureset.

[dhellmann]

#1555 is changing the enhancement template in a way that will cause the header check in the linter job to fail for existing PRs. If this PR is merged within the development period for 4.16 you may override the linter if the only failures are caused by issues with the headers (please make sure the markdown formatting is correct). If this PR is not merged before 4.16 development closes, please update the enhancement to conform to the new template.

[openshift-bot]

Inactive enhancement proposals go stale after 28d of inactivity.

See https://github.com/openshift/enhancements#life-cycle for details.

Mark the proposal as fresh by commenting /remove-lifecycle stale.
Stale proposals rot after an additional 7d of inactivity and eventually close.
Exclude this proposal from closing by commenting /lifecycle frozen.

If this proposal is safe to close now please do so with /close.

/lifecycle stale

[openshift-bot]

Stale enhancement proposals rot after 7d of inactivity.

See https://github.com/openshift/enhancements#life-cycle for details.

Mark the proposal as fresh by commenting /remove-lifecycle rotten.
Rotten proposals close after an additional 7d of inactivity.
Exclude this proposal from closing by commenting /lifecycle frozen.

If this proposal is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[openshift-bot]

Rotten enhancement proposals close after 7d of inactivity.

See https://github.com/openshift/enhancements#life-cycle for details.

Reopen the proposal by commenting /reopen.
Mark the proposal as fresh by commenting /remove-lifecycle rotten.
Exclude this proposal from closing again by commenting /lifecycle frozen.

/close

[openshift-ci]

@openshift-bot: Closed this PR.

In response to this:

Rotten enhancement proposals close after 7d of inactivity.

See https://github.com/openshift/enhancements#life-cycle for details.

Reopen the proposal by commenting /reopen.
Mark the proposal as fresh by commenting /remove-lifecycle rotten.
Exclude this proposal from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[dhellmann]

(automated message) This pull request is closed with lifecycle/rotten. The associated Jira ticket, MON-2483, has status "Closed, Done". Should the PR be reopened, updated, and merged? If not, removing the lifecycle/rotten label will tell this bot to ignore it in the future.

[rexagod]

/reopen
/remove-lifecycle rotten
/lifecycle frozen

[openshift-ci]

@rexagod: Reopened this PR.

In response to this:

/reopen
/remove-lifecycle rotten
/lifecycle frozen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci]

@rexagod: The lifecycle/frozen label cannot be applied to Pull Requests.

In response to this:

/reopen
/remove-lifecycle rotten
/lifecycle frozen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci]

@JoaoBraveCoding: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/markdownlint	48a5c98	link	true	/test markdownlint

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[openshift-bot]

Inactive enhancement proposals go stale after 28d of inactivity.

See https://github.com/openshift/enhancements#life-cycle for details.

Mark the proposal as fresh by commenting /remove-lifecycle stale.
Stale proposals rot after an additional 7d of inactivity and eventually close.
Exclude this proposal from closing by commenting /lifecycle frozen.

If this proposal is safe to close now please do so with /close.

/lifecycle stale

[openshift-bot]

Stale enhancement proposals rot after 7d of inactivity.

See https://github.com/openshift/enhancements#life-cycle for details.

Mark the proposal as fresh by commenting /remove-lifecycle rotten.
Rotten proposals close after an additional 7d of inactivity.
Exclude this proposal from closing by commenting /lifecycle frozen.

If this proposal is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale