reduce KSM cardinality by denylisting unused metrics

[paulfantom]

In most scenarios KSM ships a lot of metrics and it is one of main sources of cardinality issues in prometheus. This addon allows running KSM in "lite" mode by deny listing metrics that aren't used in mixins and don't have much operational value (mostly because the information they carry is already included in other metrics).

I decided to make this an optional addon as such configuration may be disruptive to end-users and IMHO this should be opt-in.

[paulfantom]

Following metrics are removed with this addon:

kube_*_created - There is not much value (from monitoring perspective) in knowing when each pod started, but this data has high cardinality
kube_*_metadata_resource_version - There is not much value (from monitoring perspective) in knowing which resource version, but this data has high cardinality
kube_replicaset_metadata_generation - In most cases kube_deployment_metadata_generation is more useful due
kube_replicaset_status_observed_generation - Same as above
kube_pod_restart_policy - This is obtainable from manifests and doesn't carry much value in runtime
kube_pod_init_container_status_terminated - kube_pod_init_container_status_terminated carries similar information with additional data about reason
kube_pod_init_container_status_running - kube_pod_init_container_status_running covers similar data
kube_pod_container_status_terminated - kube_pod_container_status_terminated_reason carries similar information with additional data about reason
kube_pod_container_status_running - kube_pod_container_status_ready covers similar data
kube_pod_completion_time - Completion time is useful mostly with cronjobs and in that case kube_job_status_completion_time better reflects this
kube_pod_status_scheduled - kube_pod_status_scheduled_time carries similar information with additional data about time

There are more metrics that aren't used directly in kube-prometheus, but I decided to remove only the ones above as those had a high impact on cardinality, and removing them shouldn't create issues during normal operations.

The idea is not to force all users to use this patch but allow ones who want to and gradually introduce those changes in the default installation.

/cc @prometheus-operator/kube-prometheus-reviewers

[simonpasquier]

lgtm