[prometheus] Add extra manifest templates

[katsew]

Signed-off-by: Yu Kato yu-kato@colopl.co.jp

What this PR does / why we need it:

This PR offers to deploy extra manifests, which includes any template inside prometheus helper.
When we want to deploy extra manifests using template helper the chart provides,
server.extraObjects does not help in some cases.

Which issue this PR fixes

fixes #2139

Special notes for your reviewer:

Checklist

• DCO signed
• Chart Version bumped
• Title of the PR starts with chart name (e.g. [prometheus-couchdb-exporter])

[jean-pierre-schoeman]

@zanhsieh Have a question about the functionality in this PR
The original block in the extra-manifests.yaml was added in this PR 2044

This PR 2140 then extended on it with template based variation

PR 2294 then did exactly the same functionality as 2044 while removing the functionality of this PR 2140

Was this an intentional decision to remove the functionality?
If so, what was the rational behind the removal?

[monotek]

See: https://github.com/prometheus-community/helm-charts/blob/main/charts/prometheus/README.md#to-190

[jean-pierre-schoeman]

@monotek That was the expectation, However we are not able to get template from previous changes to work on the new variation
I suspect that it could be ignorance from my side

Would you be able to provide additional guidance on how to get this to work with new mechanism or point in a direction on how to get this working

Help would be appreciated

extraTemplates:
   - |
     apiVersion: v1
     kind: ConfigMap
     metadata:
       name: extra-config
       labels:
       {{- include "prometheus.server.labels" . | nindent 4 }}
     data:
     {{ ( tpl (.Files.Glob "rules/**/*.yaml").AsConfig .) | indent 2 }}`

The files being loaded in basic form look as follows

groups:
- name: SomeRule
  rules:
  - alert: "SomeAlert"
    expr: some_metric
    for: 3m
    labels:
      priority: P1
      service: "some_service"
      severity: High
    annotations:
      runBook: "https://somelink"
      summary: "extra Info"

Edit: code block is not rendering indentation properly

[monotek]

You have to use "```" for codeblocks.

Current value shuld be "extraManifests" not "extraTemplates".

[jean-pierre-schoeman]

@monotek Please ignore the value name, it was an excerpt from current implementation on our side
This does not work due to toYaml function being executed before the tpl

   - apiVersion: v1
     kind: ConfigMap
     metadata:
       name: extra-config
       labels:
       {{- include "prometheus.server.labels" . | nindent 4 }}
     data:
     {{ ( tpl (.Files.Glob "rules/**/*.yaml").AsConfig .) | indent 2 }}