[bitnami/thanos] Support creation of ThanosRuler

[kir4h]

Name and Version

bitnami/thanos

What is the problem this feature will solve?

Currently, Thanos Ruler can be only be deployed in thanos chart by means of a Statefulset.

Prometheus Operator supports creation of Thanos Ruler using a CR.

Using a CR takes away complexity from the Statefulset, but also allows the usage of PrometheusRule objects to define rules in a more flexible manner (using rule selectors to state which Thanos Ruler is the target for a given rule).

This is already the way kube-prometheus-stack deploys it.

This issue relates to #28525

What is the feature you are proposing to solve the problem?

Given we probably don't want to get rid of the Satefulset deployment for backwards compatibility, we should probably add a new template to deploy it using the ThanosRuler CR, and then put it behind a feature flag that selects how ruler will be deployed.

What alternatives have you considered?

Having the CR created as additional resources, but it is not pretty since the auxiliary resources (secret, service, etc...) are controlled by the same feature flag that enables ruler.

We could maybe change that logic to allow creating all those resources but skipping Statefulset creation (and then one can add the CR as extra manifests), but I think having the CR inside as an option would be a best choice.

[juan131]

Hi @kir4h

Our Thanos chart isn't designed to work as an "operator-like" chart. I mean, Thanos components resources are directly managed by the chart instead of relying on CRDs.

If you want to deploy Thanos excluding some components, we could add parameters to disable them. However, we won't support deploying the components via CRDs on this chart.

[kir4h]

Our Thanos chart isn't designed to work as an "operator-like" chart. I mean, Thanos components resources are directly managed by the chart instead of relying on CRDs.

Thanks for the clarification!

If you want to deploy Thanos excluding some components, we could add parameters to disable them. However, we won't support deploying the components via CRDs on this chart.

Thinking about it, I see two possible approaches:

• We add some extra flag to allow disabling ThanosRuler statefulset (https://github.com/bitnami/charts/blob/main/bitnami/thanos/templates/ruler/statefulset.yaml#L6) but still allow deploying of auxiliary resources
  • This would allow someone using .Values.extraList to declare a ThanosRuler and benefit from those resources that are already in the chart
  • We could even document "this is not supported by the chart but if you want to use the CR then... "
• We move this feature request to kube-prometheus chart
  • Makes kind of sense since that chart installs the operator, that in turn supports the CR
  • But at the same time still feeling Thanos ruler should sit in Thanos chart so 🤷

[juan131]

Hi @kir4h

I'm thinking on a 3rd alternative: deploying Thanos setting ruler.replicaCount to 0 (it's more like a workaround but it should work).

We move this feature request to kube-prometheus chart

It seems Thanos Ruler isn't part of the upstream kube-prometheus, see https://github.com/prometheus-operator/kube-prometheus/tree/main/manifests

[kir4h]

I'm thinking on a 3rd alternative: deploying Thanos setting ruler.replicaCount to 0 (it's more like a workaround but it should work).

That was my first thought and is totally doable as a workaround, but feels a little weird as a long term plan since there will be a 0 replicas workload. Good thing is that this change doesn't require any change to the chart, just maybe some example in the documentation.

It seems Thanos Ruler isn't part of the upstream kube-prometheus, see https://github.com/prometheus-operator/kube-prometheus/tree/main/manifests

Not included under manifests folder, but:

• Included in clusterrole https://github.com/prometheus-operator/kube-prometheus/blob/696ce89f1f4d9107bd3a3b026178b320bac03b8e/manifests/prometheusOperator-clusterRole.yaml#L24
• CRD installed along with the manifests https://github.com/prometheus-operator/kube-prometheus/blob/main/manifests/setup/0thanosrulerCustomResourceDefinition.yaml
• Included into kube-prometheus-stack Helm chart https://github.com/prometheus-community/helm-charts/blob/main/charts/kube-prometheus-stack/templates/thanos-ruler/ruler.yaml#L3

At the moment Bitnami chart can deploy:

• Prometheus Operator
• A set of exporters
• An Alertmanager
• A Prometheus

So didn't feel weird to me adding the ability to create ThanosRuler or PrometheusRule given those are implemented by Prometheus Operator.

Anyway, I am fine with either solution even if it's the 0 scaled ruler with some documentation as it will help people looking for a solution to the use case.

[juan131]

Good catch, I missed the ThanosRuler from the chart templates! Thanks so much for sharing it.

If it's supported by upstream, then we should add it to kube-prometheus. Would you like to contribute with a PR adding it? I'll be glad to help you through the review process.

[kir4h]

If it's supported by upstream, then we should add it to kube-prometheus. Would you like to contribute with a PR adding it? I'll be glad to help you through the review process.

Sure, I can contribute with a PR to add it

[github-actions]

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.