[bitnami/thanos] receiveDistributor is not set up to work properly in the chart

[AlexDCraig]

Name and Version

bitnami/thanos 15.4.6

What is the problem this feature will solve?

It's a good idea to run Thanos receive in "dual" mode in this chart, where there is a fleet of distributors and a fleet of ingesters. Unfortunately, the chart isn't set up right for this though, because it doesn't allow for a service or ingress under receiveDistributor.

I'm assuming that when you're using the receiveDistributor, your Prometheus remote write instances should be pointed at an ingress that routes traffic to receiveDistributor. Therefore, I think these two components are necessary.

What is the feature you are proposing to solve the problem?

Add a service that's on by default. Add an ingress that you can turn on.

What alternatives have you considered?

You can always create services and ingresses out of band of the chart.

[AlexDCraig]

Looking at this more, I'm puzzled how receiveDistributor is supposed to work here... How does it form a hash ring with other members of its deployment? Or is the intention that there is no replicating at all going on in this setup? Maybe I'm misreading https://thanos.io/tip/proposals-accepted/202012-receive-split.md/

[srekkas]

As i understand with this config two services is created
thanos-receive -
thanos-receive-distributor pods
thanos-receive-headless - thanos-receive pods

then you use like this "thanos-receive.observability.svc.cluster.local" for prometheus remote write.

receive:
  enabled: true
  logLevel: info
  mode: "dual-mode"
  tsdbRetention: 15d
  replicationFactor: 3
  replicaCount: 3
  service:
    additionalHeadless: true

receiveDistributor:
  enabled: true
  replicaCount: 3
  replicationFactor: 3

[AlexDCraig]

I understand now that the chart does work properly, it's just confusing because the service and ingress of receive flip to point at receiveDistributor when you've enabled dual mode.

Still, I would appreciate a separate service for the ingesters in this case, because as-is you can't monitor the ingesters in the chart, because there's only one service and it points at receiveDistributor.

For instance, I just added this service purely for metrics. I suggest the chart allows you to do this, so we don't need to manage it out of band:

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: receive
    app.kubernetes.io/instance: thanos-bitnami-receiver
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: thanos
    app.kubernetes.io/version: 0.35.0
    prometheus-operator/monitor: "true"
  name: thanos-bitnami-receiver-receive-metrics
  namespace: thanos-bitnami
spec:
  ports:
  - name: http
    port: 10902
    protocol: TCP
    targetPort: http
  selector:
    app.kubernetes.io/component: receive
    app.kubernetes.io/instance: thanos-bitnami-receiver
    app.kubernetes.io/name: thanos