Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Thanos Receive doesn't announce external_labels which are set in hashrings.json when it works in routing and ingesting mode. #7362

Open
pvlltvk opened this issue May 15, 2024 · 1 comment

Comments

@pvlltvk
Copy link

pvlltvk commented May 15, 2024

Thanos, Prometheus and Golang version used:

Thanos: 0.34.1; go1.21.7, Prometheus 2.48.1

Object Storage Provider:

AWS S3

What happened:

Thanos Receive doesn't announce external_labels set in hashrings.json when it's working in routing and ingesting mode.
I explicitly set need external labels for the tenant in the hashrings.json:

[
  {
    "endpoints": [
      {
        "address": "endpoint-1:10901"
      },
      {
        "address": "endpoint-2:10901"
      },
      {
        "address": "endpoint-3:10901"
      },
      {
        "address": "endpoint-4:10901"
      },
      {
        "address": "endpoint-5:10901"
      }
    ],
    "external_labels": {
      "team": "frontend",
      "prometheus": "prometheus-1"
    },
    "hashring": "frontend",
    "tenants": [
      "frontend-prometheus-1"
    ]
  }
]

When I used the default mode of the receive deployment (without splitting to route and ingester) this external_labels were announced by the receive instances (I could check it via Query UI), but after switching to the split mode I noticed that this labels are gone in Query UI.

What you expected to happen:
I expect that external_labels functionality works correctly in route/ingester mode.

How to reproduce it (as minimally and precisely as possible):

Run Thanos Receive in route/ingester mode and try to use external_labels in hashrings configuration.

@haanhvu
Copy link
Contributor

haanhvu commented May 15, 2024

Yes, currently this is just supported in RouterIngester mode. But we do have plan to support it in the RouterOnly and IngesterOnly mode too. Here's the proposal: https://github.com/thanos-io/thanos/blob/main/docs/proposals-accepted/202209-receive-tenant-external-labels.md

Actually, before supporting this in these two modes, it's best to split their implementation logics. I remember there's a PR that's very close to finish here: #6043

PRs are welcome to finish these^^

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants