Need deduplication when both metric and metric_total exist

[RRAlex]

On a namespace where the following queries returns a result (say "2"):

sum( rabbitmq_queue_messages_ready{})  by (kubernetes_namespace ) > 0

Trying it directly seems to return 0:

kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1/namespaces/staging/metrics/rabbitmq_queue_messages_ready

Result:

{
  "kind": "MetricValueList",
  "apiVersion": "custom.metrics.k8s.io/v1beta1",
  "metadata": {
    "selfLink": "/apis/custom.metrics.k8s.io/v1beta1/namespaces/staging/metrics/rabbitmq_queue_messages_ready"
  },
  "items": [
    {
      "describedObject": {
        "kind": "Namespace",
        "name": "staging",
        "apiVersion": "/__internal"
      },
      "metricName": "rabbitmq_queue_messages_ready",
      "timestamp": "2018-02-05T22:12:21Z",
      "value": "0"
    }
  ]
}

From the logs it's seems to be calling rate() within the sum():

7471c5_0[1298]: I0205 22:12:21.725052       1 api.go:74] GET http://prometheus.kube-system-monitoring.svc:9090/prometheus/api/v1/query?query=sum%28rate%28rabbitmq_queue_messages_ready_total%7Bkubernetes_namespace%3D%22staging%22%7D%5B5m%5D%29%29+by+%28kubernetes_namespace%29&time=1517868741.722 200 OK

This seems to be a bug as per:
https://github.com/DirectXMan12/k8s-prometheus-adapter/#presentation

Cheers!

[RRAlex]

I just realized that directly in Prometheus, we have both rabbitmq_queue_messages_ready and rabbitmq_queue_messages_ready_total.
So it seems the second one gets shorten to the first and overrides it.
So the bug might actually be an override protection problem rather than a miscategorization!

[bradenwright]

@RRAlex @DirectXMan12 any further information on this, I have the exact use case. We are trying to scale workers off length of rabbitmq length.

[bradenwright]

I saw #46 may the answer is that we need to wait on this PR.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.