Labels are overwritten even if honor_labels set to true

[lgogolin]

Bug Report

What did you do?

Having 2 prometheus instances:

1. instance in us-east-1 region with external_labels set:

        external_labels:
          project: monitoring
          region: "us-east-1"

2. instance in eu-central-1 region with external_labels set:

        external_labels:
          project: monitoring
          region: "eu-central-1"

Instance no.2 has also configured federate job, that scrape us-east-1 region prometheus.

      - job_name: 'federate'
        scrape_interval: 15s

        honor_labels: true
        metrics_path: '/federate'

        params:
          'match[]':
            - '{__name__=~".+"}'

        static_configs:
          - targets:
            - 'prom-monitoring.us-east-1.domain.name'

What did you expect to see?

Metrics with both labels:

region="eu-central-1"

and

region="us-east-1"

What did you see instead? Under which circumstances?

All metrics have label

region="eu-central-1"

even those from us-east-1

• Prometheus version:

prometheus, version 2.4.3 (branch: HEAD, revision: 167a4b4e73a8eca8df648d2d2043e21bdb9a7449)
  build user:       root@1e42b46043e9
  build date:       20181004-08:42:02
  go version:       go1.11.1

[roberth1988]

Hi guys am interested in the solution of this regex problem too. Anyone can help or provide a fix here?

[brian-brazil]

All metrics have label

Where exactly are you seeing this? I'd expect to see us-east-1 and no value.