New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't scrape metrics from coreDNS #2110

Closed
kolesoffac opened this Issue Nov 8, 2018 · 8 comments

Comments

Projects
None yet
5 participants
@kolesoffac
Copy link

kolesoffac commented Nov 8, 2018

What did you do?
Install from helm:
helm install coreos/prometheus-operator --name prometheus-operator --namespace monitoring
helm install coreos/kube-prometheus --name kube-prometheus --namespace monitoring

What did you expect to see?
Target up kube-dns

What did you see instead? Under which circumstances?
image

image

image

  • Kubernetes version information:
Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState:"clean", BuildDate:"2018-10-05T16:46:06Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState:"clean", BuildDate:"2018-10-05T16:36:14Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}
  • Kubernetes cluster kind:

    kubeadm

Screenshots:
image

image

image

For curl http://10.244.3.11:9153/metrics , i have:

...
# HELP coredns_build_info A metric with a constant '1' value labeled by version, revision, and goversion from which CoreDNS was built.
# TYPE coredns_build_info gauge
coredns_build_info{goversion="go1.11",revision="eb51e8b",version="1.2.2"} 1
# HELP coredns_cache_hits_total The count of cache hits.
# TYPE coredns_cache_hits_total counter
coredns_cache_hits_total{server="dns://:53",type="denial"} 66
coredns_cache_hits_total{server="dns://:53",type="success"} 46
# HELP coredns_cache_misses_total The count of cache misses.
# TYPE coredns_cache_misses_total counter
coredns_cache_misses_total{server="dns://:53"} 13157
# HELP coredns_cache_size The number of elements in the cache.
# TYPE coredns_cache_size gauge
coredns_cache_size{server="dns://:53",type="denial"} 83
coredns_cache_size{server="dns://:53",type="success"} 17
# HELP coredns_dns_request_count_total Counter of DNS requests made per zone, protocol and family.
# TYPE coredns_dns_request_count_total counter
coredns_dns_request_count_total{family="1",proto="tcp",server="dns://:53",zone="."} 3
coredns_dns_request_count_total{family="1",proto="udp",server="dns://:53",zone="."} 13266
# HELP coredns_dns_request_duration_seconds Histogram of the time (in seconds) each request took.
# TYPE coredns_dns_request_duration_seconds histogram
coredns_dns_request_duration_seconds_bucket{server="dns://:53",zone=".",le="0.00025"} 6664
coredns_dns_request_duration_seconds_bucket{server="dns://:53",zone=".",le="0.0005"} 8392
coredns_dns_request_duration_seconds_bucket{server="dns://:53",zone=".",le="0.001"} 9399
coredns_dns_request_duration_seconds_bucket{server="dns://:53",zone=".",le="0.002"} 13023
coredns_dns_request_duration_seconds_bucket{server="dns://:53",zone=".",le="0.004"} 13140
coredns_dns_request_duration_seconds_bucket{server="dns://:53",zone=".",le="0.008"} 13152
coredns_dns_request_duration_seconds_bucket{server="dns://:53",zone=".",le="0.016"} 13181
coredns_dns_request_duration_seconds_bucket{server="dns://:53",zone=".",le="0.032"} 13193
....

How can i add target for core-dns?

Thanks

@mathieuherbert

This comment has been minimized.

Copy link

mathieuherbert commented Nov 9, 2018

Hello,
The default values in the helm chart are:
deployCoreDNS: false deployKubeDNS: true

it is needed in this case to override these values in helm install command I guess:
helm install coreos/kube-prometheus --name kube-prometheus --namespace monitoring --set deployKubeDNS=false --set deployCoreDNS=true

@kolesoffac

This comment has been minimized.

Copy link

kolesoffac commented Nov 9, 2018

From upgrade can i do it? Or delete/install only?

@mathieuherbert

This comment has been minimized.

Copy link

mathieuherbert commented Nov 9, 2018

It will work with an upgrade

@kolesoffac

This comment has been minimized.

Copy link

kolesoffac commented Nov 9, 2018

@mathieuherbert Many thanks

@kolesoffac kolesoffac closed this Nov 9, 2018

@kolesoffac kolesoffac reopened this Nov 12, 2018

@kolesoffac

This comment has been minimized.

Copy link

kolesoffac commented Nov 12, 2018

@mathieuherbert,

I do it, but invisible section kube-dns in targets and i see in service discovery dropped as screenshot above.

@kolesoffac

This comment has been minimized.

Copy link

kolesoffac commented Nov 12, 2018

i solved it. Thanks

@kolesoffac kolesoffac closed this Nov 12, 2018

@squat

This comment has been minimized.

Copy link
Member

squat commented Nov 13, 2018

@kolesoffac if you don't mind, please post your solution for any future users who may run into this issue :)

@kolesoffac

This comment has been minimized.

Copy link

kolesoffac commented Nov 13, 2018

@squat
I add endpoint to service kube-prometheus-exporter-coredns:

"ports": [
      {
        "name": "http-metrics-coredns",
        "protocol": "TCP",
        "port": 9153,
        "targetPort": 9153
      }
    ]

and update selector

"selector": {
  "k8s-app": "kube-dns"
},
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment