prometheus cannot find all k8s resouces, such as pod/services etc

[ghost]

I deployed prometheus with image on my cluster, and added scc privileged and admin role permission to system:serviceaccount:monitoring:default, I hoped I could see the cluster monitoring states from prometheus, but it kept reporting errors:

time="2017-07-14T10:01:50Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:207: Failed to list *v1.Pod: User "system:serviceaccount:monitoring:default" cannot list all pods in the cluster" component="kube_client_runtime" source="kubernetes.go:75"
 time="2017-07-14T10:01:51Z" level=error msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:179: Failed to list *v1.Endpoints: User "system:serviceaccount:monitoring:default" cannot list all endpoints in the cluster" component="kube_client_runtime" source="kubernetes.go:75"

Environment
openshift 1.5

System information:
Linux 3.10.0-514.21.1.el7.x86_64 x86_64

• Prometheus image version:
  docker.io/prom/prometheus latest 4da113bb6ae3 4 weeks ago 74.49 MB

• Alertmanager image version:
  docker.io/prom/alertmanager latest 822973257511 4 weeks ago 17.63 MB

• Prometheus DeployConfig file:

apiVersion: v1
kind: DeploymentConfig
metadata:
  labels:
    app: prometheus
  name: prometheus
spec:
 replicas: 1
 selector:
   app: prometheus
   deploymentconfig: prometheus
 strategy:
   type: Rolling
 template:
   metadata:
     labels:
       app: prometheus
       deploymentconfig: prometheus
   spec:
     containers:
     - args:
       - -storage.local.retention=6h
       - -storage.local.memory-chunks=500000
       - -config.file=/etc/prometheus/prometheus.yml
       - -web.console.libraries=/etc/prometheus/console_libraries
       - -web.console.templates=/etc/prometheus/console
       - -alertmanager.url=http://alertmanager:9093
       image: prom/prometheus:latest
       imagePullPolicy: Always
       securityContext:
         runAsUser: 0
         privileged: true
         capabilities: {}
       name: prometheus
       ports:
       - containerPort: 9090
         name: web
         protocol: TCP
       volumeMounts:
       - mountPath: /prometheus
         name: prometheus-data
       - mountPath: /etc/prometheus
         name: prometheusconf
     dnsPolicy: ClusterFirst
     restartPolicy: Always
     nodeSelector:
       kubernetes.io/hostname: tj1-container-cloud02.kscn
     serviceAccountName: default
     securityContext:
       runAsUser: 0
       privileged: true
     volumes:
     - hostPath:
         path: /prometheus
       name: prometheus-data
     - hostPath:
         path: /etc/prometheus
       name: prometheusconf

• Alertmanager DeployConfig file:

apiVersion: v1
kind: DeploymentConfig
metadata:
  labels:
    app: alertmanager
  name: alertmanager
spec:
 replicas: 1
 selector:
   app: alertmanager
   deploymentconfig: alertmanager
 strategy:
   type: Rolling
 template:
   metadata:
     labels:
       app: alertmanager
       deploymentconfig: alertmanager
   spec:
     containers:
     - image: prom/alertmanager:latest
       imagePullPolicy: Always
       securityContext:
         runAsUser: 0
         privileged: true
         capabilities: {}
       name: alertmanager
       ports:
       - containerPort: 9093
         name: web
         protocol: TCP
       volumeMounts:
       - mountPath: /alertmanager
         name: alertmanager-data
       - mountPath: /etc/alertmanager
         name: alertmanager-config
     dnsPolicy: ClusterFirst
     restartPolicy: Always
     nodeSelector:
       kubernetes.io/hostname: tj1-container-cloud03.kscn
     serviceAccountName: default
     securityContext:
       runAsUser: 0
       privileged: true
     volumes:
       - name: alertmanager-data
         hostPath:
           path: /home/data/alertmanager
       - hostPath:
           path: /etc/alertmanager
         name: alertmanager-config

[brian-brazil]

Usage questinos are best asked at https://groups.google.com/forum/#!forum/prometheus-users

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.