prometheus fetcher can‘t pull metrics

[blank-1]

Please answer these questions before submitting your issue.

• Why do you submit this issue?
• Question or discussion
• [ x] Bug
• Requirement
• Feature or performance improvement

---

Question

• What do you want to know?
  how to pull prom metrics

---

Bug

• Which version of SkyWalking, OS and JRE?
  skywalking 8.0.1

• Which company or project?

• What happen?
  If possible, provide a way for reproducing the error. e.g. demo application, component version.

docker-compose

      SW_PROMETHEUS_FETCHER: default
      SW_PROMETHEUS_FETCHER_ACTIVE: 'true'
    volumes:
      - ./config/alarm-settings.yml:/skywalking/config/alarm-settings.yml:rw
      - ./config/prom-fetcher.yml:/skywalking/fetcher-prom-rules/prom-fetcher.yml:rw

prom-fetcher.yml

fetcherInterval: PT5S
fetcherTimeout: PT0S
metricsPath: /metrics
staticConfig:
  targets:
    - localhost:9100
  labels:
    service: exporter-server
metricsRules:
  - name: process_cpu_seconds_total
    scope: SERVICE
    operation: avg
    sources:
      process_cpu_seconds_total:
        counterFunction: RATE
        range: PT5M
        scale: 2
        relabel:
          service:
            - service
          instance:
            - instance

post http://localhost:8080/graphql

{"query":"query queryTypeOfMetrics($name: String!) {typeOfMetrics(name: $name)}","variables":{"name":"node_cpu_seconds_total"}}

result

{
    "data": {
        "typeOfMetrics": "UNKNOWN"
    }
}

Requirement or improvement

• Please describe about your requirements or improvement suggestions.

[wu-sheng]

Without metrics rule, nothing will happen. SkyWalking is based on pre-calculation, even we accept Prometheus format, nothing changes there. The document requires you to config those rules clearly.

[blank-1]

how to do?prom-fetcher.yml is not metric setting? this's is node-exporter
http://localhost:9100/metrics?collect%5B%5D=cpu&collect%5B%5D=meminfo can get metric

[wu-sheng]

how to do?prom-fetcher.yml is not metric setting?

You changed your issue description after submit, please don't do this again, other people can't follow the context.

Several things,

1. Make sure source is correct, really exist in the Prometheus.
2. You are doing rate in 5 mins, so the metrics will be valid after 5 mins clearly.
3. Why a service level metric has instance relabel?

There is a lot of rules and codes in there, all things start at PrometheusFetcherProvider#notifyAfterCompleted. It should be easy to debug by yourself.

[blank-1]

prom metrics page can get graph!

[wu-sheng]

There is no point in proving Prometheus itself is working. You are just fetching data in Prometheus format. Nothing related to the Prometheus server.

[blank-1]

metrics need add ’meter_‘ prefix

[wu-sheng]

@blank-1 Welcome to send a pull request to make the document more clear.

[wu-sheng]

@blank-1 I rechecked the document, the description is included already in the YAML file format definition.

https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-fetcher.md#metric_rules

# The name of rule, which combinates with a prefix 'meter_' as the index/table name in storage.
name: <string>

[blank-1]

sw ui graph can't get metric data . setting right, es index is exist,ui can get metric, and prom metric url can show real graph !

[wu-sheng]

Check the data in the index. Query is not related to Prometheus in anyway. It is as same as the native matrics.