scrape of ceph luminous export fails with 'no token found'

[jmamma]

What did you do?
Set up scrape from CEPH Luminous cluster

What did you expect to see?

Metrics ingested correctly

What did you see instead? Under which circumstances?

No metrics are ingested

level=warn ts=2018-02-20T05:39:49.829143167Z caller=scrape.go:690 component="scrape manager" scrape_pool=ceph2 target=http://mon3-ceph2-qh2.storage.unimelb.edu.au:9283/metrics msg="append failed" err="no token found"

Environment

• System information:

Linux 4.13.0-32-generic x86_64

• Prometheus version:

Tested with 2 versions:

root@grafana:~/prometheus-2.2.0-rc.0.linux-amd64# ./prometheus --version
prometheus, version 2.2.0-rc.0 (branch: HEAD, revision: 1fe05d4)
build user: root@f7abb25edc70
build date: 20180213-11:40:47
go version: go1.9.2

root@grafana:~/prometheus-2.1.0.linux-amd64# ./prometheus --version
prometheus, version 2.1.0 (branch: HEAD, revision: 85f23d8)
build user: root@6e784304d3ff
build date: 20180119-12:01:23
go version: go1.9.2

• Prometheus configuration file:

# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ['127.0.0.1:9090']
  - job_name: 'ceph2'
    static_configs:
      - targets: ['mon1-ceph2-qh2.storage.unimelb.edu.au:9283', 'mon2-ceph2-qh2.storage.unimelb.edu.au:9283', 'mon3-ceph2-qh2.storage.unimelb.edu.au:9283']

• Alertmanager configuration file:

insert configuration here (if relevant to the issue)

• Logs:

insert Prometheus and Alertmanager logs relevant to the issue here

root@grafana:~/prometheus-2.1.0.linux-amd64# cat index2.html | ./promtool check metrics
error while linting: text format parsing error in line 11035: invalid metric name in comment

ceph_export.tar.gz

[simonpasquier]

As promtool is telling you, the exporter generates an invalid metric name (eg mds_mem_cap+). This exporter needs to be fixed.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.