Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scrape target returns Empty body/Content-Lenght: 0, but no target error is raised #4937

Closed
zstyblik opened this Issue Nov 30, 2018 · 3 comments

Comments

Projects
None yet
2 participants
@zstyblik
Copy link

zstyblik commented Nov 30, 2018

Proposal

Prometheus should raise/show error for targets which return either empty body or Content-Lenght: 0. Such condition doesn't seem like a valid state to me. Also, it'd help to identify targets which don't produce metrics for one reason or another(I guess counter to this is "add alert rule which is going to check whether metrics X is present").

I remember I had to deal with validation error due to missing \n at the end of metrics list, but that seems to be gone now. Could it be regression? Could it be a new bug? It could be a new

Bug Report

What did you do?

Hooked up Prometheus to consul. However, one of the scrape targets returns 200 OK, empty body and Content-Lenght: 0 due to bug in the code. However, this is not related to consul as the same can be observed with static targets as well.

What did you expect to see?

Scrape target down/scrape error.

What did you see instead? Under which circumstances?

Target is being reported as UP, no errors are being show in Web UI.

Environment

  • System information:

      Linux 3.10.0-862.2.3.el7.x86_64 x86_64
    
  • Prometheus version:

      Observerd at prom/prometheus:v2.4.x and prom/prometheus:v2.5.0
    
  • Prometheus configuration file:

---
global:
  scrape_interval: 10s
  evaluation_interval: 1m
  external_labels:
    environment: abc

rule_files:
  - /prometheus/rules/*.yml

scrape_configs:
- job_name: 'prometheus'
  static_configs:
    - targets: ['localhost:9090']

- job_name: 'file-service-discovery'
  file_sd_configs:
    - files:
      - /prometheus/service-discovery/*.json
      refresh_interval: 1m

- consul_sd_configs:
  - allow_stale: true
    refresh_interval: 30s
    server: 1.2.3.4:8500
    services: [some_service]
  job_name: some_service
@brian-brazil

This comment has been minimized.

Copy link
Member

brian-brazil commented Nov 30, 2018

An empty result is valid. OpenMetrics handles this. Dupe of #1169

@zstyblik

This comment has been minimized.

Copy link
Author

zstyblik commented Nov 30, 2018

@brian-brazil sorry for the noise. I just couldn't find any issue that would match this description. Thanks.

@brian-brazil

This comment has been minimized.

Copy link
Member

brian-brazil commented Nov 30, 2018

No problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.