Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

panic: runtime error: integer divide by zero #3064

Closed
WeiBanjo opened this Issue Aug 12, 2017 · 2 comments

Comments

Projects
None yet
2 participants
@WeiBanjo
Copy link

WeiBanjo commented Aug 12, 2017

What did you expect to see?
No panic

What did you see instead? Under which circumstances?
Prometheus crashes after running for a day.

Environment
Centos 7.2

  • System information:

Linux 3.10.0-327.4.4.el7.x86_64 x86_64

  • Prometheus version:

prometheus, version 2.0.0-beta.1 (branch: HEAD, revision: 4dcb465)
build user: root@d3f9974fac5a
build date: 20170811-12:12:36
go version: go1.8.3

  • Prometheus configuration file:
global:
  scrape_interval: 600s
  evaluation_interval: 60s
  scrape_timeout: 30s

scrape_configs:
  - job_name: 'prometheus-federation-jmx-exporter'
    scrape_interval: 60s
    scrape_timeout: 60s
    scheme: 'https'
    honor_labels: true
    metrics_path: /federate
    params:
      'match[]':
        - '{job="prod-jmx-exporter"}'
    static_configs:
      - targets:
        - 127.0.0.1:9001
        - 127.0.0.1:9000
  - job_name: 'prometheus-federation-statsd-exporter'
    scrape_interval: 60s
    scrape_timeout: 60s
    scheme: 'https'
    honor_labels: true
    metrics_path: /federate
    params:
      'match[]':
        - '{job="prod-statsd-exporter"}'
    static_configs:
      - targets:
        - 127.0.0.1:9001
        - 127.0.0.1:9000
  • Logs:
Aug 12 21:15:56 prometheus[1482]: panic: runtime error: integer divide by zero
Aug 12 21:15:56 prometheus[1482]: goroutine 129 [running]:
Aug 12 21:15:56 prometheus[1482]: github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*LeveledCompactor).Plan(0xc4204f7ec0, 0x7ffe084aee1f, 0x11, 0x0, 0x15a390f7a00, 0x15dd3bd8800, 0x0, 0x0)
Aug 12 21:15:56 prometheus[1482]: /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:173 +0x621
Aug 12 21:15:56 prometheus[1482]: github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*DB).compact(0xc4202b0460, 0xc420514e00, 0x0, 0x0)
Aug 12 21:15:56 prometheus[1482]: /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:406 +0x2c5
Aug 12 21:15:56 prometheus[1482]: github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*DB).run(0xc4202b0460)
Aug 12 21:15:56 prometheus[1482]: /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:281 +0x3c6
Aug 12 21:15:56 prometheus[1482]: created by github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.Open
Aug 12 21:15:56 prometheus[1482]: /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:250 +0x488
@klausenbusk

This comment has been minimized.

Copy link

klausenbusk commented Aug 12, 2017

I just expected the same with 2.0.0-beta.1.
Config:

scrape_configs:
  - job_name: "prometheus"

    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'clients'
    scrape_interval: 30s
    file_sd_configs:
      - files: ["/etc/prometheus/clients.json"]

Log:

time="2017-08-12T21:25:54Z" level=info msg="Starting prometheus (version=2.0.0-beta.1, branch=HEAD, revision=4dcb465029f39a2a3030333a81753f8a40ec90ce)" source="main.go:204" 
time="2017-08-12T21:25:54Z" level=info msg="Build context (go=go1.8.3, user=root@d3f9974fac5a, date=20170811-12:12:36)" source="main.go:205" 
time="2017-08-12T21:25:54Z" level=info msg="Host details (Linux 4.11.11-coreos #1 SMP Tue Jul 18 23:06:59 UTC 2017 x86_64 prometheus-153384988-pdsn7 (none))" source="main.go:206" 
time="2017-08-12T21:25:54Z" level=info msg="Starting tsdb" source="main.go:218" 
time="2017-08-12T21:26:52Z" level=info msg="tsdb started" source="main.go:224" 
time="2017-08-12T21:26:52Z" level=info msg="Loading configuration file /etc/prometheus/prometheus.yml" source="main.go:357" 
time="2017-08-12T21:26:52Z" level=info msg="Starting target manager..." source="targetmanager.go:67" 
time="2017-08-12T21:26:52Z" level=info msg="Server is Ready to receive requests." source="main.go:334" 
time="2017-08-12T21:26:52Z" level=info msg="Listening on 0.0.0.0:9090" source="web.go:327" 
panic: runtime error: integer divide by zero

goroutine 62 [running]:
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*LeveledCompactor).Plan(0xc4203b9260, 0x7ffc1cece0c2, 0x13, 0x0, 0x15d8ac8c600, 0x15dd3bd8800, 0x0, 0x0)
	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:173 +0x621
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*DB).compact(0xc4201962a0, 0xc4201acc00, 0x0, 0x0)
	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:406 +0x2c5
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*DB).run(0xc4201962a0)
	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:281 +0x3c6
created by github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.Open
	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:250 +0x488

Edit:
I just restarted the server and got this in the log (not sure if it is related):

time="2017-08-12T22:31:09Z" level=error msg="Opening storage failed: open block /var/lib/prometheus/01BQC7T69FGJ86BKB44N31NY3W: open WAL %s: invalid magic header 0 in "/var/lib/prometheus/01BQC7T69FGJ86BKB44N31NY3W/wal/000001"" source="main.go:221" 
@lock

This comment has been minimized.

Copy link

lock bot commented Mar 23, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Mar 23, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.