Possible Bug: runtime error: invalid memory address or nil pointer dereference

[alevchuk]

Bug Report

Highlight: panic: runtime error: invalid memory address or nil pointer dereference. created by github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.Open /home/prometheus/gocode/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:272 +0x468

What did you do?
Used https://github.com/prometheus/prometheus#building-from-source
to build Prometheus on Rasbperry Pi 3+, started node_exporter, and started Prometheus.

What did you expect to see?
Prometheus to keep running.

What did you see instead? Under which circumstances?
prometheus crashes about 1 minute after start

Environment

• System information:

  Linux 4.14.62-v7+ armv7l

• Prometheus version:

 prometheus --version
prometheus, version  (branch: , revision: )
  build user:
  build date:
  go version:       go1.10.3

It was built from f033f48

• Prometheus configuration file:

scrape_configs:
- job_name: 'node'
  static_configs:
  - targets: ['localhost:9100']

• Logs:

prometheus@lnpi3:~/gocode/src/github.com/prometheus/prometheus $ prometheus --config.file=$HOME/.prometheus/prometheus.yml --storage.tsdb.path=$HOME/.promet
heus/data
level=info ts=2018-10-05T06:48:22.102210902Z caller=main.go:244 msg="Starting Prometheus" version="(version=, branch=, revision=)"
level=info ts=2018-10-05T06:48:22.102744076Z caller=main.go:245 build_context="(go=go1.10.3, user=, date=)"
level=info ts=2018-10-05T06:48:22.103079021Z caller=main.go:246 host_details="(Linux 4.14.62-v7+ #1134 SMP Tue Aug 14 17:10:10 BST 2018 armv7l lnpi3 (none))"
level=info ts=2018-10-05T06:48:22.103520789Z caller=main.go:247 fd_limits="(soft=1024, hard=1048576)"
level=info ts=2018-10-05T06:48:22.10383136Z caller=main.go:248 vm_limits="(soft=unlimited, hard=unlimited)"
level=info ts=2018-10-05T06:48:22.117884807Z caller=web.go:399 component=web msg="Start listening for connections" address=0.0.0.0:9090                     level=info ts=2018-10-05T06:48:22.117893244Z caller=main.go:562 msg="Starting TSDB ..."
level=info ts=2018-10-05T06:48:22.447124822Z caller=main.go:572 msg="TSDB started"                                                                          level=info ts=2018-10-05T06:48:22.447780911Z caller=main.go:632 msg="Loading configuration file" filename=/home/prometheus/.prometheus/prometheus.yml       level=info ts=2018-10-05T06:48:22.451325783Z caller=main.go:658 msg="Completed loading of configuration file" filename=/home/prometheus/.prometheus/prometheus.yml                                                                                                                                                      level=info ts=2018-10-05T06:48:22.451899009Z caller=main.go:531 msg="Server is ready to receive web requests."

panic: runtime error: invalid memory address or nil pointer dereference
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x4 pc=0x12388]

goroutine 135 [running]:                                                                                                                                    sync/atomic.addUint64(0x13eae75c, 0x2, 0x0, 0x458ec442, 0x3fc82921)
        /home/prometheus/lightning_src/go/src/sync/atomic/64bit_arm.go:31 +0x4c
github.com/prometheus/prometheus/vendor/github.com/prometheus/client_golang/prometheus.(*histogram).Observe(0x13eae700, 0x458ec442, 0x3fc82921)
        /home/prometheus/gocode/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/client_golang/prometheus/histogram.go:272 +0x60
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*LeveledCompactor).write.func1(0x13e65ddc, 0x13fad200, 0x13e94840, 0x40, 0x60273cb1, 0xbee5e03b, 0x20da4c7, 0xd, 0x1f0dca0)
        /home/prometheus/gocode/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:436 +0x124
panic(0x10fdd98, 0x1f00f48)
        /home/prometheus/lightning_src/go/src/runtime/panic.go:502 +0x204
sync/atomic.addUint64(0x13eae7cc, 0x2, 0x0, 0x0, 0x40510000)
        /home/prometheus/lightning_src/go/src/sync/atomic/64bit_arm.go:31 +0x4c
github.com/prometheus/prometheus/vendor/github.com/prometheus/client_golang/prometheus.(*histogram).Observe(0x13eae770, 0x0, 0x40510000)
        /home/prometheus/gocode/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/client_golang/prometheus/histogram.go:272 +0x60
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*instrumentedChunkWriter).WriteChunks(0x149a30c0, 0x14950320, 0x1, 0x1, 0x14950320, 0x1)
        /home/prometheus/gocode/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:414 +0x74
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*LeveledCompactor).populateBlock(0x13fad200, 0x13e65e28, 0x1, 0x1, 0x148702a0, 0x14d17d0, 0x13f64000, 0x14c2e28, 0x149a30c0, 0x0, ...)
        /home/prometheus/gocode/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:620 +0xc70
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*LeveledCompactor).write(0x13fad200, 0x7edd97c8, 0x21, 0x148702a0, 0x13e65e28, 0x1, 0x1, 0x0, 0x0)
        /home/prometheus/gocode/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:472 +0x2f0
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*LeveledCompactor).Write(0x13fad200, 0x7edd97c8, 0x21, 0x14c7218, 0x149a2fc0, 0x3fcea800, 0x166, 0x403c8500, 0x166, 0x0, ...)
        /home/prometheus/gocode/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/compact.go:393 +0x294
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*DB).compact(0x13eae540, 0x0, 0x0, 0x0)
        /home/prometheus/gocode/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:404 +0x2ac
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*DB).run(0x13eae540)
        /home/prometheus/gocode/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:303 +0x218
created by github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.Open
        /home/prometheus/gocode/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:272 +0x468

[beorn7]

I'll take this. We simply have to update the vendoring for client_golang to include prometheus/client_golang@7973e47 .

Note: This is not in the release-2.4 brunch, but only in master.