Slice bounds out of range during compaction

[gebolze]

What did you do?

Prometheus was running without any noticeable problems for several days. Also block compaction was successful in the past.

What did you expect to see?

What did you see instead? Under which circumstances?

We've noticed a high memory consumption on the server. The server itself has 32GB and prometheus reached about 99% of the available memory the error below appeared in the log and the process terminated.

When restarting the process the same error is logged.

Environment

• Prometheus version:

caller=main.go:225 msg="Starting Prometheus" version="(version=2.1.0, branch=HEAD, revision=85f23d82a045d103ea7f3c89a91fba4a93e6367a)"
caller=main.go:226 build_context="(go=go1.9.2, user=root@6e784304d3ff, date=20180119-12:13:43)"
caller=main.go:227 host_details=(windows)
caller=main.go:228 fd_limits=N/A

• Logs:

level=info ts=2018-03-07T07:00:18.1350829Z caller=compact.go:387 component=tsdb msg="compact blocks" count=3 mint=1520380800000 maxt=1520402400000
level=info ts=2018-03-07T07:00:36.663172Z caller=compact.go:387 component=tsdb msg="compact blocks" count=3 mint=1520337600000 maxt=1520402400000
level=info ts=2018-03-07T07:01:25.0616255Z caller=compact.go:387 component=tsdb msg="compact blocks" count=3 mint=1520208000000 maxt=1520402400000
level=info ts=2018-03-07T07:04:20.6429566Z caller=compact.go:387 component=tsdb msg="compact blocks" count=3 mint=1519819200000 maxt=1520402400000
level=info ts=2018-03-07T07:11:17.5247753Z caller=compact.go:387 component=tsdb msg="compact blocks" count=3 mint=1518652800000 maxt=1520402400000
panic: runtime error: slice bounds out of range

goroutine 361 [running]:
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/fileutil.mmap(0xc0a681f2c0, 0xa00f242f, 0xc0ad883f80, 0x0, 0x0, 0x45, 0x0)
	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/fileutil/mmap_windows.go:38 +0x460
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/fileutil.OpenMmapFile(0xc0a2062370, 0x45, 0x2, 0xc0a2062370, 0x45)
	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/fileutil/mmap.go:24 +0x9b
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/index.NewFileReader(0xc0a2062370, 0x45, 0x2, 0xc0a2062370, 0x45, 0x0)
	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/index/index.go:582 +0x40
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.OpenBlock(0xc052a9c840, 0x3f, 0x2940580, 0xc0424c5480, 0x0, 0xc07e00ff00, 0x4)
	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/block.go:256 +0x15d
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*DB).reload(0xc042666600, 0xc09de62740, 0x3, 0x4, 0x0, 0x0)
	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:486 +0x388
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*DB).compact(0xc042666600, 0x1, 0x0, 0x0)
	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:395 +0x4b1
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*DB).run(0xc042666600)
	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:263 +0x311
created by github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.Open
	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:227 +0x590

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.