Skip to content

/prometheus

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prometheus 2.0.0 dies on headChunkReader #3554

Closed
Savemech opened this Issue Dec 7, 2017 · 3 comments

Comments

Assignees
No one assigned
Labels
component/local storage kind/bug
Projects
None yet
Milestone
No milestone
3 participants
@Savemech @gouthamve @brian-brazil
@Savemech
Copy link

Savemech commented Dec 7, 2017

What did you do?
Actually nothing, few people watch Grafana, some alerts firing, some metrics collected. Nothing special

What did you expect to see?
Prometheus work smooth and by the numbers
What did you see instead? Under which circumstances?
Prometheus process got

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x159134a]

Environment
Prometheus running in docker, from prom/prometheus
Server Version: 17.09.0-ce
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: xfs
Kernel Version: 4.4.0-95-generic
Operating System: Ubuntu 16.04.3 LTS
OSType: linux
Architecture: x86_64
CPUs: 12
Total Memory: 79.42GiB

Prometheus was started with followed command:
docker run --restart=always -d -p 9090:9090 -h="fancyname" --name=prometheus -v /opt/prom-docker/:/etc/prometheus/ -v /opt/prometheus/data:/prometheus --entrypoint="" prom/prometheus sh -c '/bin/prometheus --config.file=/etc/prometheus/prometheus.yml --storage.tsdb.path=/prometheus --storage.tsdb.min-block-duration=1h --storage.tsdb.retention=91d'

  • System information:

Linux 4.4.0-95-generic x86_64

  • Prometheus version:
    docker exec -it prometheus /bin/sh
    /prometheus $ prometheus --version
    prometheus, version 2.0.0 (branch: HEAD, revision: 0a74f98)
    build user: root@615b82cb36b6
    build date: 20171108-07:11:59
    go version: go1.9.2

  • Alertmanager version:

no issues with alertmanager actually

  • Logs:

This happens time-to-time actually

level=info ts=2017-12-07T02:30:00.085795682Z caller=compact.go:361 component=tsdb msg="compact blocks" count=1 mint=1512608400000 maxt=1512612000000
level=info ts=2017-12-07T02:30:03.080997533Z caller=head.go:345 component=tsdb msg="head GC completed" duration=135.564515ms
level=info ts=2017-12-07T02:30:04.84143907Z caller=head.go:354 component=tsdb msg="WAL truncation completed" duration=1.760338949s
level=info ts=2017-12-07T03:30:00.081869338Z caller=compact.go:361 component=tsdb msg="compact blocks" count=1 mint=1512612000000 maxt=1512615600000
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x159134a]

goroutine 7029625 [running]:
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*headChunkReader).Chunk(0xc4289568a0, 0xcdf7c0001fb, 0x28a0f40, 0xc4507dc560, 0x0, 0x0)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:775 +0x8a
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*populatedChunkSeries).Next(0xc423fae5a0, 0xc4556e5bc0)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/querier.go:533 +0x12a
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*blockSeriesSet).Next(0xc433a9b340, 0xc4507dc001)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/querier.go:564 +0x48
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*mergedSeriesSet).Next(0xc433a9b380, 0xc4e098d901)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/querier.go:416 +0x245
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*mergedSeriesSet).Next(0xc433a9b400, 0xc4a2433401)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/querier.go:416 +0x245
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*mergedSeriesSet).Next(0xc433a9b540, 0xc4e10fecb8)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/querier.go:416 +0x245
github.com/prometheus/prometheus/storage/tsdb.seriesSet.Next(0x2893c80, 0xc433a9b540, 0xc4ae9cf0f0)
        /go/src/github.com/prometheus/prometheus/storage/tsdb/tsdb.go:208 +0x31
github.com/prometheus/prometheus/storage/tsdb.(*seriesSet).Next(0xc450c63220, 0x2890540)
        <autogenerated>:1 +0x45
github.com/prometheus/prometheus/promql.expandSeriesSet(0x289b2c0, 0xc450c63220, 0x5, 0x8, 0x289b2c0, 0xc450c63220, 0x500)
        /go/src/github.com/prometheus/prometheus/promql/engine.go:546 +0x6a
github.com/prometheus/prometheus/promql.(*Engine).populateIterators.func2(0x2881f00, 0xc48587c120, 0x0)
        /go/src/github.com/prometheus/prometheus/promql/engine.go:518 +0xf2
github.com/prometheus/prometheus/promql.inspector.Visit(0xc428e83860, 0x2881f00, 0xc48587c120, 0x2889100, 0xc428e83860)
        /go/src/github.com/prometheus/prometheus/promql/ast.go:305 +0x3a
github.com/prometheus/prometheus/promql.Walk(0x2889100, 0xc428e83860, 0x2881f00, 0xc48587c120)
        /go/src/github.com/prometheus/prometheus/promql/ast.go:254 +0x58
github.com/prometheus/prometheus/promql.Walk(0x2889100, 0xc428e83860, 0x2881cc0, 0xc43f084380)
        /go/src/github.com/prometheus/prometheus/promql/ast.go:281 +0x832
github.com/prometheus/prometheus/promql.Walk(0x2889100, 0xc428e83860, 0x2881cc0, 0xc43f084440)
        /go/src/github.com/prometheus/prometheus/promql/ast.go:280 +0x7d9
github.com/prometheus/prometheus/promql.Walk(0x2889100, 0xc428e83860, 0x2881cc0, 0xc43f084500)
        /go/src/github.com/prometheus/prometheus/promql/ast.go:280 +0x7d9
github.com/prometheus/prometheus/promql.Walk(0x2889100, 0xc428e83860, 0x2881cc0, 0xc43f084540)
        /go/src/github.com/prometheus/prometheus/promql/ast.go:281 +0x832
github.com/prometheus/prometheus/promql.Inspect(0x2881cc0, 0xc43f084540, 0xc428e83860)
        /go/src/github.com/prometheus/prometheus/promql/ast.go:315 +0x4b
github.com/prometheus/prometheus/promql.(*Engine).populateIterators(0xc4201c0ed0, 0x289cbc0, 0xc48587c180, 0xc4e07420a0, 0x411bcc, 0xc4202126c0, 0xc4e10ff2b8, 0x43c8ac)
        /go/src/github.com/prometheus/prometheus/promql/engine.go:515 +0x409
github.com/prometheus/prometheus/promql.(*Engine).execEvalStmt(0xc4201c0ed0, 0x289cbc0, 0xc48587c180, 0xc43f084580, 0xc4e07420a0, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/prometheus/prometheus/promql/engine.go:361 +0xf5
github.com/prometheus/prometheus/promql.(*Engine).exec(0xc4201c0ed0, 0x289cbc0, 0xc48587c180, 0xc43f084580, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/prometheus/prometheus/promql/engine.go:341 +0x317
github.com/prometheus/prometheus/promql.(*query).Exec(0xc43f084580, 0x289cb40, 0xc420134540, 0xbe824b4ea59e679f)
        /go/src/github.com/prometheus/prometheus/promql/engine.go:179 +0xb8
github.com/prometheus/prometheus/rules.(*AlertingRule).Eval(0xc4b0456600, 0x289cb40, 0xc420134540, 0xbe824b4ea59e679f, 0x1164df035159b, 0x2967e40, 0xc4201c0ed0, 0xc42038cd00, 0x0, 0x0, ...)
        /go/src/github.com/prometheus/prometheus/rules/alerting.go:163 +0x133
github.com/prometheus/prometheus/rules.(*Group).Eval.func1(0x1bb1720, 0x8, 0xc4b0456680, 0xbe824b4ea59e679f, 0x1164df035159b, 0x2967e40, 0x0, 0x289ce80, 0xc4b0456600)
        /go/src/github.com/prometheus/prometheus/rules/manager.go:307 +0x194
github.com/prometheus/prometheus/rules.(*Group).Eval(0xc4b0456680, 0xbe824b4ea59e679f, 0x1164df035159b, 0x2967e40)
        /go/src/github.com/prometheus/prometheus/rules/manager.go:375 +0xbf
github.com/prometheus/prometheus/rules.(*Group).run.func1()
        /go/src/github.com/prometheus/prometheus/rules/manager.go:182 +0x81
github.com/prometheus/prometheus/rules.(*Group).run(0xc4b0456680)
        /go/src/github.com/prometheus/prometheus/rules/manager.go:207 +0x23c
github.com/prometheus/prometheus/rules.(*Manager).ApplyConfig.func1.1(0xc420134580, 0xc4b0456680)
        /go/src/github.com/prometheus/prometheus/rules/manager.go:511 +0x46
created by github.com/prometheus/prometheus/rules.(*Manager).ApplyConfig.func1
        /go/src/github.com/prometheus/prometheus/rules/manager.go:506 +0x56
level=info ts=2017-12-07T03:30:03.976175161Z caller=main.go:215 msg="Starting Prometheus" version="(version=2.0.0, branch=HEAD, revision=0a74f98628a0463dddc90528220c94de5032d1a0)"
level=info ts=2017-12-07T03:30:03.976263503Z caller=main.go:216 build_context="(go=go1.9.2, user=root@615b82cb36b6, date=20171108-07:11:59)"
level=info ts=2017-12-07T03:30:03.976307744Z caller=main.go:217 host_details="(Linux 4.4.0-95-generic #118-Ubuntu SMP Thu Sep 7 16:07:16 UTC 2017 x86_64 172.30.47.13 (none))"
level=info ts=2017-12-07T03:30:03.978955907Z caller=web.go:380 component=web msg="Start listening for connections" address=0.0.0.0:9090
level=info ts=2017-12-07T03:30:03.979089041Z caller=targetmanager.go:71 component="target manager" msg="Starting target manager..."
level=info ts=2017-12-07T03:30:03.979067183Z caller=main.go:314 msg="Starting TSDB"
level=info ts=2017-12-07T03:30:14.228461082Z caller=main.go:326 msg="TSDB started"
level=info ts=2017-12-07T03:30:14.228554358Z caller=main.go:394 msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml
level=info ts=2017-12-07T03:30:14.244134657Z caller=main.go:371 msg="Server is ready to receive requests."


level=info ts=2017-12-02T11:30:05.008243231Z caller=head.go:354 component=tsdb msg="WAL truncation completed" duration=1.962326067s
level=info ts=2017-12-02T12:30:00.110095549Z caller=compact.go:361 component=tsdb msg="compact blocks" count=1 mint=1512212400000 maxt=1512216000000
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x159134a]

goroutine 104644555 [running]:
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*headChunkReader).Chunk(0xc499324060, 0xb319c0006c2, 0x28a0f40, 0xc436294240, 0x0, 0x0)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:775 +0x8a
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*populatedChunkSeries).Next(0xc421562120, 0xc4963d7440)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/querier.go:533 +0x12a
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*blockSeriesSet).Next(0xc4e5b76140, 0xc4715d3f01)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/querier.go:564 +0x48
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*mergedSeriesSet).Next(0xc4e5b76180, 0xc4715d2d01)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/querier.go:416 +0x245
github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*mergedSeriesSet).Next(0xc4e5b76200, 0xc4759eecb8)
        /go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/querier.go:416 +0x245
github.com/prometheus/prometheus/storage/tsdb.seriesSet.Next(0x2893c80, 0xc4e5b76200, 0xc51944eb40)
        /go/src/github.com/prometheus/prometheus/storage/tsdb/tsdb.go:208 +0x31
github.com/prometheus/prometheus/storage/tsdb.(*seriesSet).Next(0xc43313a090, 0x2890540)
        <autogenerated>:1 +0x45
github.com/prometheus/prometheus/promql.expandSeriesSet(0x289b2c0, 0xc43313a090, 0x5, 0x8, 0x289b2c0, 0xc43313a090, 0x500)
        /go/src/github.com/prometheus/prometheus/promql/engine.go:546 +0x6a
github.com/prometheus/prometheus/promql.(*Engine).populateIterators.func2(0x2881f00, 0xc43cc4e6c0, 0x0)
        /go/src/github.com/prometheus/prometheus/promql/engine.go:518 +0xf2
github.com/prometheus/prometheus/promql.inspector.Visit(0xc42b480150, 0x2881f00, 0xc43cc4e6c0, 0x2889100, 0xc42b480150)
        /go/src/github.com/prometheus/prometheus/promql/ast.go:305 +0x3a
github.com/prometheus/prometheus/promql.Walk(0x2889100, 0xc42b480150, 0x2881f00, 0xc43cc4e6c0)
        /go/src/github.com/prometheus/prometheus/promql/ast.go:254 +0x58
github.com/prometheus/prometheus/promql.Walk(0x2889100, 0xc42b480150, 0x2881cc0, 0xc4f6b56780)
        /go/src/github.com/prometheus/prometheus/promql/ast.go:281 +0x832
github.com/prometheus/prometheus/promql.Walk(0x2889100, 0xc42b480150, 0x2881cc0, 0xc4f6b56840)
        /go/src/github.com/prometheus/prometheus/promql/ast.go:280 +0x7d9
github.com/prometheus/prometheus/promql.Walk(0x2889100, 0xc42b480150, 0x2881cc0, 0xc4f6b56900)
        /go/src/github.com/prometheus/prometheus/promql/ast.go:280 +0x7d9
github.com/prometheus/prometheus/promql.Walk(0x2889100, 0xc42b480150, 0x2881cc0, 0xc4f6b56940)
        /go/src/github.com/prometheus/prometheus/promql/ast.go:281 +0x832
github.com/prometheus/prometheus/promql.Inspect(0x2881cc0, 0xc4f6b56940, 0xc42b480150)
        /go/src/github.com/prometheus/prometheus/promql/ast.go:315 +0x4b
github.com/prometheus/prometheus/promql.(*Engine).populateIterators(0xc42037d140, 0x289cbc0, 0xc43cc4e720, 0xc49e2b21e0, 0x411bcc, 0xc42013e5a0, 0xc4759ef2b8, 0x43c8ac)
        /go/src/github.com/prometheus/prometheus/promql/engine.go:515 +0x409
github.com/prometheus/prometheus/promql.(*Engine).execEvalStmt(0xc42037d140, 0x289cbc0, 0xc43cc4e720, 0xc4f6b56980, 0xc49e2b21e0, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/prometheus/prometheus/promql/engine.go:361 +0xf5
github.com/prometheus/prometheus/promql.(*Engine).exec(0xc42037d140, 0x289cbc0, 0xc43cc4e720, 0xc4f6b56980, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/prometheus/prometheus/promql/engine.go:341 +0x317
github.com/prometheus/prometheus/promql.(*query).Exec(0xc4f6b56980, 0x289cb40, 0xc42044a380, 0xbe80c512a123be69)
        /go/src/github.com/prometheus/prometheus/promql/engine.go:179 +0xb8
github.com/prometheus/prometheus/rules.(*AlertingRule).Eval(0xc434812000, 0x289cb40, 0xc42044a380, 0xbe80c512a123be69, 0x5cb30e510f33a, 0x2967e40, 0xc42037d140, 0xc4203f8a80, 0x0, 0x0, ...)
        /go/src/github.com/prometheus/prometheus/rules/alerting.go:163 +0x133
github.com/prometheus/prometheus/rules.(*Group).Eval.func1(0x1bb1720, 0x8, 0xc434812080, 0xbe80c512a123be69, 0x5cb30e510f33a, 0x2967e40, 0x0, 0x289ce80, 0xc434812000)
        /go/src/github.com/prometheus/prometheus/rules/manager.go:307 +0x194
github.com/prometheus/prometheus/rules.(*Group).Eval(0xc434812080, 0xbe80c512a123be69, 0x5cb30e510f33a, 0x2967e40)
        /go/src/github.com/prometheus/prometheus/rules/manager.go:375 +0xbf
github.com/prometheus/prometheus/rules.(*Group).run.func1()
        /go/src/github.com/prometheus/prometheus/rules/manager.go:182 +0x81
github.com/prometheus/prometheus/rules.(*Group).run(0xc434812080)
        /go/src/github.com/prometheus/prometheus/rules/manager.go:207 +0x23c
github.com/prometheus/prometheus/rules.(*Manager).ApplyConfig.func1.1(0xc42044a3c0, 0xc434812080)
        /go/src/github.com/prometheus/prometheus/rules/manager.go:511 +0x46
created by github.com/prometheus/prometheus/rules.(*Manager).ApplyConfig.func1
        /go/src/github.com/prometheus/prometheus/rules/manager.go:506 +0x56
then
        /go/src/github.com/prometheus/prometheus/rules/manager.go:506 +0x56
level=info ts=2017-12-07T03:30:03.976175161Z caller=main.go:215 msg="Starting Prometheus" version="(version=2.0.0, branch=HEAD, revision=0a74f98628a0463dddc90528220c94de5032d1a0)"
level=info ts=2017-12-07T03:30:03.976263503Z caller=main.go:216 build_context="(go=go1.9.2, user=root@615b82cb36b6, date=20171108-07:11:59)"
level=info ts=2017-12-07T03:30:03.976307744Z caller=main.go:217 host_details="(Linux 4.4.0-95-generic #118-Ubuntu SMP Thu Sep 7 16:07:16 UTC 2017 x86_64 172.30.47.13 (none))"
level=info ts=2017-12-07T03:30:03.978955907Z caller=web.go:380 component=web msg="Start listening for connections" address=0.0.0.0:9090
level=info ts=2017-12-07T03:30:03.979089041Z caller=targetmanager.go:71 component="target manager" msg="Starting target manager..."
level=info ts=2017-12-07T03:30:03.979067183Z caller=main.go:314 msg="Starting TSDB"
level=info ts=2017-12-07T03:30:14.228461082Z caller=main.go:326 msg="TSDB started"
level=info ts=2017-12-07T03:30:14.228554358Z caller=main.go:394 msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml
level=info ts=2017-12-07T03:30:14.244134657Z caller=main.go:371 msg="Server is ready to receive requests."
level=info ts=2017-12-07T03:31:14.249036959Z caller=compact.go:361 component=tsdb msg="compact blocks" count=3 mint=1512604800000 maxt=1512615600000
level=info ts=2017-12-07T04:30:00.045000136Z caller=compact.go:361 component=tsdb msg="compact blocks" count=1 mint=1512615600000 maxt=1512619200000
level=info ts=2017-12-07T04:30:02.717930284Z caller=head.go:345 component=tsdb msg="head GC completed" duration=100.131542ms
level=info ts=2017-12-07T04:30:05.739198927Z caller=head.go:354 component=tsdb msg="WAL truncation completed" duration=3.021178884s
level=info ts=2017-12-07T05:30:00.047424468Z caller=compact.go:361 component=tsdb msg="compact blocks" count=1 mint=1512619200000 maxt=1512622800000
level=info ts=2017-12-07T05:30:02.861662196Z caller=head.go:345 component=tsdb msg="head GC completed" duration=109.142216ms
level=info ts=2017-12-07T05:30:04.670708457Z caller=head.go:354 component=tsdb msg="WAL truncation completed" duration=1.808948151s
level=info ts=2017-12-07T06:30:00.048508309Z caller=compact.go:361 component=tsdb msg="compact blocks" count=1 mint=1512622800000 maxt=1512626400000
level=info ts=2017-12-07T06:30:02.622201659Z caller=head.go:345 component=tsdb msg="head GC completed" duration=117.724399ms
level=info ts=2017-12-07T06:30:04.617015123Z caller=head.go:354 component=tsdb msg="WAL truncation completed" duration=1.994717745s
level=info ts=2017-12-07T06:30:04.914124132Z caller=compact.go:361 component=tsdb msg="compact blocks" count=3 mint=1512615600000 maxt=1512626400000
level=info ts=2017-12-07T06:30:10.168261642Z caller=compact.go:361 component=tsdb msg="compact blocks" count=3 mint=1512594000000 maxt=1512626400000
level=info ts=2017-12-07T06:30:17.453283854Z caller=compact.go:361 component=tsdb msg="compact blocks" count=3 mint=1512529200000 maxt=1512626400000
level=info ts=2017-12-07T07:30:00.044470646Z caller=compact.go:361 component=tsdb msg="compact blocks" count=1 mint=1512626400000 maxt=1512630000000
level=info ts=2017-12-07T07:30:02.782369931Z caller=head.go:345 component=tsdb msg="head GC completed" duration=115.945847ms
level=info ts=2017-12-07T07:30:04.612401997Z caller=head.go:354 component=tsdb msg="WAL truncation completed" duration=1.829943545s
level=info ts=2017-12-07T08:30:00.049439956Z caller=compact.go:361 component=tsdb msg="compact blocks" count=1 mint=1512630000000 maxt=1512633600000
level=info ts=2017-12-07T08:30:02.797654172Z caller=head.go:345 component=tsdb msg="head GC completed" duration=109.787385ms
level=info ts=2017-12-07T08:30:04.910216728Z caller=head.go:354 component=tsdb msg="WAL truncation completed" duration=2.112471002s
level=info ts=2017-12-07T09:30:00.045521549Z caller=compact.go:361 component=tsdb msg="compact blocks" count=1 mint=1512633600000 maxt=1512637200000
level=info ts=2017-12-07T09:30:02.944224081Z caller=head.go:345 component=tsdb msg="head GC completed" duration=114.337792ms
level=info ts=2017-12-07T09:30:04.98754891Z caller=head.go:354 component=tsdb msg="WAL truncation completed" duration=2.043224835s
level=info ts=2017-12-07T09:30:05.138117446Z caller=compact.go:361 component=tsdb msg="compact blocks" count=3 mint=1512626400000 maxt=1512637200000
level=info ts=2017-12-07T10:30:00.045366801Z caller=compact.go:361 component=tsdb msg="compact blocks" count=1 mint=1512637200000 maxt=1512640800000
level=info ts=2017-12-07T10:30:02.735508351Z caller=head.go:345 component=tsdb msg="head GC completed" duration=114.772166ms
level=info ts=2017-12-07T10:30:04.224979255Z caller=head.go:354 component=tsdb msg="WAL truncation completed" duration=1.48938585s
level=info ts=2017-12-07T10:43:21.599136103Z caller=main.go:394 msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml

@brian-brazil brian-brazil added component/local storage kind/bug labels Dec 8, 2017

@gouthamve

This comment has been minimized.

Sign in to view
Copy link
Member

gouthamve commented Dec 13, 2017

Being tracked here: prometheus/tsdb#214
@Savemech

This comment has been minimized.

Sign in to view
Copy link
Author

Savemech commented Dec 18, 2017

Closed this as DUP

@Savemech Savemech closed this Dec 18, 2017

@lock

This comment has been minimized.

Sign in to view
Copy link

lock bot commented Mar 23, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Mar 23, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.