REGRESSION: Test deadlocks

[swsnider]

on a clean checkout of HEAD (facf60e), running make, I get a deadlock repeatably.

$ make
curl -o /Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/.build/cache/go1.4.2.darwin-amd64-osx10.8.tar.gz -L https://golang.org/dl/go1.4.2.darwin-amd64-osx10.8.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 96 100 96 0 0 364 0 --:--:-- --:--:-- --:--:-- 365
100 58.7M 100 58.7M 0 0 8137k 0 0:00:07 0:00:07 --:--:-- 9162k
tar -C /Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/.build/root -xzf /Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/.build/cache/go1.4.2.darwin-amd64-osx10.8.tar.gz
touch /Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/.build/root/go/bin/go
cp -a /Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/Godeps/_workspace "/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/.build/root/gopath"
ln -s /Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus /Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/.build/root/gopath/src/github.com/prometheus/prometheus
TMPDIR=/tmp GOROOT=/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/.build/root/go GOPATH=/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/.build/root/gopath /Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/.build/root/go/bin/go build -o prometheus -ldflags " -X github.com/prometheus/prometheus/version.Version 0.15.1 -X github.com/prometheus/prometheus/version.Revision facf60e -X github.com/prometheus/prometheus/version.Branch master -X github.com/prometheus/prometheus/version.BuildUser swsnider@ssnider-mini.apple.com -X github.com/prometheus/prometheus/version.BuildDate 20150827-19:38:59 -X github.com/prometheus/prometheus/version.GoVersion 1.4.2" github.com/prometheus/prometheus/cmd/prometheus
TMPDIR=/tmp GOROOT=/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/.build/root/go GOPATH=/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/.build/root/gopath /Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/.build/root/go/bin/go build -o promtool -ldflags " -X github.com/prometheus/prometheus/version.Version 0.15.1 -X github.com/prometheus/prometheus/version.Revision facf60e -X github.com/prometheus/prometheus/version.Branch master -X github.com/prometheus/prometheus/version.BuildUser swsnider@ssnider-mini.apple.com -X github.com/prometheus/prometheus/version.BuildDate 20150827-19:38:59 -X github.com/prometheus/prometheus/version.GoVersion 1.4.2" github.com/prometheus/prometheus/cmd/promtool
TMPDIR=/tmp GOROOT=/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/.build/root/go GOPATH=/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/.build/root/gopath /Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/.build/root/go/bin/go test -short ./...
? _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/cmd/prometheus [no test files]
? _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/cmd/promtool [no test files]
ok _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/config 0.017s
ok _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/notification 0.014s
ok _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/promql 0.278s
ok _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/retrieval 2.599s
time="2015-08-27T19:39:23-07:00" level=error msg="Error reading file "fixtures/_test.yml": yaml: did not find expected node content" file=file.go line=170
fatal error: all goroutines are asleep - deadlock!

goroutine 1 [chan receive]:
testing.RunTests(0x569bb8, 0x6b8120, 0x7, 0x7, 0xc20803aa01)
/usr/local/go/src/testing/testing.go:556 +0xad6
testing.(*M).Run(0xc2080525f0, 0x6c34e0)
/usr/local/go/src/testing/testing.go:485 +0x6c
main.main()
_/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/retrieval/discovery/_test/_testmain.go:64 +0x1d5

goroutine 9 [chan receive]:
_/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/retrieval/discovery.testFileSD(0xc2080321b0, 0x492000, 0x4)
/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/retrieval/discovery/file_test.go:109 +0xdbd
_/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/retrieval/discovery.TestFileSD(0xc2080321b0)
/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/retrieval/discovery/file_test.go:16 +0x95
testing.tRunner(0xc2080321b0, 0x6b8120)
/usr/local/go/src/testing/testing.go:447 +0xbf
created by testing.RunTests
/usr/local/go/src/testing/testing.go:555 +0xa8b

goroutine 10 [chan send]:
gopkg.in/fsnotify%2ev1.(_Watcher).Close(0xc2080549c0, 0x0, 0x0)
/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/.build/root/gopath/src/gopkg.in/fsnotify.v1/kqueue.go:87 +0x220
_/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/retrieval/discovery.(_FileDiscovery).stop(0xc20803de00)
/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/retrieval/discovery/file.go:215 +0x16f
_/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/retrieval/discovery.(*FileDiscovery).Run(0xc20803de00, 0xc208054780, 0xc2080547e0)
/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/retrieval/discovery/file.go:131 +0x2fe
created by _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/retrieval/discovery.testFileSD
/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/retrieval/discovery/file_test.go:32 +0x282

goroutine 12 [chan send]:
gopkg.in/fsnotify%2ev1.(*Watcher).readEvents(0xc2080549c0)
/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/.build/root/gopath/src/gopkg.in/fsnotify.v1/kqueue.go:301 +0x711
created by gopkg.in/fsnotify%2ev1.NewWatcher
/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/.build/root/gopath/src/gopkg.in/fsnotify.v1/kqueue.go:61 +0x31f

goroutine 13 [chan receive]:
_/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/retrieval/discovery.func·006()
/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/retrieval/discovery/file_test.go:85 +0x6d
created by _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/retrieval/discovery.testFileSD
/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/retrieval/discovery/file_test.go:93 +0xa47
FAIL _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/retrieval/discovery 15.054s
? _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/retrieval/discovery/kubernetes [no test files]
? _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/retrieval/discovery/marathon [no test files]
ok _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/rules 0.059s
? _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/storage [no test files]
ok _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/storage/local 3.699s
ok _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/storage/local/codable 0.007s
? _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/storage/local/index [no test files]
ok _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/storage/metric 0.007s
ok _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/storage/remote 0.015s
ok _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/storage/remote/influxdb 0.013s
ok _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/storage/remote/opentsdb 0.011s
ok _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/template 0.040s
? _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/util/cli [no test files]
ok _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/util/flock 0.007s
? _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/util/httputil [no test files]
? _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/util/route [no test files]
? _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/util/stats [no test files]
? _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/util/strutil [no test files]
? _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/util/testutil [no test files]
? _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/version [no test files]
ok _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/web 0.015s
ok _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/web/api/legacy 0.037s
ok _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/web/api/v1 0.023s
? _/Users/swsnider/Documents/Projects/Code/src/github.com/prometheus/prometheus/web/blob [no test files]
make: *** [test] Error 1

[fabxc]

File watch library biting us again by just not wanting to finish its Close() occasionally:
https://github.com/prometheus/prometheus/blob/master/retrieval/discovery/file.go#L203-L215

Will look into it.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.