x/build/coordinator: negative waitgroup counter panic

[bradfitz]

The coordinator was seen crashing with:

]: panic: sync: negative WaitGroup counter
]: goroutine 295 [running]:
]: sync.(*WaitGroup).Add(0xc820421430, 0xffffffffffffffff)
]: /home/bradfitz/go/src/sync/waitgroup.go:70 +0xd0
]: sync.(*WaitGroup).Done(0xc820421430)
]: /home/bradfitz/go/src/sync/waitgroup.go:95 +0x2a
]: main.(*buildStatus).runTests.func1(0xc820421430, 0xc8206c2e00, 0xc82017cea0, 0xc820421420, 0xb)
]: /home/bradfitz/src/golang.org/x/build/cmd/coordinator/coordinator.go:2030 +0x2a8
]: created by main.(*buildStatus).runTests
]: /home/bradfitz/src/golang.org/x/build/cmd/coordinator/coordinator.go:2030 +0x789
]: goroutine 1 [select]:
]: main.main()
]: /home/bradfitz/src/golang.org/x/build/cmd/coordinator/coordinator.go:317 +0xff3
]: goroutine 17 [syscall, 10 minutes, locked to thread]:
]: runtime.goexit()
]: /home/bradfitz/go/src/runtime/asm_amd64.s:1745 +0x1
]: goroutine 23 [IO wait, 3 minutes]:
]: net.runtime_pollWait(0x7faf85820900, 0x72, 0xc82000a150)
]: /home/bradfitz/go/src/runtime/netpoll.go:157 +0x60
]: net.(*pollDesc).Wait(0xc8204a66f0, 0x72, 0x0, 0x0)
]: /home/bradfitz/go/src/net/fd_poll_runtime.go:73 +0x3a
]: net.(*pollDesc).WaitRead(0xc8204a66f0, 0x0, 0x0)
]: /home/bradfitz/go/src/net/fd_poll_runtime.go:78 +0x36
]: net.(*netFD).accept(0xc8204a6690, 0x0, 0x7faf8582de88, 0xc8203e6780)
]: /home/bradfitz/go/src/net/fd_unix.go:408 +0x27c
]: net.(*TCPListener).AcceptTCP(0xc82001e400, 0x4ab160, 0x0, 0x0)
]: /home/bradfitz/go/src/net/tcpsock_posix.go:254 +0x4d
]: net/http.tcpKeepAliveListener.Accept(0xc82001e400, 0x0, 0x0, 0x0, 0x0)
]: /home/bradfitz/go/src/net/http/server.go:2149 +0x41
]: net/http.(*Server).Serve(0xc8204a4a20, 0x7faf8582de50, 0xc82001e400, 0x0, 0x0)
]: /home/bradfitz/go/src/net/http/server.go:1900 +0xa4
]: net/http.(*Server).ListenAndServe(0xc8204a4a20, 0x0, 0x0)
]: /home/bradfitz/go/src/net/http/server.go:1889 +0x136
]: net/http.ListenAndServe(0xb343f0, 0x3, 0x7faf8582ddf0, 0xead290, 0x0, 0x0)
]: /home/bradfitz/go/src/net/http/server.go:1979 +0x94
]: main.main.func1()
]: /home/bradfitz/src/golang.org/x/build/cmd/coordinator/coordinator.go:284 +0xea
]: created by main.main

/cc @aclements @adg