Data race detected in our race test

[jzelinskie]

While trying to setup GitHub pipelines, I randomly hit this:

$ go test -v -race $(go list ./...)
...
2020-02-13T22:45:50.4837092Z === RUN   TestStartStopRaceIssue437
2020-02-13T22:45:50.4837535Z time="2020-02-13T22:45:50Z" level=warning msg="falling back to default configuration" default=1024 name=memory.ShardCount provided=0
2020-02-13T22:45:50.4837991Z time="2020-02-13T22:45:50Z" level=warning msg="falling back to default configuration" default=3m0s name=memory.GarbageCollectionInterval provided=0s
2020-02-13T22:45:50.4838449Z time="2020-02-13T22:45:50Z" level=warning msg="falling back to default configuration" default=1s name=memory.PrometheusReportingInterval provided=0s
2020-02-13T22:45:50.4838897Z time="2020-02-13T22:45:50Z" level=warning msg="falling back to default configuration" default=30m0s name=memory.PeerLifetime provided=0s
2020-02-13T22:45:50.4839385Z time="2020-02-13T22:45:50Z" level=warning msg="UDP private key was not provided, using generated key" key=MwUbwAnHLThimnHh3G1rYqeB4mM9rcscMuzUE2Z3xSq6XEtWOT6xdgWUt4wAR6Qg
2020-02-13T22:45:50.4839829Z time="2020-02-13T22:45:50Z" level=warning msg="falling back to default configuration" default=100 name=udp.MaxNumWant provided=0
2020-02-13T22:45:50.4840268Z time="2020-02-13T22:45:50Z" level=warning msg="falling back to default configuration" default=50 name=udp.DefaultNumWant provided=0
2020-02-13T22:45:50.4840697Z time="2020-02-13T22:45:50Z" level=warning msg="falling back to default configuration" default=50 name=udp.MaxScrapeInfoHashes provided=0
2020-02-13T22:45:50.4840846Z ==================
2020-02-13T22:45:50.4840977Z WARNING: DATA RACE
2020-02-13T22:45:50.4841134Z Write at 0x00c0000ea298 by goroutine 45:
2020-02-13T22:45:50.4841263Z   sync.(*WaitGroup).Wait()
2020-02-13T22:45:50.4842146Z       /opt/hostedtoolcache/go/1.13.8/x64/src/internal/race/race.go:41 +0xee
2020-02-13T22:45:50.4842359Z   github.com/chihaya/chihaya/frontend/udp.(*Frontend).Stop.func1()
2020-02-13T22:45:50.4842574Z       /home/runner/work/chihaya/chihaya/frontend/udp/frontend.go:152 +0xd9
2020-02-13T22:45:50.4842646Z 
2020-02-13T22:45:50.4842813Z Previous read at 0x00c0000ea298 by goroutine 44:
2020-02-13T22:45:50.4842954Z   sync.(*WaitGroup).Add()
2020-02-13T22:45:50.4843139Z       /opt/hostedtoolcache/go/1.13.8/x64/src/internal/race/race.go:37 +0x18d
2020-02-13T22:45:50.4843427Z   github.com/chihaya/chihaya/frontend/udp.(*Frontend).serve()
2020-02-13T22:45:50.4843647Z       /home/runner/work/chihaya/chihaya/frontend/udp/frontend.go:174 +0x8c
2020-02-13T22:45:50.4843841Z   github.com/chihaya/chihaya/frontend/udp.NewFrontend.func2()
2020-02-13T22:45:50.4844027Z       /home/runner/work/chihaya/chihaya/frontend/udp/frontend.go:132 +0x3c
2020-02-13T22:45:50.4844111Z 
2020-02-13T22:45:50.4844261Z Goroutine 45 (running) created at:
2020-02-13T22:45:50.4844431Z   github.com/chihaya/chihaya/frontend/udp.(*Frontend).Stop()
2020-02-13T22:45:50.4844635Z       /home/runner/work/chihaya/chihaya/frontend/udp/frontend.go:149 +0xe3
2020-02-13T22:45:50.4844844Z   github.com/chihaya/chihaya/frontend/udp_test.TestStartStopRaceIssue437()
2020-02-13T22:45:50.4845056Z       /home/runner/work/chihaya/chihaya/frontend/udp/frontend_test.go:23 +0x4f9
2020-02-13T22:45:50.4845184Z   testing.tRunner()
2020-02-13T22:45:50.4845379Z       /opt/hostedtoolcache/go/1.13.8/x64/src/testing/testing.go:909 +0x199
2020-02-13T22:45:50.4845462Z 
2020-02-13T22:45:50.4845647Z Goroutine 44 (running) created at:
2020-02-13T22:45:50.4845823Z   github.com/chihaya/chihaya/frontend/udp.NewFrontend()
2020-02-13T22:45:50.4846027Z       /home/runner/work/chihaya/chihaya/frontend/udp/frontend.go:131 +0x40b
2020-02-13T22:45:50.4846237Z   github.com/chihaya/chihaya/frontend/udp_test.TestStartStopRaceIssue437()
2020-02-13T22:45:50.4846435Z       /home/runner/work/chihaya/chihaya/frontend/udp/frontend_test.go:19 +0x479
2020-02-13T22:45:50.4846578Z   testing.tRunner()
2020-02-13T22:45:50.4846775Z       /opt/hostedtoolcache/go/1.13.8/x64/src/testing/testing.go:909 +0x199
2020-02-13T22:45:50.4846913Z ==================
2020-02-13T22:45:50.4847308Z --- FAIL: TestStartStopRaceIssue437 (0.00s)
2020-02-13T22:45:50.4851841Z ##[error]    testing.go:853: race detected during execution of test
2020-02-13T22:45:50.4855353Z FAIL
2020-02-13T22:45:50.4855543Z FAIL	github.com/chihaya/chihaya/frontend/udp	0.023s
...

Full Logs

[mrd0ll4r]

I believe it wants us to call wg.Wait from the same goroutine from which we called the previous wg.Add's. We can do that, we already exit the loop in serve during shutdown, so after that we can wait for the outstanding requests. Then we could use a channel to signal to the stopper goroutine that we're done closing...