panic after upgrade from 0.12 to 0.13.0

[rubycut]

Bug report

System info: influxdb 0.13, 64-bit debian wheezy, prebuilt package

Steps to reproduce:

1. Upgrade influxdb from 0.12.2 to 0.13
2. Put influxdb under decent load with auth enabled. (8k write points per second).

Expected behavior: should work same as previous version

Actual behavior: panic

4 0 - Go-http-client/1.1 9ee3bd1a-27da-11e6-8045-000000000000 604.51962ms
fatal error: concurrent map writes

goroutine 7153 [running]:
runtime.throw(0xcd5a80, 0x15)
        /usr/local/go/src/runtime/panic.go:547 +0x90 fp=0xca3ca810e0 sp=0xca3ca810c8
runtime.mapassign1(0xa19f00, 0xc820181ad0, 0xca3ca81248, 0xca3ca812c8)
        /usr/local/go/src/runtime/hashmap.go:445 +0xb1 fp=0xca3ca81188 sp=0xca3ca810e0
github.com/influxdata/influxdb/services/meta.(*Client).Authenticate(0xc8200769c0, 0xc9ebbf2420, 0xe, 0xc9ebbf242f, 0x20, 0x0, 0x0, 0x0)
        /root/go/src/github.com/influxdata/influxdb/services/meta/client.go:603 +0x5d2 fp=0xca3ca81350 sp=0xca3ca81188
github.com/influxdata/influxdb/services/httpd.authenticate.func1(0x7fbdcd5b0268, 0xc95b47bae0, 0xc9ac20d880)
        /root/go/src/github.com/influxdata/influxdb/services/httpd/handler.go:666 +0x2b2 fp=0xca3ca81420 sp=0xca3ca81350
net/http.HandlerFunc.ServeHTTP(0xc820268b60, 0x7fbdcd5b0268, 0xc95b47bae0, 0xc9ac20d880)
        /usr/local/go/src/net/http/server.go:1618 +0x3a fp=0xca3ca81440 sp=0xca3ca81420
github.com/influxdata/influxdb/services/httpd.gzipFilter.func1(0x7fbdcd5b0190, 0xc95b47baa0, 0xc9ac20d880)
        /root/go/src/github.com/influxdata/influxdb/services/httpd/handler.go:705 +0x2af fp=0xca3ca81500 sp=0xca3ca81440
net/http.HandlerFunc.ServeHTTP(0xc820268b80, 0x7fbdcd5b0190, 0xc95b47baa0, 0xc9ac20d880)
        /usr/local/go/src/net/http/server.go:1618 +0x3a fp=0xca3ca81520 sp=0xca3ca81500
github.com/influxdata/influxdb/services/httpd.versionHeader.func1(0x7fbdcd5b0190, 0xc95b47baa0, 0xc9ac20d880)
        /root/go/src/github.com/influxdata/influxdb/services/httpd/handler.go:714 +0xc0 fp=0xca3ca81568 sp=0xca3ca81520
net/http.HandlerFunc.ServeHTTP(0xc820268ba0, 0x7fbdcd5b0190, 0xc95b47baa0, 0xc9ac20d880)
        /usr/local/go/src/net/http/server.go:1618 +0x3a fp=0xca3ca81588 sp=0xca3ca81568
github.com/influxdata/influxdb/services/httpd.cors.func1(0x7fbdcd5b0190, 0xc95b47baa0, 0xc9ac20d880)
        /root/go/src/github.com/influxdata/influxdb/services/httpd/handler.go:752 +0x406 fp=0xca3ca816f8 sp=0xca3ca81588
net/http.HandlerFunc.ServeHTTP(0xc820268bc0, 0x7fbdcd5b0190, 0xc95b47baa0, 0xc9ac20d880)
        /usr/local/go/src/net/http/server.go:1618 +0x3a fp=0xca3ca81718 sp=0xca3ca816f8
github.com/influxdata/influxdb/services/httpd.requestID.func1(0x7fbdcd5b0190, 0xc95b47baa0, 0xc9ac20d880)
        /root/go/src/github.com/influxdata/influxdb/services/httpd/handler.go:762 +0x15d fp=0xca3ca81780 sp=0xca3ca81718
net/http.HandlerFunc.ServeHTTP(0xc820268be0, 0x7fbdcd5b0190, 0xc95b47baa0, 0xc9ac20d880)
        /usr/local/go/src/net/http/server.go:1618 +0x3a fp=0xca3ca817a0 sp=0xca3ca81780

Additional info:

It happens sometimes only, sometimes it starts without problems.

[rubycut]

Panic will happen sooner or later. Maybe not immediately after server starts listening.

[rubycut]

https://golang.org/doc/go1.6#runtime

The runtime has added lightweight, best-effort detection of concurrent misuse of maps. As always, if one goroutine is writing to a map, no other goroutine should be reading or writing the map concurrently. If the runtime detects this condition, it prints a diagnosis and crashes the program. The best way to find out more about the problem is to run the program under the race detector, which will more reliably identify the race and give more detail.

[rubycut]

https://github.com/influxdata/influxdb/blob/master/services/meta/client.go#L603

authCache map is changed without calling Lock first, rather RLock was called.

[rubycut]

I fixed it. I'll submit patch this evening.

[jwilder]

Fixed via #6769