perf: Fanout for async updates in global

[miparnisari]

This is a follow-up of #111 which was abandoned.

I don't know if the existing benchmarks would cover this code, but:

With this PR:

[26/10/23 12:27:25] ~/GitHub/gubernator (concurrent-updates) $ go test ./... -bench . -benchtime 5s -timeout 0 -run=XXX -cpu 1 -benchmem
goos: darwin
goarch: amd64
pkg: github.com/mailgun/gubernator/v2
cpu: Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz
BenchmarkReplicatedConsistantHash/fasthash/fnv1a                100000000               57.23 ns/op            0 B/op          0 allocs/op
BenchmarkReplicatedConsistantHash/fasthash/fnv1                 153649881               41.15 ns/op            0 B/op          0 allocs/op
BenchmarkCache/LRUCache/Sequential_reads                        19600915               424.4 ns/op            23 B/op          1 allocs/op
BenchmarkCache/LRUCache/Sequential_writes                        8498610               824.4 ns/op           129 B/op          4 allocs/op
BenchmarkCache/LRUCache/Concurrent_reads                         5372332              1248 ns/op              48 B/op          3 allocs/op
BenchmarkCache/LRUCache/Concurrent_writes                        3417484              1737 ns/op             154 B/op          5 allocs/op
BenchmarkServer/GetPeerRateLimit()_with_no_batching                37059            167426 ns/op           15021 B/op        263 allocs/op
BenchmarkServer/GetRateLimit()                                      6613           1003143 ns/op           33625 B/op        536 allocs/op
BenchmarkServer/GetRateLimitGlobal()                               20452            254354 ns/op           29986 B/op        462 allocs/op
BenchmarkServer/HealthCheck                                        53686            117003 ns/op           13312 B/op        228 allocs/op
BenchmarkServer/Thundering_herd                                    39678            165926 ns/op           24575 B/op        387 allocs/op
BenchmarkLRUCache/Sequential_reads                              13612419               901.9 ns/op            23 B/op          1 allocs/op
BenchmarkLRUCache/Sequential_writes                              6430135               921.2 ns/op           129 B/op          4 allocs/op
BenchmarkLRUCache/Concurrent_reads                               5914514              3509 ns/op              16 B/op          1 allocs/op
BenchmarkLRUCache/Concurrent_writes                              5376588              1114 ns/op             121 B/op          3 allocs/op
BenchmarkLRUCache/Concurrent_reads_and_writes_of_existing_keys           3086575              3945 ns/op             135 B/op          3 allocs/op
BenchmarkLRUCache/Concurrent_reads_and_writes_of_non-existent_keys       2330882              2804 ns/op             163 B/op          7 allocs/op
PASS
ok      github.com/mailgun/gubernator/v2        375.087s
PASS
ok      github.com/mailgun/gubernator/v2/cluster        1.404s
?       github.com/mailgun/gubernator/v2/cmd/gubernator [no test files]
?       github.com/mailgun/gubernator/v2/cmd/gubernator-cli     [no test files]
?       github.com/mailgun/gubernator/v2/cmd/gubernator-cluster [no test files]
?       github.com/mailgun/gubernator/v2/cmd/healthcheck        [no test files]
[26/10/23 12:34:52] ~/GitHub/gubernator (concurrent-updates) $ 

Before this PR:

[26/10/23 12:35:26] ~/GitHub/gubernator (master) $ go test ./... -bench . -benchtime 5s -timeout 0 -run=XXX -cpu 1 -benchmem
goos: darwin
goarch: amd64
pkg: github.com/mailgun/gubernator/v2
cpu: Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz
BenchmarkReplicatedConsistantHash/fasthash/fnv1a                100000000               50.50 ns/op            0 B/op          0 allocs/op
BenchmarkReplicatedConsistantHash/fasthash/fnv1                 143573410               43.39 ns/op            0 B/op          0 allocs/op
BenchmarkCache/LRUCache/Sequential_reads                        17901451               439.6 ns/op            23 B/op          1 allocs/op
BenchmarkCache/LRUCache/Sequential_writes                        7545012               821.0 ns/op           129 B/op          4 allocs/op
BenchmarkCache/LRUCache/Concurrent_reads                         5261803              1384 ns/op              48 B/op          3 allocs/op
BenchmarkCache/LRUCache/Concurrent_writes                        3382041              1865 ns/op             154 B/op          5 allocs/op
BenchmarkServer/GetPeerRateLimit()_with_no_batching                40202            162941 ns/op           15044 B/op        263 allocs/op
BenchmarkServer/GetRateLimit()                                      5414           1056861 ns/op           33663 B/op        545 allocs/op
BenchmarkServer/GetRateLimitGlobal()                               19072            306601 ns/op           30463 B/op        470 allocs/op
BenchmarkServer/HealthCheck                                        42639            161470 ns/op           13307 B/op        228 allocs/op
BenchmarkServer/Thundering_herd                                    35431            190765 ns/op           24832 B/op        391 allocs/op
BenchmarkLRUCache/Sequential_reads                              13398789               982.5 ns/op            23 B/op          1 allocs/op
BenchmarkLRUCache/Sequential_writes                              6833612               965.7 ns/op           129 B/op          4 allocs/op
BenchmarkLRUCache/Concurrent_reads                               5432090              2260 ns/op              16 B/op          1 allocs/op
BenchmarkLRUCache/Concurrent_writes                              4194040              1260 ns/op             122 B/op          3 allocs/op
BenchmarkLRUCache/Concurrent_reads_and_writes_of_existing_keys           2714426              2149 ns/op             135 B/op          3 allocs/op
BenchmarkLRUCache/Concurrent_reads_and_writes_of_non-existent_keys       2504059              2326 ns/op             163 B/op          7 allocs/op
PASS
ok      github.com/mailgun/gubernator/v2        372.707s
PASS
ok      github.com/mailgun/gubernator/v2/cluster        1.100s
?       github.com/mailgun/gubernator/v2/cmd/gubernator [no test files]
?       github.com/mailgun/gubernator/v2/cmd/gubernator-cli     [no test files]
?       github.com/mailgun/gubernator/v2/cmd/gubernator-cluster [no test files]
?       github.com/mailgun/gubernator/v2/cmd/healthcheck        [no test files]
[26/10/23 12:41:53] ~/GitHub/gubernator (master) $ 

[Baliedge]

Great finding and good solution! Just have a recommendation for you.

[Baliedge]

Looks good, just one detail left.

[miparnisari]

Applied! Thanks