Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates for go1.21 rand changes #28

Merged
merged 4 commits into from
Aug 13, 2023
Merged

Updates for go1.21 rand changes #28

merged 4 commits into from
Aug 13, 2023

Conversation

mroth
Copy link
Owner

@mroth mroth commented Aug 7, 2023

Work in progress

Fixes #26

As discussed in #26, the changes in go1.21 will remove rand contention
when the (now deprecated) rand.Seed() is not called.

In order to check if this brings performance parity, start to benchmark
both Pick and PickSource functions in parallel, and no longer rand.Seed
in test init.
@mroth
Copy link
Owner Author

mroth commented Aug 7, 2023

Benchmarks using above commit, showing PickSource still quite necessary on go1.20

$ go version
go version go1.20.7 darwin/arm64
$ go test -bench=Parallel$ -cpu 8 -count 10 > go120.txt
$ benchstat -col=".name" go120.txt
goos: darwin
goarch: arm64
pkg: github.com/mroth/weightedrand/v2
            │ PickParallel  │          PickSourceParallel          │
            │    sec/op     │    sec/op     vs base                │
*/10-8        117.550n ± 1%   2.479n ±  8%  -97.89% (p=0.000 n=10)
*/100-8       125.250n ± 2%   4.143n ±  4%  -96.69% (p=0.000 n=10)
*/1000-8      131.000n ± 3%   5.942n ±  2%  -95.46% (p=0.000 n=10)
*/10000-8     154.100n ± 3%   8.178n ± 18%  -94.69% (p=0.000 n=10)
*/100000-8     180.90n ± 1%   11.18n ±  0%  -93.82% (p=0.000 n=10)
*/1000000-8    221.20n ± 1%   14.51n ±  0%  -93.44% (p=0.000 n=10)
geomean         151.1n        6.578n        -95.65%

On go1.21, the difference mostly negligible (some minor noise in the tests here due to running on laptop):

$ go1.21.0 version
go version go1.21.0 darwin/arm64
go1.21.0 test -bench=Parallel$ -cpu 8 -count 10 > go121.txt
$ benchstat -col=".name" go121.txt
goos: darwin
goarch: arm64
pkg: github.com/mroth/weightedrand/v2
            │ PickParallel │         PickSourceParallel         │
            │    sec/op    │   sec/op     vs base               │
*/10-8         2.620n ± 5%   2.449n ± 2%  -6.55% (p=0.002 n=10)
*/100-8        4.227n ± 1%   4.142n ± 1%  -2.01% (p=0.000 n=10)
*/1000-8       5.900n ± 0%   5.878n ± 0%  -0.36% (p=0.001 n=10)
*/10000-8      7.784n ± 0%   7.986n ± 0%  +2.60% (p=0.000 n=10)
*/100000-8     10.99n ± 0%   11.14n ± 0%  +1.36% (p=0.000 n=10)
*/1000000-8    14.43n ± 1%   14.52n ± 0%  +0.59% (p=0.013 n=10)
geomean        6.573n        6.522n       -0.77%

Direct comparison of go1.20 to go1.21:

$ benchstat go120.txt go121.txt
goos: darwin
goarch: arm64
pkg: github.com/mroth/weightedrand/v2
                             │   go120.txt    │              go121.txt              │
                             │     sec/op     │   sec/op     vs base                │
PickParallel/10-8              117.550n ±  1%   2.620n ± 5%  -97.77% (p=0.000 n=10)
PickParallel/100-8             125.250n ±  2%   4.227n ± 1%  -96.63% (p=0.000 n=10)
PickParallel/1000-8            131.000n ±  3%   5.900n ± 0%  -95.50% (p=0.000 n=10)
PickParallel/10000-8           154.100n ±  3%   7.784n ± 0%  -94.95% (p=0.000 n=10)
PickParallel/100000-8           180.90n ±  1%   10.99n ± 0%  -93.92% (p=0.000 n=10)
PickParallel/1000000-8          221.20n ±  1%   14.43n ± 1%  -93.48% (p=0.000 n=10)
PickSourceParallel/10-8          2.479n ±  8%   2.449n ± 2%        ~ (p=0.190 n=10)
PickSourceParallel/100-8         4.143n ±  4%   4.142n ± 1%        ~ (p=0.684 n=10)
PickSourceParallel/1000-8        5.942n ±  2%   5.878n ± 0%   -1.08% (p=0.000 n=10)
PickSourceParallel/10000-8       8.178n ± 18%   7.986n ± 0%   -2.34% (p=0.007 n=10)
PickSourceParallel/100000-8      11.18n ±  0%   11.14n ± 0%   -0.36% (p=0.005 n=10)
PickSourceParallel/1000000-8     14.51n ±  0%   14.52n ± 0%        ~ (p=0.806 n=10)
geomean                          31.52n         6.548n       -79.23%

@mroth
Copy link
Owner Author

mroth commented Aug 12, 2023

A search of the dependents for this module reveal that many who use PickSource are using it incorrectly, so in order to simplify the surface area going to just go ahead and mark as deprecated as of go1.21 rather than waiting another cycle for go1.22.

There is still a potential valid use case for wanting to provide a custom randomness source, but would rather get it out of the API for now, and revisit if it can be done a more elegant way with the proposed math/rand/v2 in discussion.

results on my Apple M2 Pro laptop:

$ make analysis
benchstat -filter=".name:Multiple" -col="/lib" -row="/size" -table="/concurrency" results.txt
/concurrency: single
        │    randutil     │             weightedrand             │
        │     sec/op      │   sec/op     vs base                 │
1e1          455.70n ± 0%   19.16n ± 2%   -95.80% (p=0.000 n=10)
1e2          438.20n ± 0%   32.81n ± 0%   -92.51% (p=0.000 n=10)
1e3         1179.00n ± 0%   46.00n ± 0%   -96.10% (p=0.000 n=10)
1e4         6591.00n ± 1%   60.34n ± 0%   -99.08% (p=0.000 n=10)
1e5        61188.00n ± 0%   84.83n ± 0%   -99.86% (p=0.000 n=10)
1e6        631928.5n ± 1%   112.6n ± 2%   -99.98% (p=0.000 n=10)
1e7       6402254.0n ± 1%   240.7n ± 0%  -100.00% (p=0.000 n=10)
geomean       16.84µ        63.16n        -99.62%

/concurrency: parallel
        │     randutil     │             weightedrand             │
        │      sec/op      │   sec/op     vs base                 │
1e1          589.300n ± 0%   2.026n ± 1%   -99.66% (p=0.000 n=10)
1e2          490.000n ± 0%   3.337n ± 1%   -99.32% (p=0.000 n=10)
1e3          766.500n ± 0%   4.518n ± 1%   -99.41% (p=0.000 n=10)
1e4          808.700n ± 0%   5.951n ± 2%   -99.26% (p=0.000 n=10)
1e5         6438.500n ± 1%   8.184n ± 1%   -99.87% (p=0.000 n=10)
1e6        108285.50n ± 1%   12.00n ± 1%   -99.99% (p=0.000 n=10)
1e7       2011263.00n ± 2%   28.95n ± 1%  -100.00% (p=0.000 n=10)
geomean        5.907µ        6.548n        -99.89%
key-value format is nicer for later filtering in benchstat
@mroth mroth marked this pull request as ready for review August 13, 2023 16:36
@mroth mroth merged commit 38f771d into main Aug 13, 2023
9 checks passed
@mroth mroth deleted the go1.21 branch August 13, 2023 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

update docs regarding PickSource for go1.21
1 participant