Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: testing: add CPU name to standard benchmark labels #39214

Open
martisch opened this issue May 22, 2020 · 5 comments
Open

proposal: testing: add CPU name to standard benchmark labels #39214

martisch opened this issue May 22, 2020 · 5 comments
Labels
Projects
Milestone

Comments

@martisch
Copy link
Member

@martisch martisch commented May 22, 2020

Currently only goarch and goos are printed as bechmark labels.

This proposal suggests to add the CPU name as cpu: to the labels printed once at the top of benchmark output.

This will only happen when internal/cpu is able to detect the cpu type and otherwise the cpu label will not be printed.

Example with prototype http://golang.org/cl/234977:

goos: darwin
goarch: amd64
pkg: strconv
cpu: Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz
BenchmarkAtof64Decimal-4        	24431032	        46.8 ns/op

Optionally disabled/enabled CPU features could be printed in addition:
features: aesni, avx, sse41 ...

This will help to understand, reproduce and document benchmark results better in context of different system configurations.

Some commit messages with benchmarks made by Go contributors already contain this information, but added manually:
https://go-review.googlesource.com/c/go/+/230737
https://go-review.googlesource.com/c/crypto/+/169037
https://go-review.googlesource.com/c/go/+/40693
https://go-review.googlesource.com/c/go/+/10367
https://go-review.googlesource.com/c/go/+/76470
https://go-review.googlesource.com/c/go/+/171736
https://go-review.googlesource.com/c/go/+/125316
https://go-review.googlesource.com/c/go/+/171731
https://go-review.googlesource.com/c/crypto/+/39693
....

Related Proposal: #28398

@gopherbot gopherbot added this to the Proposal milestone May 22, 2020
@gopherbot gopherbot added the Proposal label May 22, 2020
@mvdan
Copy link
Member

@mvdan mvdan commented May 22, 2020

Should we also print information about the memory and disk models/speeds? I imagine this will greatly depend on what the bottleneck for a benchmark is.

@martisch
Copy link
Member Author

@martisch martisch commented May 22, 2020

I would leave it at CPU information for now as this is a common factor for many microbenchmarks. For me this also identifies the system I was benchmarking on, which is helpful when examining logs of benchmarks later.

@mvdan
Copy link
Member

@mvdan mvdan commented May 22, 2020

I agree; I was thinking outloud more than anything. There could be many other factors at play too, such as what kernel version and if any special boot options were used (such as intel mitigations which could slow down syscalls).

@gopherbot
Copy link

@gopherbot gopherbot commented May 22, 2020

Change https://golang.org/cl/234977 mentions this issue: testing: print CPU name for benchmarks

@martisch
Copy link
Member Author

@martisch martisch commented May 22, 2020

@mvdan agreed. This could potentially be expanded in the future with a flag for printing even more information (ram, caches, kernel, ...) but I would leave that to a future extra and separate proposal.

The same information could then be easily added to e.g. go env or other commands with information for bug reporting if considered helpful.

@rsc rsc changed the title proposal: add CPU name to standard benchmark labels proposal: testing: add CPU name to standard benchmark labels Jun 10, 2020
@rsc rsc added this to Incoming in Proposals Jun 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Proposals
Incoming
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.