cmd/go: go test running bench twice with -benchtime=1x #32051
Comments
Working as intended. The benchmark support code calls each benchmark function as many times as necessary to establish a reliable performance figure. |
By the way, you're using the benchtime flag incorrectly. I'm surprised you didn't see an error. Its argument is a duration. |
As of Go 1.12 (https://golang.org/doc/go1.12#testing):
|
benchtime=1x is a new feature in 1.12. According to doc, 100x means 100 times. BTW, before benchtime=1x, I used to use -benchtime=0s to trick go test to only run my test case once. 1.12 will error, stating 0s is not a valid duration. -benchtime=1ns still does the trick. |
Looking at the code, it looks like we always call the benchmark once to initialize things (for example, to discover sub-benchmarks). Then we run it n times after that. Perhaps we just need to subtract 1 from n. In any case, it does seem confusing that 1x runs it twice. (And I suspect that 1 is the most common number to use with |
@rsc for a decision on behavior. |
Well, benchtime needs some documentation then. The testing package doesn't mention it in the comment with the other flags. Not a fan of this hack, but I obviously missed it when it went in. |
#24735 has the context for the -benchtime=Nx syntax.
As far as I can see, the testing package documentation only mentions three of the testing flags: |
Maybe that's enough, but I must say that 'go doc testing' is a lot easier to handle than the output of the go command's own documentation. |
- Skip the bazel clean command on the last run of the benchmark. - Use --test.benchtime=1ns to force running the benchmark once (golang/go#32051) PiperOrigin-RevId: 347095825
- Skip the bazel clean command on the last run of the benchmark. - Use --test.benchtime=1ns to force running the benchmark once (golang/go#32051) PiperOrigin-RevId: 347095825
- Skip the bazel clean command on the last run of the benchmark. - Use --test.benchtime=1ns to force running the benchmark once (golang/go#32051) PiperOrigin-RevId: 347095825
- Skip the bazel clean command on the last run of the benchmark. - Use --test.benchtime=1ns to force running the benchmark once (golang/go#32051) PiperOrigin-RevId: 347124606
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
ftian@ftian-xps:~/tmp/x$ cat xxx_test.go
package x
import (
"fmt"
"testing"
)
func BenchmarkOnce(b *testing.B) {
fmt.Printf("Calling me once.\n")
}
ftian@ftian-xps:~/tmp/x$ go test -run=NONONO -bench=Once -benchtime=1x
Calling me once.
goos: linux
goarch: amd64
BenchmarkOnce-12 Calling me once.
1 17290 ns/op
PASS
ok _/home/ftian/tmp/x 0.002s
What did you expect to see?
The BencharkOnce should be run only once. The line should be printed once.
What did you see instead?
Calling me once. printed twice.
The text was updated successfully, but these errors were encountered: