testing: benchmark verbose output changed in 1.14

[robaho]

Using github.com/robaho/fixed and go bench:

In 1.9.7 results are similar to:

goos: darwin
goarch: amd64
pkg: github.com/robaho/fixed
BenchmarkAddFixed-8         	2000000000	         1.26 ns/op
BenchmarkAddDecimal-8       	 5000000	       322 ns/op
BenchmarkAddBigInt-8        	100000000	        14.5 ns/op
BenchmarkAddBigFloat-8      	20000000	        76.5 ns/op
BenchmarkMulFixed-8         	300000000	         4.24 ns/op
BenchmarkMulDecimal-8       	20000000	        69.5 ns/op
BenchmarkMulBigInt-8        	100000000	        17.0 ns/op
BenchmarkMulBigFloat-8      	50000000	        34.4 ns/op
BenchmarkDivFixed-8         	300000000	         4.59 ns/op
BenchmarkDivDecimal-8       	 2000000	       854 ns/op
BenchmarkDivBigInt-8        	30000000	        54.7 ns/op
BenchmarkDivBigFloat-8      	20000000	       118 ns/op

and in 1.14

goos: darwin
goarch: amd64
pkg: github.com/robaho/fixed
BenchmarkAddFixed
BenchmarkAddFixed-8         	1000000000	         0.519 ns/op
BenchmarkAddDecimal
BenchmarkAddDecimal-8       	 5005356	       236 ns/op
BenchmarkAddBigInt
BenchmarkAddBigInt-8        	82658592	        14.9 ns/op
BenchmarkAddBigFloat
BenchmarkAddBigFloat-8      	15501452	        77.2 ns/op
BenchmarkMulFixed
BenchmarkMulFixed-8         	293514025	         4.02 ns/op
BenchmarkMulDecimal
BenchmarkMulDecimal-8       	16031581	        72.6 ns/op
BenchmarkMulBigInt
BenchmarkMulBigInt-8        	70126652	        17.7 ns/op
BenchmarkMulBigFloat
BenchmarkMulBigFloat-8      	33527037	        35.1 ns/op
BenchmarkDivFixed
BenchmarkDivFixed-8         	265002393	         4.53 ns/op

Notice the extra rows without the PROCs count.

[bcmills]

Please fill out the complete issue template. What was the exact command? What does go env report?

[robaho]

Sorry, I thought it was a trivial report. I run the tests via Intellij so I didn't change any parameters - just the GOROOT for the project.

In 1.9.7 the command is

GOROOT=/usr/local/Cellar/go@1.9/1.9.7/libexec #gosetup
GOPATH=/Users/robertengels/go #gosetup
/usr/local/Cellar/go@1.9/1.9.7/libexec/bin/go test -c -o /private/var/folders/bb/dk8qf0zx6zg7kw3m6dgmj40h0000gn/T/___gobench_fixed_bench_test_go github.com/robaho/fixed #gosetup
/private/var/folders/bb/dk8qf0zx6zg7kw3m6dgmj40h0000gn/T/___gobench_fixed_bench_test_go -test.v -test.bench "^BenchmarkAddFixed|BenchmarkAddDecimal|BenchmarkAddBigInt|BenchmarkAddBigFloat|BenchmarkMulFixed|BenchmarkMulDecimal|BenchmarkMulBigInt|BenchmarkMulBigFloat|BenchmarkDivFixed|BenchmarkDivDecimal|BenchmarkDivBigInt|BenchmarkDivBigFloat|BenchmarkCmpFixed|BenchmarkCmpDecimal|BenchmarkCmpBigInt|BenchmarkCmpBigFloat|BenchmarkStringFixed|BenchmarkStringNFixed|BenchmarkStringDecimal|BenchmarkStringBigInt|BenchmarkStringBigFloat|BenchmarkWriteTo$" -test.run ^$ #gosetup

in 1.14 the command is

GOROOT=/usr/local/Cellar/go/1.14/libexec #gosetup
GOPATH=/Users/robertengels/go #gosetup
/usr/local/Cellar/go/1.14/libexec/bin/go test -c -o /private/var/folders/bb/dk8qf0zx6zg7kw3m6dgmj40h0000gn/T/___gobench_fixed_bench_test_go github.com/robaho/fixed #gosetup
/private/var/folders/bb/dk8qf0zx6zg7kw3m6dgmj40h0000gn/T/___gobench_fixed_bench_test_go -test.v -test.bench "^BenchmarkAddFixed|BenchmarkAddDecimal|BenchmarkAddBigInt|BenchmarkAddBigFloat|BenchmarkMulFixed|BenchmarkMulDecimal|BenchmarkMulBigInt|BenchmarkMulBigFloat|BenchmarkDivFixed|BenchmarkDivDecimal|BenchmarkDivBigInt|BenchmarkDivBigFloat|BenchmarkCmpFixed|BenchmarkCmpDecimal|BenchmarkCmpBigInt|BenchmarkCmpBigFloat|BenchmarkStringFixed|BenchmarkStringNFixed|BenchmarkStringDecimal|BenchmarkStringBigInt|BenchmarkStringBigFloat|BenchmarkWriteTo$" -test.run ^$ #gosetup

[robaho]

The output of 'go env' is:

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/robertengels/Library/Caches/go-build"
GOENV="/Users/robertengels/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/robertengels/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.14/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.14/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/bb/dk8qf0zx6zg7kw3m6dgmj40h0000gn/T/go-build436672849=/tmp/go-build -gno-record-gcc-switches -fno-common"

[robaho]

Interesting too that the 'operations' counts are not whole numbers in 1.14

[mengzhuo]

You have to remove -v

#24929
a813d3c7

[bcmills]

@robaho, it seems like the extra lines here are intended to aid in debugging if one of the benchmarks unexpectedly stalls. Other than being different, are they causing any particular harm?