Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/perf/benchstat: no group headers shown if only one group #33169

Open
shawn-xdji opened this issue Jul 18, 2019 · 2 comments

Comments

@shawn-xdji
Copy link

commented Jul 18, 2019

What version of Go are you using (go version)?

$ go version
go version devel +a6a7b148f8 Mon Jul 15 23:00:52 2019 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/xiaji01/.cache/go-build"
GOENV="/home/xiaji01/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/xiaji01/.go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/mnt/share/homes/xiaji01/go1.12/x86"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/mnt/share/homes/xiaji01/go1.12/x86/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build378631038=/tmp/go-build -gno-record-gcc-switches"

What did you do?

#install benchstat
go get -d golang.org/x/perf/cmd/benchstat
go install golang.org/x/perf/cmd/benchstat

#run benchmark of a SINGLE package under /src
go test -count 5 -timeout 20m -run=^$ -bench=. crypto/sha256 | tee bench.log

#run benchstat to do analysis, using default options
benchstat bench.log

#output of the benchstat doesn't have group header (pkg, goos, goarch)
name time/op
Hash8Bytes-40 294ns ± 3%
Hash1K-40 3.50µs ± 5%
Hash8K-40 26.2µs ± 4%

name speed
Hash8Bytes-40 27.2MB/s ± 3%
Hash1K-40 293MB/s ± 4%
Hash8K-40 313MB/s ± 4%

What did you expect to see?

Expect benchstat to output group header even the benchmarking data only contains one group.
Since there are a few Go benchmarks from different packages sharing the same name, for example, both crypto/sha256 and crypto/sha512 have benchmarks "BenchmarkHash8Bytes", "BenchmarkHash1K", etc., having a package name, which is part of the group header by default, would facilitate recording their performance data in a database.

We can fulfill the above requirement via multiple ways, just wondering if having benchstat support it is a better option.

The restriction was introduced in commit add18dd08e6be79a3a9beb0d810efac191ce1eae,

  •                           if len(c.Groups) > 1 {
    
  •                                   // Show group headers if there is more than one group.
    
  •                                   row.Group = key.Group
    

Any comment is highly appreciated.

Attaching two benchmark log files for your reference.

bench_one_pkg.log: benchmark log of package crypto/sha256, benchstat's output has no group

bench_two_pkgs.log: benchmark log of packages crypto/sha256 and crypto/sha512, benchstat's output has groups.
bench_one_pkg.log
bench_two_pkgs.log

What did you see instead?

@gopherbot gopherbot added this to the Unreleased milestone Jul 18, 2019

@katiehockman

This comment has been minimized.

Copy link
Contributor

commented Jul 29, 2019

/cc @rsc

@katiehockman

This comment has been minimized.

Copy link
Contributor

commented Jul 30, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.