Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: benchmark performance misaligned for fast tests #30928

Open
kokes opened this issue Mar 19, 2019 · 2 comments

Comments

@kokes
Copy link

@kokes kokes commented Mar 19, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12.1 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ondrej.kokes/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/ondrej.kokes/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12.1/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12.1/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/mk/dqryy0_947936yjsdw8jl6v00000gp/T/go-build420651873=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I ran some fast tests, where each iteration took less than a nanosecond. This led to the results table to be misaligned.

Here's a reproducible example:

package foo

import (
	"testing"
	"time"
)

func BenchmarkFoo(b *testing.B) {
	for j := 0; j < b.N; j++ {
		time.Sleep(time.Millisecond)
	}
}

func BenchmarkBar(b *testing.B) {
	for j := 0; j < b.N; j++ {

	}
}

What did you expect to see?

goos: darwin
goarch: amd64
BenchmarkFoo-4   	    1000	   1357941 ns/op
BenchmarkBar-4        2000000000	      0.32 ns/op

What did you see instead?

goos: darwin
goarch: amd64
BenchmarkFoo-4   	    1000	   1357941 ns/op
BenchmarkBar-4   	2000000000	         0.32 ns/op

Thoughts

I guess this has to do with performance numbers being printed as %8d, so anything larger than that will get shifted.

@katiehockman katiehockman added this to the Unplanned milestone Mar 22, 2019
@katiehockman

This comment has been minimized.

Copy link
Member

@katiehockman katiehockman commented Mar 22, 2019

/cc @mpvl
This seems like a small bug, I'll leave it up to you on whether or not this should/can be fixed easily.

@cespare

This comment has been minimized.

Copy link
Contributor

@cespare cespare commented Mar 22, 2019

Looks like the ns/op are aligned in the example (by decimal point).

The iteration counts don't look great, though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.