New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: tiny benchmark with StopTimer runs forever #27217

Open
peterGo opened this Issue Aug 25, 2018 · 2 comments

Comments

Projects
None yet
4 participants
@peterGo
Contributor

peterGo commented Aug 25, 2018

From at least Go 1,4 onward,

$ go version
go version devel +e03220a594 Sat Aug 25 02:39:49 2018 +0000 linux/amd64
$

If we run a tiny benchmark (count++) and use StopTimer() for initialization (count = 0) then the benchmark runs for a very long time, perhaps forever.

$ cat tiny_test.go
package main

import (
	"testing"
)

var count int64

func BenchmarkTiny(b *testing.B) {
	for n := 0; n < b.N; n++ {
		b.StopTimer()
		count = 0
		b.StartTimer()
		count++
	}
}
$
$ go test tiny_test.go -bench=.
goos: linux
goarch: amd64
BenchmarkTiny-4   	^Csignal: interrupt
FAIL	command-line-arguments	143.633s
$

If we comment out StopTimer() (//b.StopTimer()) then the benchmark quickly runs to completion.

$ cat tiny_test.go
package main

import (
	"testing"
)

var count int64

func BenchmarkTiny(b *testing.B) {
	for n := 0; n < b.N; n++ {
		// b.StopTimer()
		count = 0
		b.StartTimer()
		count++
	}
}
$
$ go test tiny_test.go -bench=.
goos: linux
goarch: amd64
BenchmarkTiny-4   	1000000000	         2.15 ns/op
PASS
ok  	command-line-arguments	2.374s
$ 

@peterGo peterGo changed the title from testing: tiny benchmarks with StopTimer run forever to testing: tiny benchmark with StopTimer runs forever Aug 25, 2018

@ianlancetaylor ianlancetaylor added this to the Go1.12 milestone Aug 25, 2018

@meirf

This comment has been minimized.

Member

meirf commented Aug 25, 2018

runtime.ReadMemStats which is called by both StartTimer and StopTimer is known to be slow, though I'm not sure it's expected to be as slow as shown here.

Calling just StartTimer without StopTimer is a noop because b.timerOn is true.

The uses of StartTimer/StopTimer are usually around one time setup/teardown code that happens before or after the b.N loop.

@josharian

This comment has been minimized.

Contributor

josharian commented Aug 26, 2018

I suspect that this is (effectively) a duplicate of #20875.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment