testing: single benchmark starts multiple time

[jeffreystoke]

What version of Go are you using (go version)?

go version go1.10beta1 darwin/amd64

Does this issue reproduce with the latest release?

yes with go1.9.2

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/jeffrey/Library/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/jeffrey/GoSpace:/Users/jeffrey/GolangProjects"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/r9/zplfsltx4mx8yf05hrx0dnwm0000gp/T/go-build516212567=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I was trying to make a benchmark with my mqtt client lib (libmqtt), using go test as

go test -v -bench . -benchmem -count=1

And here is the code

func BenchmarkLibmqttClient(b *testing.B) {
	b.N = 1
	b.ReportAllocs()
	var client lib.Client
	var err error

	b.Log("init client")
	if client, err = lib.NewClient(
		lib.WithServer("localhost:1883"),
		lib.WithKeepalive(3600, 1.2),
		lib.WithCleanSession(true)); err != nil {
		b.Error(err)
	}

	client.HandleUnSub(func(topic []string, err error) {
		if err != nil {
			b.Error(err)
		}
                // destroy client after  unsub success
		client.Destroy(true)
	})

	client.Connect(func(server string, code lib.ConnAckCode, err error) {
		if err != nil {
			b.Error(err)
		} else if code != lib.ConnAccepted {
			b.Error(code)
		}

		b.ResetTimer()
		for i := 0; i < b.N; i++ {
			client.Publish(&lib.PublishPacket{
				TopicName: testTopic,
				Qos:       testQos,
				Payload:   testTopicMsg,
			})
		}
		client.UnSubscribe(testTopic)
	})
	client.Wait()
}

Once added the client.Destroy(true) call (which just cancel the context of the client, so all workers can return and client.Wait() will return) will result in multiple restart

What did you expect to see?

benchmark starts and finishes both only one time

What did you see instead?

benchmark starts multiple times and finishes only one time

Here is the output for the benchmark

go test -v -bench . -benchmem -count=1
--- BENCH: BenchmarkLibmqttClient-4
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        ... [output truncated]
PASS
ok      github.com/goiiot/libmqtt/benchmark     0.173s

[davecheney]

This is expected, the testing package runs your benchmark multiple times with increasing values of b.N.

http://talks.godoc.org/github.com/davecheney/high-performance-go-workshop/high-performance-go-workshop.slide#21

Closing as there is no bug to be found here.

[jeffreystoke]

@davecheney thanks for your nice slides.

[davecheney]

You're welcome, I hope they're useful.

…

On 10 January 2018 at 19:04, Jeff Ctor ***@***.***> wrote: @davecheney <https://github.com/davecheney> thanks for your nice slides. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#23398 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA69wfBXrvG67_OT9cnXMAkkIP9Bsks5tJG78gaJpZM4RY6c7> .