Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: single benchmark starts multiple time #23398

Closed
jeffreystoke opened this issue Jan 10, 2018 · 3 comments
Closed

testing: single benchmark starts multiple time #23398

jeffreystoke opened this issue Jan 10, 2018 · 3 comments

Comments

@jeffreystoke
Copy link

@jeffreystoke jeffreystoke commented Jan 10, 2018

What version of Go are you using (go version)?

go version go1.10beta1 darwin/amd64

Does this issue reproduce with the latest release?

yes with go1.9.2

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/jeffrey/Library/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/jeffrey/GoSpace:/Users/jeffrey/GolangProjects"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/r9/zplfsltx4mx8yf05hrx0dnwm0000gp/T/go-build516212567=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I was trying to make a benchmark with my mqtt client lib (libmqtt), using go test as

go test -v -bench . -benchmem -count=1

And here is the code

func BenchmarkLibmqttClient(b *testing.B) {
	b.N = 1
	b.ReportAllocs()
	var client lib.Client
	var err error

	b.Log("init client")
	if client, err = lib.NewClient(
		lib.WithServer("localhost:1883"),
		lib.WithKeepalive(3600, 1.2),
		lib.WithCleanSession(true)); err != nil {
		b.Error(err)
	}

	client.HandleUnSub(func(topic []string, err error) {
		if err != nil {
			b.Error(err)
		}
                // destroy client after  unsub success
		client.Destroy(true)
	})

	client.Connect(func(server string, code lib.ConnAckCode, err error) {
		if err != nil {
			b.Error(err)
		} else if code != lib.ConnAccepted {
			b.Error(code)
		}

		b.ResetTimer()
		for i := 0; i < b.N; i++ {
			client.Publish(&lib.PublishPacket{
				TopicName: testTopic,
				Qos:       testQos,
				Payload:   testTopicMsg,
			})
		}
		client.UnSubscribe(testTopic)
	})
	client.Wait()
}

Once added the client.Destroy(true) call (which just cancel the context of the client, so all workers can return and client.Wait() will return) will result in multiple restart

What did you expect to see?

benchmark starts and finishes both only one time

What did you see instead?

benchmark starts multiple times and finishes only one time

Here is the output for the benchmark

go test -v -bench . -benchmem -count=1
--- BENCH: BenchmarkLibmqttClient-4
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        client_test.go:53: init client
        ... [output truncated]
PASS
ok      github.com/goiiot/libmqtt/benchmark     0.173s
@davecheney

This comment has been minimized.

Copy link
Contributor

@davecheney davecheney commented Jan 10, 2018

This is expected, the testing package runs your benchmark multiple times with increasing values of b.N.

http://talks.godoc.org/github.com/davecheney/high-performance-go-workshop/high-performance-go-workshop.slide#21

Closing as there is no bug to be found here.

@davecheney davecheney closed this Jan 10, 2018
@jeffreystoke

This comment has been minimized.

Copy link
Author

@jeffreystoke jeffreystoke commented Jan 10, 2018

@davecheney thanks for your nice slides.

@davecheney

This comment has been minimized.

Copy link
Contributor

@davecheney davecheney commented Jan 10, 2018

@mikioh mikioh changed the title single benchmark starts multiple time testing: single benchmark starts multiple time Jan 14, 2018
@golang golang locked and limited conversation to collaborators Jan 14, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.