Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/trace: "failed to parse trace: no consistent ordering of events possible" #29707

Open
256dpi opened this issue Jan 12, 2019 · 4 comments

Comments

Projects
None yet
4 participants
@256dpi
Copy link

commented Jan 12, 2019

What version of Go are you using (go version)?

$ go version
go version go1.11.4 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/256dpi/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/256dpi/Development/Go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.11.4/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.11.4/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/mj/lp3r462x6tqfd4v_93bvh65h0000gn/T/go-build000614795=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I ran a network intensive go program (closed source) with a fairly light workload.

What did you expect to see?

I expect to be able to obtain a trace profile and explore it with the viewer program.

What did you see instead?

After obtaining the profile with wget -O trace.out "http://localhost:6060/debug/pprof/trace?seconds=10". When running go tool trace trace.out I received:

2019/01/12 12:37:26 Parsing trace...
failed to parse trace: no consistent ordering of events possible
@256dpi

This comment has been minimized.

Copy link
Author

commented Jan 15, 2019

I have to add, I was testing a binary that was linked with C code (CGO). Is it possible that tracing does not work properly in CGO builds?

@bcmills bcmills changed the title failed to parse trace: no consistent ordering of events possible net/http/pprof: "failed to parse trace: no consistent ordering of events possible" Jan 30, 2019

@bcmills

This comment has been minimized.

Copy link
Member

commented Jan 30, 2019

CC @rsc @hyangah @matloob @dvyukov for pprof and trace

@bcmills bcmills added this to the Go1.13 milestone Jan 30, 2019

@bcmills bcmills changed the title net/http/pprof: "failed to parse trace: no consistent ordering of events possible" cmd/trace: "failed to parse trace: no consistent ordering of events possible" Jan 30, 2019

@hyangah

This comment has been minimized.

Copy link
Contributor

commented Jan 30, 2019

#16755 was the first one that came to my mind, but I've never seen the issue with darwin/amd64, so I am not sure. I am not aware of cmd/trace issues involving cgo.

@256dpi is it possible to share the captured trace with me?

@AlexRouSg

This comment has been minimized.

Copy link
Contributor

commented May 4, 2019

@hyangah I just ran into this and made a repo.

Basically it happens when you export a Go function to C and then call that function inside a C created thread.

package test

/*
	#include <pthread.h>

	extern void* callback(void*);
	typedef void* (*cb)(void*);

	static void testCallback(cb cb) {
		pthread_t thread_id;
		pthread_create(&thread_id, NULL, cb, NULL);
    	        pthread_join(thread_id, NULL);
	}
*/
import "C"
import (
	"context"
	"runtime/trace"
	"time"
	"unsafe"
)

var traceCtx, traceTask = trace.NewTask(context.Background(), "Debug")

func test() {
	C.testCallback(C.cb(C.callback))
}

//export callback
func callback(unsafe.Pointer) unsafe.Pointer {
	defer trace.StartRegion(traceCtx, "callback").End()
	time.Sleep(time.Millisecond)

	return nil
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.