cmd/trace: runs out of memory processing 30 MB file

[AlexRouSg]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.9 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GORACE=""
GOROOT="/.local/go"
GOTOOLDIR="~/.local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build582547481=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

What did you do?

go tool trace trace.out

What did you expect to see?

2017/09/14 03:16:35 Parsing trace...
2017/09/14 03:16:40 Serializing trace...
2017/09/14 03:16:41 Splitting trace...
2017/09/14 03:16:47 Opening browser

What did you see instead?

2017/09/14 04:31:17 Parsing trace...
2017/09/14 04:31:32 Serializing trace...
go tool trace: signal: killed

Something to note, OS froze (keyboard/mouse unresponsive) for a few seconds before trace terminated.

trace.out.tar.gz

[AlexRouSg]

So I ran it with the ram/cpu monitor opened and noticed it was eating 100% ram before my linux box froze. Added some swap and looks like it ate 6 GB of ram to open the webpage and then trying to view a trace ate another 1.5 GB - 2 GB and then froze my linux box without showing the graph.

Do these numbers sound right? They are from looking at the total ram +swap usage so it's not too accurate.

[davecheney]

Yes. Traces can be very large.

…

On 14 Sep 2017, at 07:11, Chotepud Teo ***@***.***> wrote: So I ran it with the ram/cpu monitor opened and noticed it was eating 100% ram before my linux box froze. Added some swap and looks like it ate 6 GB of ram to open the webpage and then trying to view a trace ate another 1.5 GB - 2 GB and then froze my linux box without showing the graph. Do these numbers sound right? They are from looking at the total ram +swap usage so it's not too accurate. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[AlexRouSg]

Any options? Or am I just screwed now cause this is from just 5 seconds of run time.

[davecheney]

How much ram does your host have?

…

On Thu, Sep 14, 2017 at 7:25 AM, Chotepud Teo ***@***.***> wrote: Any options? Or am I just screwed now cause this is from just 5 seconds of run time. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#21870 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcAyArgyXPeVnm9xqtz-b5dFInhBejks5siEhIgaJpZM4PWswJ> .

[AlexRouSg]

8 GB (7.7 according to ram monitor), but even with swap I couldn't get the graphs to load before it freezes.

[davecheney]

/cc @francesc TL;DR you need more ram. Sorry.

…

On Thu, Sep 14, 2017 at 9:14 AM, Chotepud Teo ***@***.***> wrote: 8 GB, but even with swap I couldn't get the graphs to load before it freezes. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#21870 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA0rLZcXwOmi1L_7zCGKgTFz6CfTXks5siGHEgaJpZM4PWswJ> .

[AlexRouSg]

Was expecting that...
But if this is from 5 seconds of runtime, then there is no amount of RAM that would be enough for longer runs. And turning trace on/off (runtime/trace) wouldn't be a very nice experience if you are trying to catch a event you can't predict.

Do you know of any other similar tools but live so shouldn't eat a ton of ram?

[davecheney]

What was the program doing during that trace. Based on what I've seen the resources required to analyse a trace are proportional to the number of goroutines events during that time. Where an event is, a goroutines starting and stopping execution.

…

On 14 Sep 2017, at 09:38, Chotepud Teo ***@***.***> wrote: Was expecting that... But if this is from 5 seconds of runtime, then there is no amount of RAM that would be enough for longer runs. And turning trace on/off (runtime/trace) wouldn't be a very nice experience if you are trying to catch a event you can't predict. Do you know of any other similar tools but live so shouldn't eat a ton of ram? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[AlexRouSg]

Running I think 4 goroutines and calling a bunch of C funcs in a loop (each goroutine does this)... I guess the worst case for trace resource consumption?

[davecheney]

Pretty much.

…

On Thu, Sep 14, 2017 at 9:55 AM, Chotepud Teo ***@***.***> wrote: Running I think 4 goroutines and calling a bunch of C funcs in a loop... I guess the worst case for trace resource consumption? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#21870 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA4DCvtKghJD_PTfl1pKm6poQDv0Nks5siGtUgaJpZM4PWswJ> .