Web UI should support flame graphs

[rakyll]

Flame graphs allow you to move in a specific ancestry path from a very compact UI and lets the users to zoom in/out specific paths easily. Flame graphs became one of the defato representation of profiling data in the industry in the recent years, e.g. go-torch is a highly successful tool and often user's entry point to visualize pprof data in Go.

Consider supporting flame graphs in addition to the dot visualization on the new web UI, so we can provide this representation out-of-the-box with minimal setup and external dependencies.

(If you need this feature as well, you can 👍 so the team knows about your interest.)

[nathanleclaire]

This would be amazing! Thanks for putting together the proposal. One thing I especially love from go-torch / Brendan's script is that you can search and highlight based on keyword, e.g., reflect to show very quickly each candle that touches reflect package. Would love to see that included in a Golang official version as well.

[nathanleclaire]

I bet you could do fun things with color too, like give each package its own unique tint, so that one could quickly scan the graph for mis-behaving packages.

[nathanleclaire]

If I recall as well, I also had a pretty hard time getting a useful flame graph for anything other than CPU usage (e.g., memory) from pprof. That's something to consider, I'd love to see some Golang memory usage flame graphs.

[chenziliang]

Would really love to have this stuff built in

[huydx]

flamegraph has been my default development stack for a while now, love it is a default part of pprof

[pierods]

Yes please

[mremond]

I think it would be very useful as well.
👍

[campoy]

As someone that has written a workshop on the topic (github.com/campoy/go-tooling-workshop) I can assure you that flame graphs are an amazing tool for beginners, information is represented in a far clearer way, and removing the setup friction would be great.

👍

[ipfans]

And hope it happen in Go 1.10

[jhayotte]

Yes please! 👍🏻

[anazmy]

Yes please

[brendangregg]

FWIW... There are multiple implementations of flame graphs, and you could also just write a new one in go. :-)

My original https://github.com/brendangregg/FlameGraph is Perl, and evolved, and has become complex (if you roll back the git history you'll see how simple it once was). It would be cleaner written from scratch, now that I know that it should support differentials, annotations, etc.

Netflix has been (and will be) developing https://github.com/spiermar/d3-flame-graph , as a new d3 version. This should allow us to do more interactivity. Eg, flipping the merge order from bottom-up to top-down in d3, rather than the Perl version where you have to rerun it with --reverse --inverted.

FWIW, if I were advising someone on a go rewrite based on what I've learned, I'd encourage them to do the following :

• Mouse-overs for more info.
• Click to zoom (you could ask Adrien Mahieux if you can just use his JavaScript zoom code from flamegraph.pl)
• Search. A key to this being useful is that it shows the percent time in the bottom right, excluding vertical overlap.
• A few different palettes. Apart from random warm colors, highlighting code in different ways. Eg, my java palette.
• An architecture so you can add differentials later (however you want to implement them).
• A way to save self-contained interactive SVGs, like those my Perl program emit. They've been handy to store, email, and refer to later on when they still work, despite the flame graph code changing.

Now, if it's consuming profiled stack traces that are time-stamped (which isn't necessary, the x-axis just sorts alphabetically), then you could have a toggle that flips between "flame charts" (time-ordered, as seen in Google chrome dev tools) and "flame graphs" (alphabet ordered, to maximize merging). Both can help solve different kinds of issues.

[Dieterbe]

This proposal is great. personally i've found the current state of affairs requires a bit too many hurdles (specifically getting perl and the perl scripts). would it make most sense to collaborate on d3-flame-graph and reuse it as much as possible, and just putting the connecting logic in go tool pprof ? we could then have in addition to web a command like flame and it would generate a html page that uses d3-flame-graph ?

[spiermar]

If anyone is looking into implementing this, specially the d3-flame-graph version, let me know. I'm hoping to get a bit of time next week to look into this. Creating a self-contained HTML file and having the flame graph data structure inline should be fairly easy to get done. The plugin takes a simple JSON file as input and the stack-convert tool has a good example of how to convert the data. As @brendangregg mentioned, we're looking to improve and add more features to the D3 plugin in the future.

[ajstarks]

I've considered doing a flame graph implementation with SVGo. Happy to contribute.

[rakyll]

We have an existing front-end implementation of Flame Graphs at Google and we are considering about publicizing it. It has no external dependencies and covering some of the flavors @brendangregg suggested above. I am willing to export it and then we can iterate from there. My plan is to get this work done before Go 1.10 freeze, so we can have 1.10 shipped with flame graphs.

[spiermar]

Submitted #188

It's using the D3 version of flame graphs and implements all features mentioned by @brendangregg, except except different palettes, which is under development and requires changes in the D3 plugin. More features are also underway on d3-flame-graph and would just require a version bump on pprof.
Tested with both CPU profiles and Memory profiles (as suggested by @nathanleclaire).

A static demo can be found in pprof_flame.html.

[aalexand]

This is available in master now, closing.

[KISSMonX]

Good Job!

[tylfin]

Doesn't look like this is available in Go 1.10.3? Is it planned for a 1.11 release?

[nolanmar511]

I think the flame graph is available as part of go 1.10, though the flame graph's location may not be clear enough.
It should be part of the web ui. So, if you run go tool pprof -http=: /path/to/profile, a new window should open in your browser, and then you should be able to switch the view from the graph view to the flame view using the view drop-down menu.

[aalexand]

Note that the fix to auto-assign a random port is not in Go 1.10 (see #413), so the command you want is something like go tool pprof -http=:8080 /path/to/profile. And looks like flame graph is not in pprof that is in Go 1.10.3, as https://github.com/golang/go/tree/go1.10.3/src/cmd/vendor/github.com/google/pprof/third_party doesn't have the flame graph code. Go 1.11 should have it - https://github.com/golang/go/tree/go1.11rc2/src/cmd/vendor/github.com/google/pprof/third_party has the flame graph in.

[rakyll]

Flame graph is available in 1.11.