Re-implement pstatsloader in JavaScript

[jiffyclub]

One of the bottlenecks described in #12 is that it's not sensible to construct a JSON string of the entire call tree for a lot of profiles. It just takes too long and takes up too much space. We can get around this by only JSON-ifying the pstats.Stats.stats dictionary and constructing the call tree as a JavaScript structure in much the same way that the runsnake.pstatsloader module does in Python.

[embray]

Good plan. Should be straightfoward. Still won't help when it comes to rendering the view, but I guess the idea would be to just construct the call tree up to a certain depth? (and possibly breadth; tossing out branches that contribute very little of significance to run time--this is for profiling after all and in the vast majority of cases I'm just interested in seeing the parts that take the most time--I don't need to be able to explore the entire call tree).

[jiffyclub]

Yeah, this will just be step one, then we can worry about the graphics. Making the visualization more efficient won't help if we time out on the data request.

[jiffyclub]

I've started a branch called js-statsloader for this.

[jonashaag]

I'm not sure if you guys already plan on implementing this - but one simple workaround (and feature!) would be to add an option to filter out all rows that take less than a certain amount of time.

`--min-time T: Don't show callables that took less than T seconds cummulative to run"

[embray]

@jonashaag That sounds like a good idea--I think I've suggested that before too. I would put some minimum default, though give some control over it. Probably also have a visible message somewhere that nodes with time < T are not shown. That should probably be a separate issue though.

[kburns]

What are the latest thoughts on how to best approach this? I don't have much JS experience but might have time to tinker with a few things. With the changes to stdlib marshal in python 3.4, cprofile outputs no longer seem to work with most other profile vis tools, but mine are all too big for snakeviz currently. Thanks!

[tonysyu]

I ran into the same problem, with much of my code being too large for snakeviz to handle. My workaround was to limit the depth of the tree passed to the browser. I have a branch that implements this, but it also includes a lot of additional refactoring: https://github.com/jiffyclub/snakeviz/pull/40/files.

By default, it limits trees to a depth of 10, but you can configure it with snakeviz -d 20 my-script.py.prof to set the tree depth to 20.

[kburns]

Limiting the depth is a fantastic feature, but it would also be great to have a cumulative-time cutoff as @jonashaag and @embray were previously discussing. My profiles, for instance, have many very inexpensive function calls that are bloating the tree -- with your branch, I can get a visualization for a depth of 3, but not 4.