New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Time taken by particular child underestimated #136
Comments
I know what's going wrong, but I'm not sure whether I really want to fix it. RubyProf has been computing incorrect timings in the presence of recursive methods for a long time until I fixed that in 12c8c9a, based on http://railsexpress.de/papers/callgraph.pdf. Now the overall timings per method are correct for recursive methods, but the child values in the graph display are now wrong for these methods. Fixing would require recomputing time values each time a method is displayed, which changes the graph printer into a O(n*n) algorithm. For large programs with thousands of methods, this seems to be problematic. I have a rough sketch ready on this branch, but I haven't tried it yet on larger programs. Need to think about how this can be optimized, before I can release (if at all). Could you let me know what you think? |
Hi Skaes, Thanks for looking into this. Unfortunately, I haven't been doing any code profiling lately, so I can't say what its performance would be like on my application, and whether O(n^2) would be problematic. If it does end up as WONTFIX, there should be documentation explaining that this problem can occur. |
thx for your feedback. I'll investigate whether large programs can still be analyzed. |
@agrimm I will fix it. |
Can you use some kind of cache or memorisation so that the recursion detection isn't so expensive? |
memoization, yes, of course. |
Hi. I have similar problem described here: https://groups.google.com/forum/#!topic/ruby-optimization/dq29hcsGgqc Is it the same bug/feature? |
@radarek that's impossible to tell without having the full ruby-prof output. |
@skaes Here is full output: https://gist.githubusercontent.com/radarek/21188eecded211e2a617/raw/full.txt (its new profile taken a minute ago so it is a little different than previous but the same problem exists) |
@radarek I suggest you override ActiveRecord's inspect method to produce a more readable output (see https://github.com/skaes/railsbench/blob/master/lib/railsbench/railsbenchmark.rb#L89). Apart from that: yes you are affected, as you have methods marked as recursive. Could you rerun the profiling using my own ruby prof repo using branch "recalculate-recursiveness-when-printing" : https://github.com/skaes/ruby-prof/tree/recalculate-recursiveness-when-printing? |
Ok, I rerun this and here are new results: https://gist.githubusercontent.com/radarek/21188eecded211e2a617/raw/ee3faa09a398abf3d1563215ea5604c487abb1f8/profile2.txt It definitely looks better now. It took about ~2h to generate report so it also needs optimization. Otherwise it is not usable at all for real worlds app size. Thanks for tips for |
@radarek I have released a new ruby-prof version (0.15.5), which should be faster. Can you please try it? Hopefully it'll take only minutes now ... |
Hm. I have updated ruby-prof to 0.15.5 and it generates results where almost all children of |
Merge error. Sorry. Use version 0.15.6 please. |
Still the same problem: https://gist.githubusercontent.com/radarek/21188eecded211e2a617/raw/c3ea8043a8f6e37fa8259d838282422cb94cfd1d/profile3.txt I rerun profile with updated branch |
@radarek thx for the feedback. I have tested the example given by @agrimm and it works as expected. Unfortunately your profile is really hard to read. Could you please apply my suggested change to the inspect method and also specify some useful min_percent value, e.g. 1% so we get a smaller profile? |
Yeah, sure. Give me a minute. |
@skaes Ok, here are results using 0.15.6: https://gist.githubusercontent.com/radarek/21188eecded211e2a617/raw/b9dd594d870bf602be4d13658d6f0d4ba2641473/profile4.txt (btw, in rails 4 trick https://github.com/skaes/railsbench/blob/master/lib/railsbench/railsbenchmark.rb#L89 doesn't work because |
Unfortunately, I still can't really make much of this, other than being a bit unsure about whether my implementation is correct. Could you perform another profile, this time using the multi printer and collect the resulting files in some archive and send it to my email address? The stack profile should be correct in all cases and it should show you better where most of the time is spent. And it links to the graph html profile. And it will hopefully show me where things go wrong (if at all). I'll pack it up for today. TTFN. |
One last question: how long did it take to generate the graph profile output? |
I sent you results printed with multi printer. Generating time looks as follows:
|
Closing the issue as it appears to be fixed. |
Unfortunately, my method of caching the recursive property has both a faulty implementation and is conceptually wrong. Not clear how this can be fixed. See #170 |
Given the following code
the time taken by a child of Enumerable#find is underestimated:
The values describing the parent, and the method being profiled are correct, but it wrongly claims that no time is taken in
Array#each
or its children. However, when profilingSlowClass#time_sink
, it correctly calculates the time taken byArray#each
and its children:Getting rid of
[1].each
makes the profile data correct.https://gist.github.com/agrimm/8464764 contains full profiling data, plus confirmation that it's not a problem with the stack trace, as was the case with #123 . I ran it on Ruby 2.2.0-dev, with a recent enough version to have Ruby bug 9321 fixed.
The text was updated successfully, but these errors were encountered: