-
Notifications
You must be signed in to change notification settings - Fork 245
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve performance of gimli dwarf parsing #232
Comments
This is mildly surprising since we previously found that the gimli addr2line was faster than binutils, which is using libbacktrace IIRC. I wonder if the integration in this crate is not reusing structures enough to fully amortize some of the caching / side tables cc @philipc |
er wait, that's not what that says at all... |
@fitzgen Yeah that benchmark result changed with the switch to main--'s code because it makes a different memory/time tradeoff. However, I wouldn't expect it to be 10x slower. I also haven't benchmarked gimli recently, so maybe something has regressed. I'll definitely look into what's causing this. |
I'm pretty sure binutils doesn't use libbacktrace, and I don't think we've ever benchmarked against libbacktrace since I don't think it has an addr2line clone? Changing the gimli data structures to store rows instead of sequences helps some:
Still not close to libbacktrace though. I'll keep investigating. |
Things that look like low hanging fruit: Inside gimli
Inside this crate
|
Thanks for investigating @philipc! That all sounds definitely plausible. For those that are more related to this crate:
If gimli symbolication doesn't turn up any results we fall back to calling
FWIW git2 and such often provide both a bytes accessor and a str accessor, and we could just be careful to use the bytes accessors here in this crate
This is likely something to fix pretty easily. I think that the cache lookup hasn't really been optimized at all and we could almost certainly do so in a much more efficient fashion, perhaps even avoiding
This is probably something we should cache on our end if that's the case, I wasn't sure if that was the case when I first looked into it! |
I'll work on the gimli side of this. It also looks like the interval tree we use for inlined functions is slow, but I'm still investigating that. |
Instead of relying on lazy iteration, instead iterate over all symbols up front and then sort them by address. We already do this on Windows and it looks like symbol iteration is somewhat slow in benchmarks for goblin. By sorting we should also be able to run through these lists much more quickly! Overall on top of #235 this yields another 30% speedup or so for symbolication. cc #232
This commit updates the gimli implementation to iterate over all shared libraries only once, caching information about them to get processed later. This information is usually static for the lifetime of the program, and this improves the performance of resolving a backtrace significantly in some situations (by up to 30%). The organization of the gimli backend has been updated to reflect this cache where `Cache` is now a dedicated type with methods hanging off it. Some helper code had to be duplicated from the `findshlibs` crate to do this entirely. cc #232
Instead of relying on lazy iteration, instead iterate over all symbols up front and then sort them by address. We already do this on Windows and it looks like symbol iteration is somewhat slow in benchmarks for goblin. By sorting we should also be able to run through these lists much more quickly! Overall on top of #235 this yields another 30% speedup or so for symbolication. cc #232
Ok I made some ticky boxes on #232 (comment) and I also merged a few PRs related to this crate itself. What I'm seeing now is:
which is some great progress! Still aways to go to catch up to libbacktrace:
|
Ok great, with those changes plus my current hacked up gimli changes I can get that to:
|
Holy cow, very nice! That's basically everything :) |
Just an update on my progress. The backtrace-rs benchmarks don't cover the cost of building the caches. So while my gimli changes did make things faster for repeated lookups, building the cache is slower and uses more memory. I also don't have benchmarks of this for libbacktrace, so not sure how we compare there. I'm still playing around with the gimli attribute parsing. I know we can do better there, but the challenge is keeping an API that is easy to use and not too low level. |
No worries, and thanks! I think that for this crate itself that tradeoff is fine, and if it's something that this crate opts-in to specifically I think that's also fine |
Current numbers as of #273 are:
Thanks so much for all your help here @philipc, I think we're basically done now! To get a better idea, this is a benchmark of gimli roughly when I opened this issue vs now:
Nothing like some 10-20x speedups :) |
Today I was playing around with the various smoke benchmarks we have in this repository, and I executed
cargo +nightly bench
followed bycargo +nightly bench --features gimli-symbolize
. The results were pretty interesting! Running them throughcargo benchcmp
I get:Overall it looks like the gimli implementation in this crate is 10-20x slower than libbacktrace at symbolicating symbols in a backtrace.
Some of the top entries in a
perf record
are:Nothing immediately jumps out as an obvious improvement, but this is something we should cover before considering to make a switch to use gimli by default.
cc @fitzgen, I figure you're probably interested in this, and others who work on gimli may be as well? This may very well just be bugs in how this crate works with gimli as well, it may be a local fix rather than a gimli fix.
I will say though in briefly looking at libbacktrace it looks like it's roughly doing the same thing, traversing the dwarf whenever something is symbolicated.
The text was updated successfully, but these errors were encountered: