-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Debug a wasm application with reasonable amount of RAM #537
Comments
Run wasmtime in a debugger and pass Edit: it is actually |
I'm doing that ( |
Ugh that's annoying, sorry! Is there any way you could give us a way to reproduce the crash? Ideally based on source code, but a Wasm binary would be great, too. |
I'm not actually sure if the crash is caused by a bug in wasmtime, but sure. This issue has complete build instructions. The non-stripped binary wasm files are around 300 MB, which is a bit of a pain, so if you're willing to build from source, that seems ideal. Running the resulting wasm file through binaryen wasm-opt results in a different, even more exciting crash involving |
Oh yeah and here are inputs for Yosys. You can either use |
Semi-offtopic: my end goal for this project would be to use It looks like none of the WASI runtimes are currently up to task, but I'll be happy to do the work required to make this happen, provided that I can actually run the software I need of course... |
If I build wasmtime from master, it crashes here, which I can probably fix myself. |
OK, #538 fixes the crash, but I still don't get any debug info in gdb. |
Can you try in lldb? Emitting DWARF debug info is tricky, because what's there in terms of a specification is really more a rough estimate for more corner-case-y aspects :( gdb is more discerning in its tastest than lldb in some situations, so we're not quite there yet for making it work as well. @yurydelendik that's still the situation, right? |
@tschneidereit OK so it looks like debug info works in lldb. I'm saying "looks like" because |
I don't think there is, but that sounds like a good idea! It also sounds like we have some optimization potential :) In fact, I wonder if it might be possible to lazify some of the transformation. @yurydelendik, do you think that's a possibility? |
Running wasm-opt invalidates DWARF information at this moment, see WebAssembly/binaryen#2400
Thank you!
I did not spent much time with GDB, but found that JIT debugging involves creating of "perfect" ELF image to be use with GDB JIT interface. At this moment only LLDB was tested with
Eventually, I would like to profile the performance of for large binaries/DWARF sections; the debug/non-release wasmtime takes even more time. Are you using release version of wasmtime?
Normally .debug_line and .debug_info transforms take the same time, so speed up will be at best a half of the time. If DWARF sections are missing, wasmtime generate artificial wasm-level debug information by using name section. This might help with finding the problem. |
Yes.
I profiled it, running my testcase for 10 minutes. 84% of the total time is spent in I tried to fix it but unfortunately that part of the code is very hard to understand and almost completely undocumented and I got nowhere after spending a few hours on it. |
Actually, the previous comment isn't right. I accidentally ran wasmtime on a wasm file that I ran through binaryen's wasm-opt, which indeed had debug information that lldb loaded, and which indeed made no sense. If I run wasmtime on a wasm file with non-corrupted debug information, it consumes more than 12 GB of memory after 10 minutes, spent almost entirely in |
That .debug_line table, which is line numbers info. I'll be looking into using different data structures.
Did you try to ran wasmtime on a wasm file, with stripped DWARF sections but with |
Specifically the file that I ran through wasm-opt, or any of them? I did run wasmtime on stripped wasm files with the |
By observing |
Any. The wasmtime, if DWARF sections are not found, emits JIT image with DWARF anyway but using wasm bytecode/function as "sources", so you may get wasm-y stack, which is one step better than native back trace. |
Is this with debug info, and with using |
Yes, I got a bunch of
Yes and yes. Without |
@whitequark I'm trying to improve the performance of wasmtime-debug crate. I would like to check if the your test case performance was improved (e.g. via #1260) . Is there something else needs to be done to resolve this issue? |
If it's actually possible to debug the application I mentioned earlier on a machine with reasonable amount of RAM (~8 GB) then this issue can be certainly closed. |
If possible, can you provide the detailed instructions for me to reproduce the issue locally? (I have a hard time to gather this info from the comments above) |
@yurydelendik Are the build instructions here sufficient? |
@whitequark it was not trivial but I was able to produce "yosys.wasm". I added another commit to #1260. With this PR, per Activity Monitor and Instruments on my computer, the DWARF transform does not take more than 50 sec and memory is not growing past 2.4Gb. If you have time, can you confirm that it is the case on your side? |
I greatly appreciate your work here. Unfortunately right now I'm not in a position to reproduce those results for personal reasons. The numbers you mention seem like they would allow me to proceed with my original work on wasm-ing Yosys; I won't get around to that for a while, so I'm closing the issue, because it seems like you've pretty much solved it. I'll make sure to report my results once I get around to it. |
@yurydelendik I just got around reproducing these results. Unfortunately, it seems that I hit a snag; on freshly compiled Yosys with WASI SDK 10, I get this if I pass
Here's the problematic wasm file. However, I'd like to say that even if you never fix this, y'all are still totally awesome. That's because I don't need debug information anymore! Almost everything in Yosys (except for a bizarre |
I'm trying to run a large (~15 MB) wasm application that crashes on
ud2
. Is there any way to determine which function the crash address corresponds to?The text was updated successfully, but these errors were encountered: