Very slow backtraces in bytecode : debug infos read from the filesystem everytime #6302
Original bug ID: 6302
The function read_debug_info() in byterun/backtrace.c is called every time a backtrace is requested.
Recently, we wanted to log backtraces in one piece of the program and it became extremely slow in bytecode. The .cds file has about 40Mb, and it takes about 0.5s to read it.
What's the rationale for re-reading debug infos every time instead of keeping it in memory?
If it's considered problematic to keep the data in RAM (esp. considering it can be so large), we could have at least have an option to do so, or maybe consider using a more compact representation for location information required for backtraces. Currently, we use debug events, which are much richer than what we need (only their ev_pos and ev_loc fields). We could project debug events to these two fields either upon reading the debugging info in backtrace.c, or maybe have this more compact data stored as a different section in the bytecode/.cds file to optimize reading it.
Comment author: @gasche
Indeed, I think keeping only the ev_pos and ev_loc of the "events" structure and keeping that in memory should be unproblematic. I had never noticed that event_for_location does a linear search in the event list, do you think that could also be a performance issue for backtrace-heavy applications?