This project implements efficient sourcemap handling in Rust and wraps it for Python. It's based on rust-sourcemap for the main sourcemap handling and implements a separate format that can be cached more efficiently.
Tested with Rust 1.12 and later.
This requires a recent version of pip, setuptools and Rust to build. To get rust use rustup.
To compile extension and wrappers use this with an enabled virtualenv:
This is a benchmark against one of the sentry customer files that comes in at 30MB in size:
parse_json ... 516.52ms lookup_token ... 0.05ms dump_memdb ... 217.95ms load_memdb ... 50.87ms lookup_token ... 0.02ms load_mmap ... 0.05ms lookup_token ... 0.03ms
Unexpectedly parsing is the slowest followed by dumping out an index in the memdb format. The latter can most likely be optimized. Looking up tokens is fast in most formats but fastest with memdb. Loading memdb via mmap gives the best loading time if the file to load is coming from the file system with a slight performance hit on actually looking up symbols.