Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Avoid decompressing just to calculate external size #835
Ran more numbers, a 1GB database (npm registry subset) with the whole database file in the fs cache (e.g. no read activity on the disk), seeing improvements between 10% and 30% in both CPU and wall clock time (normalised over 3 runs each), and corresponding higher write throughput to disk. All on my laptop so highly unscientific, but enough of a difference to mark as significant.
Good call on benchmarking this.
I have a compaction benchmark script and with
So it makes 100k docs of size 10k each. Then times compaction.