This is a proof of concept showing parallel decompression using a method similar to lbzip2's lbzcat. File is read and scanned for block magics, each block is sent to a different thread for decompression, then printed out in order.
Caution
Warning: unlike lbzip2, it cannot uncompress all files made by bzip2. Sometimes different bzip2 blocks might share state, and since we decompress each block independently, we will get different results. Use with caution
It is currently about 20% slower than lbzcat.
- bzip2 crate for decompressing the blocks
- memmem for fast search (SIMD-enabled on aarch64 & x86_64)
- memmap for read-only mmap-ing the input file
- core_affinity for portably pinning and enumerating the decompression threads
This program is experimental, please don't try building it if you don't know what you are doing.
cargo build --release