v1.1.0: V2 Encoding Scheme
This release includes a large change. It represents a change in the encoding scheme to support Delta and Delta-Delta (de)compression. It marks a significant improvement to compression rates and ratios on real-world data. Additionally, the new encoding scheme is compatible with a second pass compression scheme like LZ4 for excellent compression ratios in small payloads and even better ratios in large payloads.
Adding a HalfVec lazy data structure and moving away from bitvec + deku leads to big performance and power improvements on embedded systems.
What's Changed
- Add real world xyz accelerometer data index file compression benchmark by @trueb2 in #2
- Refactor BitBuffer push usage during encode by @ameykasbe in #6
- Fix clippy warning on unnecessary lifetime by @ameykasbe in #7
- Add static ring buffer FIFO Queue by @trueb2 in #10
- Update workspace resolver by @trueb2 in #11
- Add CompressV2 proc macro by @trueb2 in #12
- Add impl of ExactSizeIterator for CompressionQueueIter by @ameykasbe in #13
- Delta Compression and Decompression by @ameykasbe in #14
- Nibble-aligned Delta and Delta-Delta Encoding by @trueb2 in #18
- Fix emission of 5 samples of 6 bits by @trueb2 in #19
- Emit column start header prior to first row emission by @trueb2 in #20
- Fix decoding with additional error messages by @trueb2 in #21
- Handle decoding only no rows or one row by @trueb2 in #23
- Add seperators in binary representations by @ameykasbe in #22
- Update test cases by @ameykasbe in #24
- Add emit delta i8 by @ameykasbe in #25
- Fix bit masks and width while decoding by @ameykasbe in #30
- Require compressor to track and expose the total number of rows consumed by @trueb2 in #31
- Reduce branching during single row compression by @trueb2 in #32
- Use power of two compression queue size by @trueb2 in #33
- Hold zigzag bits in CompressionQueue by @trueb2 in #34
- Add Test: Compress all possible deltas for i16 values by @ameykasbe in #35
- Update unit test for HalfVec by @ameykasbe in #38
- Add unit tests & related functionality for emit i32 delta by @ameykasbe in #36
- Update CompressionQueue fuzz test by @ameykasbe in #37
- Fix mod for Cargo Test pass by @ameykasbe in #39
- Skip tracking number of column values emitted by @trueb2 in #40
- Fix: Implement zigzag for each type and add tests by @ameykasbe in #41
- Update queue zigzag to an array of usize. Add impl Bits for i64. by @ameykasbe in #42
- Replace hardcoded header values with named constants by @ameykasbe in #44
- Add field attributes in proc macros for delta by @ameykasbe in #43
- Code Refinement by @ameykasbe in #45
- Implement 64 bits 1 sample delta compression case by @ameykasbe in #46
- Update README with changes from feature branch by @trueb2 in #47
- Real world benchmark in a non-publishable bench crate by @trueb2 in #48
- Nibble aligned encoding scheme implementation by @trueb2 in #8
New Contributors
- @trueb2 made their first contribution in #2
- @ameykasbe made their first contribution in #6
Full Changelog: v1.0.4...v1.1.0