Skip to content

Commit

Permalink
Merge #16
Browse files Browse the repository at this point in the history
16: Do not raise error if code equals range in get_bit r=gendx a=dragly

It appears that code being equal to range is a valid state and that some files can have 
data encoded like this. An example of such a file has been added to the
tests.

### Pull Request Overview

This pull request fixes #15 

### Testing Strategy

This pull request was tested by...

- [ ] Added relevant unit tests.
- [x] Added relevant end-to-end tests (such as `.lzma`, `.lzma2`, `.xz` files).


### Supporting Documentation and References

The original data was produced by processing a randomly generated geometry with OpenCTM. OpenCTM files can contain embedded LZMA compressed data. This appears to be generated with the liblzma library. The data was then extracted to make a standalone file that is attached in this pull request.

### TODO or Help Wanted

None


Co-authored-by: Svenn-Arne Dragly <dragly@cognite.com>
Co-authored-by: gendx <gendx@users.noreply.github.com>
Co-authored-by: Svenn-Arne Dragly <s@dragly.com>
  • Loading branch information
4 people committed Dec 16, 2019
2 parents 4429567 + 0ee7b55 commit ff666f4
Show file tree
Hide file tree
Showing 5 changed files with 13,234 additions and 7 deletions.
8 changes: 1 addition & 7 deletions src/decode/rangecoder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -49,13 +49,7 @@ where
fn get_bit(&mut self) -> error::Result<bool> {
self.range >>= 1;

if self.code == self.range {
return Err(error::Error::LZMAError(String::from(
"Corrupted range coding",
)));
}

let bit = self.code > self.range;
let bit = self.code >= self.range;
if bit {
self.code -= self.range
}
Expand Down
22 changes: 22 additions & 0 deletions tests/files/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Test files

This folder contains a collection of test files to cover different use cases of the lzma-rs library.

This README describes files that are not self-explanatory in this folder.

## range-coder-edge-case

This is a file that causes the code and range to be equal at some point during decoding LZMA data.
Previously, this file would raise an `LZMAError("Corrupted range coding")`, although the file is a valid LZMA file.

The file was created by generating random geometry in [Blender](1) using the Array and Build
modifier on a cube.
The geometry was then exported as an FBX file that was converted into OpenCTM using the 3D service
in [Cognite Data Fusion](2).
The vertices in the resulting OpenCTM file are LZMA-compressed.
This LZMA-compressed section of the file was manually extracted and the header modified to include
the unpacked size.
The unpacked size is four times the vertex count found in the OpenCTM data.

[1]: https://blender.org
[2]: https://docs.cognite.com
13,206 changes: 13,206 additions & 0 deletions tests/files/range-coder-edge-case

Large diffs are not rendered by default.

Binary file added tests/files/range-coder-edge-case.lzma
Binary file not shown.
5 changes: 5 additions & 0 deletions tests/lzma.rs
Original file line number Diff line number Diff line change
Expand Up @@ -92,13 +92,18 @@ fn round_trip_hello() {
fn round_trip_files() {
let _ = env_logger::try_init();
round_trip_file("tests/files/foo.txt");
round_trip_file("tests/files/range-coder-edge-case");
}

#[test]
fn big_file() {
let _ = env_logger::try_init();
decomp_big_file("tests/files/foo.txt.lzma", "tests/files/foo.txt");
decomp_big_file("tests/files/hugedict.txt.lzma", "tests/files/foo.txt");
decomp_big_file(
"tests/files/range-coder-edge-case.lzma",
"tests/files/range-coder-edge-case",
);
}

#[test]
Expand Down

0 comments on commit ff666f4

Please sign in to comment.