Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not raise error if code equals range in get_bit #16

Merged
merged 8 commits into from
Dec 16, 2019
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/decode/lzma.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
use byteorder::{LittleEndian, ReadBytesExt};
use crate::decode::lzbuffer;
use crate::decode::rangecoder;
use crate::error;
use byteorder::{LittleEndian, ReadBytesExt};
use std::io;

pub struct LZMAParams {
Expand Down
2 changes: 1 addition & 1 deletion src/decode/lzma2.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
use byteorder::{BigEndian, ReadBytesExt};
use crate::decode::lzbuffer;
use crate::decode::lzbuffer::LZBuffer;
use crate::decode::lzma;
use crate::decode::rangecoder;
use crate::error;
use byteorder::{BigEndian, ReadBytesExt};
use std::io;
use std::io::Read;

Expand Down
21 changes: 4 additions & 17 deletions src/decode/rangecoder.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use byteorder::{BigEndian, ReadBytesExt};
use crate::decode::util;
use crate::error;
use byteorder::{BigEndian, ReadBytesExt};
use std::io;

pub struct RangeDecoder<'a, R>
Expand Down Expand Up @@ -35,19 +35,12 @@ where

#[inline]
fn normalize(&mut self) -> io::Result<()> {
trace!(
" {{ range: {:08x}, code: {:08x} }}",
self.range,
self.code
);
trace!(" {{ range: {:08x}, code: {:08x} }}", self.range, self.code);
if self.range < 0x1000000 {
self.range <<= 8;
self.code = (self.code << 8) ^ (self.stream.read_u8()? as u32);

debug!(
"+ {{ range: {:08x}, code: {:08x} }}",
self.range, self.code
);
debug!("+ {{ range: {:08x}, code: {:08x} }}", self.range, self.code);
}
Ok(())
}
Expand All @@ -56,13 +49,7 @@ where
fn get_bit(&mut self) -> error::Result<bool> {
self.range >>= 1;

if self.code == self.range {
return Err(error::Error::LZMAError(String::from(
"Corrupted range coding",
)));
}

let bit = self.code > self.range;
let bit = self.code >= self.range;
if bit {
self.code -= self.range
}
Expand Down
5 changes: 1 addition & 4 deletions src/decode/util.rs
Original file line number Diff line number Diff line change
Expand Up @@ -85,10 +85,7 @@ where
R: io::BufRead,
{
pub fn new(read: &'a mut R) -> Self {
Self {
read,
count: 0,
}
Self { read, count: 0 }
}

pub fn count(&self) -> usize {
Expand Down
4 changes: 2 additions & 2 deletions src/decode/xz.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
use byteorder::{BigEndian, LittleEndian, ReadBytesExt};
use crc::{crc32, crc64, Hasher32};
use crate::decode::lzma2;
use crate::decode::util;
use crate::error;
use byteorder::{BigEndian, LittleEndian, ReadBytesExt};
use crc::{crc32, crc64, Hasher32};
use std::hash::Hasher;
use std::io;
use std::io::Read;
Expand Down
2 changes: 1 addition & 1 deletion src/encode/dumbencoder.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use byteorder::{LittleEndian, WriteBytesExt};
use crate::encode::rangecoder;
use byteorder::{LittleEndian, WriteBytesExt};
use std::io;

pub struct Encoder<'a, W>
Expand Down
5 changes: 1 addition & 4 deletions src/encode/util.rs
Original file line number Diff line number Diff line change
Expand Up @@ -51,10 +51,7 @@ where
W: io::Write,
{
pub fn new(write: &'a mut W) -> Self {
Self {
write,
count: 0,
}
Self { write, count: 0 }
}

pub fn count(&self) -> usize {
Expand Down
4 changes: 2 additions & 2 deletions src/encode/xz.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
use byteorder::{BigEndian, LittleEndian, WriteBytesExt};
use crc::{crc32, Hasher32};
use crate::decode;
use crate::encode::lzma2;
use crate::encode::util;
use byteorder::{BigEndian, LittleEndian, WriteBytesExt};
use crc::{crc32, Hasher32};
use std::io;
use std::io::Write;

Expand Down
22 changes: 22 additions & 0 deletions tests/files/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Test files

This folder contains a collection of test files to cover different use cases of the lzma-rs library.

This README describes files that are not self-explanatory in this folder.

## range-coder-edge-case

This is a file that causes the code and range to be equal at some point during decoding LZMA data.
Previously, this file would raise an `LZMAError("Corrupted range coding")`, although the file is a valid LZMA file.

The file was created by generating random geometry in [Blender](1) using the Array and Build
modifier on a cube.
The geometry was then exported as an FBX file that was converted into OpenCTM using the 3D service
in [Cognite Data Fusion](2).
The vertices in the resulting OpenCTM file are LZMA-compressed.
This LZMA-compressed section of the file was manually extracted and the header modified to include
the unpacked size.
The unpacked size is four times the vertex count found in the OpenCTM data.

[1]: https://blender.org
[2]: https://docs.cognite.com
13,206 changes: 13,206 additions & 0 deletions tests/files/range-coder-edge-case

Large diffs are not rendered by default.

Binary file added tests/files/range-coder-edge-case.lzma
Binary file not shown.
5 changes: 5 additions & 0 deletions tests/lzma.rs
Original file line number Diff line number Diff line change
Expand Up @@ -72,13 +72,18 @@ fn round_trip_hello() {
fn round_trip_files() {
let _ = env_logger::try_init();
round_trip_file("tests/files/foo.txt");
round_trip_file("tests/files/range-coder-edge-case");
}

#[test]
fn big_file() {
let _ = env_logger::try_init();
decomp_big_file("tests/files/foo.txt.lzma", "tests/files/foo.txt");
decomp_big_file("tests/files/hugedict.txt.lzma", "tests/files/foo.txt");
decomp_big_file(
"tests/files/range-coder-edge-case.lzma",
"tests/files/range-coder-edge-case",
);
}

#[test]
Expand Down