-
Notifications
You must be signed in to change notification settings - Fork 1
Fix decompression panic #18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix decompression panic #18
Conversation
This fixes a problem where files that have a 0 decompressed length will try to double the buffer size indefinitely.
The panic in rle_decode_fixed was previously solved in actioninja#17, this moves that code to also cover the copy_from_reader sections.
|
Oh nice the new tests I added don't work at all in CI, something else to look into. Clippy fixes are pretty trivial and coverage is a non-blocker. Also gotta love when assumptions about data formats cannot hold at all due to a big pile of janky user tools and sometimes janky official tools. |
|
Looks like I just accidentally commit some files that should have been ignored, rebase on to master and tests should run again. |
|
Wow, the full symmetrical integration tests take 3 hours to run in CI, going to have to make those manually invoked. |
This currently has no real-world impact, but it's good for consistency's sake and to be sure there can be no panics anywhere
|
This is good enough for now for a patch version, I'll make the api changes in a separate pull request later. |
Some tools used for the sims 2 modding will let users create packages that have some resources mark as compressed, but have the actual stored data be uncompressed (or vice versa).
This means that all resources have to be tested by attempting to decompress them, and if decompression fails for all header formats it should be assumed to be uncompressed. (example)
The consequence is that the decompression must not panic under any circumstance, even with adversarial inputs.
So far I did three things:
There should probably also be a way for users to specify the level of strictness on buffer size checks, as some inputs are currently accepted that probably should be rejected with an error in some applications.
I'm thinking of having three strictness levels:
It would also be nice to be able to specify a maximum initial buffer size, so that garbage data can't allocate multiple gigabytes of memory for some inputs.
When doing a major version for the options change it would probably also be nice to change the compression options struct to use a builder pattern, so that future added options wouldn't be a breaking change.