Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Index out of bounds at huffman_tree.rs line 252 when reading invalid data #23

Closed
Pr0methean opened this issue Feb 29, 2024 · 9 comments
Closed

Index out of bounds at huffman_tree.rs line 252 when reading invalid data #23

Pr0methean opened this issue Feb 29, 2024 · 9 comments

Comments

@Pr0methean
Copy link
Contributor

When trying to decompress an invalid file, I get the following panic. I'll post the reproducing steps ASAP.

thread '<unnamed>' panicked at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/deflate64-0.1.7/src/huffman_tree.rs:252:21:
index out of bounds: the len is 64 but the index is 64
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
==4066== ERROR: libFuzzer: deadly signal
    #0 0x561a8eb7d991  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x20a991) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #1 0x561a8eeb250e  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x53f50e) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #2 0x561a8eeba959  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x547959) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #3 0x7f2ecfa4251f  (/lib/x86_64-linux-gnu/libc.so.6+0x4251f) (BuildId: c289da5071a3399de893d2af81d6a30c62646e1e)
    #4 0x7f2ecfa969fb  (/lib/x86_64-linux-gnu/libc.so.6+0x969fb) (BuildId: c289da5071a3399de893d2af81d6a30c62646e1e)
    #5 0x7f2ecfa42475  (/lib/x86_64-linux-gnu/libc.so.6+0x42475) (BuildId: c289da5071a3399de893d2af81d6a30c62646e1e)
    #6 0x7f2ecfa287f2  (/lib/x86_64-linux-gnu/libc.so.6+0x287f2) (BuildId: c289da5071a3399de893d2af81d6a30c62646e1e)
    #7 0x561a8ef58a06  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x5e5a06) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #8 0x561a8ead97f6  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x1667f6) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #9 0x561a8ee9d014  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x52a014) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #10 0x561a8ef4daaf  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x5daaaf) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #11 0x561a8ef4d7f1  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x5da7f1) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #12 0x561a8ef4ada5  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x5d7da5) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #13 0x561a8ef4d523  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x5da523) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #14 0x561a8eadc184  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x169184) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #15 0x561a8eadc371  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x169371) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #16 0x561a8eddc3a4  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x4693a4) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #17 0x561a8eddab1b  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x467b1b) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #18 0x561a8ede4189  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x471189) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #19 0x561a8edde7da  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x46b7da) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #20 0x561a8edddb50  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x46ab50) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #21 0x561a8ec35c3a  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x2c2c3a) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #22 0x561a8ec53552  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x2e0552) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #23 0x561a8ec56f34  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x2e3f34) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #24 0x561a8ec5ebc9  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x2ebbc9) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #25 0x561a8ebb1bb4  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x23ebb4) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #26 0x561a8ebc5868  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x252868) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #27 0x561a8ebb3326  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x240326) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #28 0x561a8ebe3432  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x270432) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #29 0x561a8ebe4272  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x271272) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #30 0x561a8ebe3c50  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x270c50) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #31 0x561a8ee97fff  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x524fff) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #32 0x561a8ee9d227  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x52a227) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #33 0x561a8ee9c74b  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x52974b) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #34 0x561a8eebaea5  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x547ea5) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #35 0x561a8eec02c3  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x54d2c3) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #36 0x561a8eec12b8  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x54e2b8) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #37 0x561a8eec36a7  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x5506a7) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #38 0x561a8eeabf3f  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x538f3f) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #39 0x561a8eadc9b6  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x1699b6) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
    #40 0x7f2ecfa29d8f  (/lib/x86_64-linux-gnu/libc.so.6+0x29d8f) (BuildId: c289da5071a3399de893d2af81d6a30c62646e1e)
    #41 0x7f2ecfa29e3f  (/lib/x86_64-linux-gnu/libc.so.6+0x29e3f) (BuildId: c289da5071a3399de893d2af81d6a30c62646e1e)
    #42 0x561a8eadcb54  (/home/runner/work/zip-next/zip-next/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_read+0x169b54) (BuildId: 9461613681e50016bf6b7ede80d99caa605d2588)
NOTE: libFuzzer has rudimentary signal handlers.
      Combine libFuzzer with AddressSanitizer or similar for better crash reports.
SUMMARY: libFuzzer: deadly signal
@anatawa12
Copy link
Owner

Could you show us example file that cuses this problem?

@Pr0methean
Copy link
Contributor Author

Still working on obtaining an example. At the time I filed this report, my CI workflow had two bugs, one that prevented it from uploading the failed test cases due to the workflow-level timeout and another that prevented me from relaunching it manually. Now that both bugs are fixed, I'll be repeating the CI run until the error recurs.

@Pr0methean
Copy link
Contributor Author

Pr0methean commented Mar 3, 2024 •
edited
Loading

Here's the input to (Deflate64Decoder as Read)::read in xxd format (obtained with the lldb command memory read -s1 -fy -c444 input[0], where 444 was input.len() or equivalently input[1]):

0x7fa098008200: 45 ff ff ff ff ff ff 45 45 45 45 45 45 3d 3d 3d
0x7fa098008210: 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 2a
0x7fa098008220: 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 45 45 45
0x7fa098008230: 45 44 45 f0 45 45 45 45 45 45 45 45 45 45 45 45
0x7fa098008240: 45 45 45 45 45 06 05 4b 50 45 45 45 45 45 45 45
0x7fa098008250: 45 45 45 45 45 45 45 45 45 45 45 45 45 45 45 45
0x7fa098008260: 45 45 45 45 45 45 45 45 45 45 45 3e 45 45 45 45
0x7fa098008270: 45 45 45 01 00 00 00 00 00 00 10 45 45 45 45 45
0x7fa098008280: 45 45 45 45 45 45 45 ec ec ec ec ec ec ec ec ec
0x7fa098008290: ec ec ec ec ec ec ec ec ec ec ec ec ec ec ec ec
0x7fa0980082a0: ec ec ec ec ec ec ec ec ec ec ec ec ec ec ec ec
0x7fa0980082b0: ec ec 81 81 81 81 81 81 81 81 81 81 81 81 81 81
0x7fa0980082c0: 07 05 4b 50 81 81 81 81 81 30 01 00 00 00 00 00
0x7fa0980082d0: 00 81 81 81 81 81 81 81 81 81 81 81 81 81 81 81
0x7fa0980082e0: 81 81 81 81 81 81 81 81 81 81 00 00 00 00 00 45
0x7fa0980082f0: 45 45 45 45 45 3d 2b 3d 3d 3d 3d 3d 3d 3d 3d 3d
0x7fa098008300: 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d
0x7fa098008310: 3d 3d 3d 3d 3d 3d 3d 3d 45 45 45 45 45 45 45 45
0x7fa098008320: 45 45 45 45 45 45 45 45 c0 c0 c0 c0 c0 c0 c0 c0
0x7fa098008330: 00 23 00 00 00 00 00 00 05 06 00 00 01 00 00 00
0x7fa098008340: 09 04 fd ff 06 05 4b 50 00 00 00 02 00 84 ff ff
0x7fa098008350: ff 6b 01 00 00 00 00 45 81 81 81 81 81 81 49 81
0x7fa098008360: 81 81 81 81 81 81 81 07 05 4b 50 81 81 81 81 81
0x7fa098008370: 81 81 81 81 81 81 81 4a 81 81 81 81 81 81 81 81
0x7fa098008380: 81 81 81 81 81 81 81 81 81 81 81 00 00 00 00 00
0x7fa098008390: 45 45 45 45 45 45 3d 3d 01 00 00 00 00 00 00 10
0x7fa0980083a0: 3d 3d 3d 3d ff ff ff ff ff ff 3d 3d 3d 3d 3d 3d
0x7fa0980083b0: 3d 3d 3d 3d 3d 3d 3d 3d 3d 50 bf 00

@Pr0methean
Copy link
Contributor Author

Pr0methean commented Mar 3, 2024 •
edited
Loading

In HuffmanTree::create_table, self is equal at the time of the crash to:

HuffmanTree {
  code_lengths_length: 32,
  table: iter::once(-32).chain(iter::repeat(0).take(511)).collect(),
  left: iter::repeat(0).take(32).chain([-33, -39, -35, -36, -37, 0, 2, -40, -41, 3, -43, 5, 8, -46, -47, 10, 11, 13, -51, 16, 18, -54, 19, -56, -57, -58, -59, 21, 24, -62, 26, 27].into_iter()).chain(iter::repeat(0).take(512)).collect(),
  right: iter::repeat(0).take(32).chain([-55, -34, -45, -42, -38, 1, 4, -53, -48, 7, -44, 6, 9, -50, -49, 12, 15, 14, -52, 17, 20, -63, 23, 0, 0, -61, -60, 22, 25, -64, 28, 0].into_iter()).chain(iter::repeat(0).take(512)).collect(),
  code_length_array: iter::repeat([15, 15, 15, 14]).take(8).flatten().chain(iter::repeat(0).take(256)).collect(),
}

Here are some of the other local variables:

  avail=65,
  len=15,
  ch=29,
  start=0x3200,
  overflow_bits=0,
  code_bit_mask=0x8000,
  index=64,

@Pr0methean
Copy link
Contributor Author

To reproduce this issue the exact same way I originally encountered it, run the unit test added in this commit: zip-rs/zip2@aff3f8d.

@anatawa12
Copy link
Owner

I tried to unzip the zip file with info-zip, windows 11, and 7zip and all of them fails to load zip file.

$ unzip file.zip
Archive:  file.zip
error [file.zip]:  NULL central directory offset
  (attempting to process anyway)
   skipping:                         need PK compat. v25.5 (can do v4.5)
error:  expected central file header signature not found (file #1).
  (please check that you have transferred or created the zipfile in the
  appropriate BINARY mode and that you have compiled UnZip properly)
$ unzip --version
caution:  both -n and -o specified; ignoring -o
UnZip 6.00 of 20 April 2009, by Info-ZIP.  Maintained by C. Spieler.  Send
bug reports using http://www.info-zip.org/zip-bug.html; see README for details.
(truncated)

@Pr0methean
Copy link
Contributor Author

Pr0methean commented Mar 4, 2024 •
edited
Loading

Right -- these are intentionally invalid inputs, because the fuzz test is designed to ensure that attempting to unzip one returns an error result rather than panicking.

@Pr0methean
Copy link
Contributor Author

Pr0methean commented Mar 4, 2024 •
edited
Loading

Here's a version with a valid CDR. zipinfo works fine on it on my MacBook (Sonoma 14.3.1).
raw_deflate64_index_out_of_bounds.zip

@anatawa12
Copy link
Owner

Oh, sorry I missed "when reading invalid data" in the title.

anatawa12 added a commit that referenced this issue Mar 8, 2024
@anatawa12
Merge pull request #24 from Pr0methean/master
2bfab78 
Fix #23: return error rather than panic if index is out of bounds
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Pr0methean @anatawa12