Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Several heap buffer overflow issues have been found. #189

Closed
fCorleone opened this issue May 23, 2018 · 17 comments
Closed

Several heap buffer overflow issues have been found. #189

fCorleone opened this issue May 23, 2018 · 17 comments

Comments

@fCorleone
Copy link

A buffer overflow has been discovered. The information is displayed as follows:

==4515==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60b00000a91a at pc 0x7f7e7a5e7df8 bp 0x7ffe878efa50 sp 0x7ffe878ef1f8
READ of size 103 at 0x60b00000a91a thread T0
#0 0x7f7e7a5e7df7 in __asan_memmove (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x8cdf7)
#1 0x40b516 in memmove /usr/include/x86_64-linux-gnu/bits/string3.h:59
#2 0x40b516 in __mkd_trim_line /home/ubuntu/mfc_fuzz/discount/mkdio.c:85
#3 0x4260e0 in codeblock /home/ubuntu/mfc_fuzz/discount/markdown.c:611
#4 0x4260e0 in compile /home/ubuntu/mfc_fuzz/discount/markdown.c:1317
#5 0x42a56e in compile_document /home/ubuntu/mfc_fuzz/discount/markdown.c:1230
#6 0x42a56e in mkd_compile /home/ubuntu/mfc_fuzz/discount/markdown.c:1436
#7 0x40272c in main /home/ubuntu/mfc_fuzz/discount/mkd2html.c:177
#8 0x7f7e7a1b182f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
#9 0x403d48 in _start (/home/ubuntu/mfc_fuzz/discount/mkd2html+0x403d48)

0x60b00000a91a is located 0 bytes to the right of 106-byte region [0x60b00000a8b0,0x60b00000a91a)
allocated by thread T0 here:
#0 0x7f7e7a5f3602 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x98602)
#1 0x41e88d in splitline /home/ubuntu/mfc_fuzz/discount/markdown.c:174
#2 0x41e88d in htmlblock /home/ubuntu/mfc_fuzz/discount/markdown.c:333

SUMMARY: AddressSanitizer: heap-buffer-overflow ??:0 __asan_memmove
Shadow bytes around the buggy address:
0x0c167fff94d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c167fff94e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c167fff94f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c167fff9500: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c167fff9510: fa fa fa fa fa fa 00 00 00 00 00 00 00 00 00 00
=>0x0c167fff9520: 00 00 00[02]fa fa fa fa fa fa fa fa 00 00 00 00
0x0c167fff9530: 00 00 00 00 00 00 00 00 04 fa fa fa fa fa fa fa
0x0c167fff9540: fa fa fd fd fd fd fd fd fd fd fd fd fd fd fd fa
0x0c167fff9550: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
0x0c167fff9560: 00 00 00 00 04 fa fa fa fa fa fa fa fa fa 00 00
0x0c167fff9570: 00 00 00 00 00 00 00 00 00 00 04 fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Heap right redzone: fb
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack partial redzone: f4
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
==4515==ABORTING

The input test case is at: https://github.com/fCorleone/fuzz_programs/blob/master/discount/issue1_testcase
The executable file is mkd2html.

@Orc
Copy link
Owner

Orc commented May 23, 2018

When I run this through mkd2html on my dev machine, I don't get any warnings on the internal debugging malloc that I put into discount (except for the two memory blocks that contain the source filename & output html name, which are expected) so I've a couple of questions:

  1. what are you using as a boundary tester?
  2. can you reproduce this with a less convoluted test case?

@fCorleone
Copy link
Author

fCorleone commented May 26, 2018

Sorry for replying so late.
The issue has been discovered under a fuzzing test tool called AFL (American Fuzz Lop).You won't get warnings when you use cc to compile the code. Sometimes , the program will not crash in certain memory environment , but if you use another compiler to compile the code, the situation will be different. You can try to use clang to compile the code and add address sanitizer as a parameter. The warning will occur. The test case was given by the AFL tool, which will mutate the initial inputs of the program until it catches a crash. I will try to give a less convoluted test case later if I succeed simplifying the test case.
By the way ,the issue has been assigned a CVE-ID, the ID is : CVE-2018-11468.

@fCorleone
Copy link
Author

Some other issues have been found by fuzzing process, the executable file is mkd2html, there are totally 2 issues.
#1:

==20897==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60200000eff2 at pc 0x00000042fde3 bp 0x7ffce5cfad30 sp 0x7ffce5cfad20
READ of size 1 at 0x60200000eff2 thread T0
#0 0x42fde2 in isfootnote /home/ubuntu/mfc_fuzz/discount/markdown.c:354
#1 0x42fde2 in compile_document /home/ubuntu/mfc_fuzz/discount/markdown.c:1209
#2 0x42fde2 in mkd_compile /home/ubuntu/mfc_fuzz/discount/markdown.c:1436
#3 0x40272c in main /home/ubuntu/mfc_fuzz/discount/mkd2html.c:177
#4 0x7f62b016082f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
#5 0x403d48 in _start (/home/ubuntu/mfc_fuzz/discount/mkd2html+0x403d48)

0x60200000eff2 is located 1 bytes to the right of 1-byte region [0x60200000eff0,0x60200000eff1)
allocated by thread T0 here:
#0 0x7f62b05a2602 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x98602)
#1 0x41e88d in splitline /home/ubuntu/mfc_fuzz/discount/markdown.c:174
#2 0x41e88d in htmlblock /home/ubuntu/mfc_fuzz/discount/markdown.c:333

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/ubuntu/mfc_fuzz/discount/markdown.c:354 isfootnote
Shadow bytes around the buggy address:
0x0c047fff9da0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9db0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9dc0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9dd0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9de0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c047fff9df0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa[01]fa
0x0c047fff9e00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9e10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9e20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9e30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9e40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Heap right redzone: fb
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack partial redzone: f4
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
==20897==ABORTING
The input test case is at: https://github.com/fCorleone/fuzz_programs/blob/master/discount/issue2_testcase

#2:

==27490==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60200000efd1 at pc 0x00000041ae5f bp 0x7ffce617bfd0 sp 0x7ffce617bfc0
READ of size 1 at 0x60200000efd1 thread T0
#0 0x41ae5e in islist /home/ubuntu/mfc_fuzz/discount/markdown.c:522
#1 0x426e47 in compile /home/ubuntu/mfc_fuzz/discount/markdown.c:1327
#2 0x42a062 in compile_document /home/ubuntu/mfc_fuzz/discount/markdown.c:1193
#3 0x42a062 in mkd_compile /home/ubuntu/mfc_fuzz/discount/markdown.c:1436
#4 0x40272c in main /home/ubuntu/mfc_fuzz/discount/mkd2html.c:177
#5 0x7ff4edfbb82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
#6 0x403d48 in _start (/home/ubuntu/mfc_fuzz/discount/mkd2html+0x403d48)

0x60200000efd1 is located 0 bytes to the right of 1-byte region [0x60200000efd0,0x60200000efd1)
allocated by thread T0 here:
#0 0x7ff4ee3fd602 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x98602)
#1 0x41e88d in splitline /home/ubuntu/mfc_fuzz/discount/markdown.c:174
#2 0x41e88d in htmlblock /home/ubuntu/mfc_fuzz/discount/markdown.c:333

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/ubuntu/mfc_fuzz/discount/markdown.c:522 islist
Shadow bytes around the buggy address:
0x0c047fff9da0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9db0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9dc0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9dd0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9de0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c047fff9df0: fa fa fa fa fa fa fa fa fa fa[01]fa fa fa 00 07
0x0c047fff9e00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9e10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9e20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9e30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9e40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Heap right redzone: fb
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack partial redzone: f4
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
==27490==ABORTING
The input test case is at:
https://github.com/fCorleone/fuzz_programs/blob/master/discount/issue3_testcase

@Orc
Copy link
Owner

Orc commented May 26, 2018

Once you pointed me at the offending compiler flag I was able to reproduce it (and holy cow the code slows waaaay down when built with -fsanitize=address -- the clang documentation that says it doesn't slow things down much is a baldfaced lie) and fix it (normally I null-terminate input lines by doing an EXPAND(t) = 0; S(t)--; but when I was splitting lines I didn't do that. Putting those two lines in for the split part of the line stopped the runtime sanitary file explosion.)

(And I also found a bug where I was setting the dle of the new line from the dle of the old line, so a bit of code like {3 spaces at the start of the line}<p>embedded html</p>' wasn't processing that ' because the dle jumped over it.)

ANYWAY, I've pushed a little patch that makes -fsanitize=address NOT abend on your issue2_testcase (and I suspect not abend on your issue1 & issue3 testcases, because the offending data block is allocated in splitline())

@carnil
Copy link

carnil commented May 27, 2018

The issues from #189 (comment) have been assigned CVE-2018-11503 and CVE-2018-11504 respectively.

@fCorleone
Copy link
Author

Another issue has been found when I fuzz markdown.Details are displayed as follows:
#3:

==30402==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60200000eff1 at pc 0x000000418988 bp 0x7ffd89495520 sp 0x7ffd89495510
READ of size 1 at 0x60200000eff1 thread T0
#0 0x418987 in quoteblock /home/ubuntu/mfc_fuzz/discount/markdown.c:804
#1 0x427708 in compile /home/ubuntu/mfc_fuzz/discount/markdown.c:1339
#2 0x429ff2 in compile_document /home/ubuntu/mfc_fuzz/discount/markdown.c:1193
#3 0x429ff2 in mkd_compile /home/ubuntu/mfc_fuzz/discount/markdown.c:1436
#4 0x402d5d in main /home/ubuntu/mfc_fuzz/discount/main.c:223
#5 0x7f8a207b482f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
#6 0x403b48 in _start (/home/ubuntu/mfc_fuzz/discount/markdown+0x403b48)

0x60200000eff1 is located 0 bytes to the right of 1-byte region [0x60200000eff0,0x60200000eff1)
allocated by thread T0 here:
#0 0x7f8a20bf6602 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x98602)
#1 0x41e81d in splitline /home/ubuntu/mfc_fuzz/discount/markdown.c:174
#2 0x41e81d in htmlblock /home/ubuntu/mfc_fuzz/discount/markdown.c:333

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/ubuntu/mfc_fuzz/discount/markdown.c:804 quoteblock
Shadow bytes around the buggy address:
0x0c047fff9da0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9db0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9dc0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9dd0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9de0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c047fff9df0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa[01]fa
0x0c047fff9e00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9e10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9e20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9e30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fff9e40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Heap right redzone: fb
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack partial redzone: f4
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
==30402==ABORTING
The input test case is at :https://github.com/fCorleone/fuzz_programs/blob/master/discount/issue4_testcase
The executable file is markdown.

@Orc
Copy link
Owner

Orc commented Jun 15, 2018

How are you calling the markdown program?

@fCorleone
Copy link
Author

I just call markdown without any parameter : ./markdown input, and the input is the test case I record at https://github.com/fCorleone/fuzz_programs/blob/master/discount/issue4_testcase.

@Orc
Copy link
Owner

Orc commented Jun 16, 2018

So you're doing

markdown < issue4_testcase.txt

by itself. What compiler flags are you passing, and what configure.sh options are you using?

I couldn't repeat the fault here, and this is configuring discount as

CC='cc' CFLAGS='-Wall -fsanitize=address' LDFLAGS=' ' ./configure.sh  \
          '--mandir=/usr/local/share/man' \
          '--with-tabstops' \
          '--with-dl=both' \
          '--shared' \
          '--debian-glitch' \
          '--with-fenced-code' \
          '--with-latex' \
          '--h1-title' \
          '--github-checkbox'

@fCorleone
Copy link
Author

I use afl-gcc as the compiler when I run the script configure. The flags are the same.

@Orc
Copy link
Owner

Orc commented Jun 16, 2018

Do you do any of the configure.sh options, or just

CC=afl-gcc ./configure.sh

?

(I'm on a mac, so no afl-gcc for me; clang's sanitize=address is good for catching boundary overflows)

@fCorleone
Copy link
Author

just

 CC=afl-gcc ./configure.sh

clang may work too.

@Orc
Copy link
Owner

Orc commented Jun 16, 2018

Clang on macos (CC='cc' CFLAGS='-Wall -fsanitize=address' LDFLAGS=' ' ./configure.sh) doesn't reproduce the error.

Can you bisect the input file to see if you can reproduce the fault with a shorter test case? If I can't duplicate the error on my build machines I'll have to walk the code by eye to see if I can figure out the problem, and I'd much rather do that with a considerably shorter piece of source.

Repository owner deleted a comment Jun 17, 2018
@Orc
Copy link
Owner

Orc commented Jun 17, 2018

And also which version of discount are you using; top of source or an earlier version?

@Orc
Copy link
Owner

Orc commented Jul 4, 2018

I'm going to close this issue because all of your overflow catches involved splitline(), which I reworked to null-terminate the lines after splitting. If you find anything that makes afl freak out on top-of-source, let me know and I'll reopen it.

@Orc Orc closed this as completed Jul 4, 2018
@Ofirnir123
Copy link

I'm going to close this issue because all of your overflow catches involved splitline(), which I reworked to null-terminate the lines after splitting. If you find anything that makes afl freak out on top-of-source, let me know and I'll reopen it.

Hi, Can you refer me to the commit that fix this issue ?

@Orc
Copy link
Owner

Orc commented May 5, 2019

b002a5a

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants