Invalid read in dfxml_reader.cpp:164 dfxml::file_object_reader::endElement(void*, char const*)

[fgeek]

Hello,

I found invalid read from dfxml_demo in badb3e9 by using afl and ASan. You can use following minimized reproducer to crash dfxml_demo: dfxml-crash-1.txt

Credits: Henri Salo of Nixu Corporation

$ cat dfxml-crash-1.txt
<?l?><fileobject><metadata c=''></metadata><c0eato0><p0og0am></p0og0am><e0ec0tio00e00i0o0me0t><os0s0s0ame></os0s0s0ame><os00e0sio0></os00e0sio0></e0ec0tio00e00i0o0me0t></c0eato0><co0fig00atio0><algo0ithms></algo0ithms></co0fig00atio0><fileobject><file0ame></file0ame><mtime></mtime><atime></atime><hashdigest e=''></hashdigest></fileobject><file0ame></file0ame><filesi0e></filesi0e><atime></atime><hashdigest e=''></hashdigest>

Hexdump

$ hexdump -C dfxml-crash-1.txt
00000000  3c 3f 6c 3f 3e 3c 66 69  6c 65 6f 62 6a 65 63 74  |<?l?><fileobject|
00000010  3e 3c 6d 65 74 61 64 61  74 61 20 63 3d 27 27 3e  |><metadata c=''>|
00000020  3c 2f 6d 65 74 61 64 61  74 61 3e 3c 63 30 65 61  |</metadata><c0ea|
00000030  74 6f 30 3e 3c 70 30 6f  67 30 61 6d 3e 3c 2f 70  |to0><p0og0am></p|
00000040  30 6f 67 30 61 6d 3e 3c  65 30 65 63 30 74 69 6f  |0og0am><e0ec0tio|
00000050  30 30 65 30 30 69 30 6f  30 6d 65 30 74 3e 3c 6f  |00e00i0o0me0t><o|
00000060  73 30 73 30 73 30 61 6d  65 3e 3c 2f 6f 73 30 73  |s0s0s0ame></os0s|
00000070  30 73 30 61 6d 65 3e 3c  6f 73 30 30 65 30 73 69  |0s0ame><os00e0si|
00000080  6f 30 3e 3c 2f 6f 73 30  30 65 30 73 69 6f 30 3e  |o0></os00e0sio0>|
00000090  3c 2f 65 30 65 63 30 74  69 6f 30 30 65 30 30 69  |</e0ec0tio00e00i|
000000a0  30 6f 30 6d 65 30 74 3e  3c 2f 63 30 65 61 74 6f  |0o0me0t></c0eato|
000000b0  30 3e 3c 63 6f 30 66 69  67 30 30 61 74 69 6f 30  |0><co0fig00atio0|
000000c0  3e 3c 61 6c 67 6f 30 69  74 68 6d 73 3e 3c 2f 61  |><algo0ithms></a|
000000d0  6c 67 6f 30 69 74 68 6d  73 3e 3c 2f 63 6f 30 66  |lgo0ithms></co0f|
000000e0  69 67 30 30 61 74 69 6f  30 3e 3c 66 69 6c 65 6f  |ig00atio0><fileo|
000000f0  62 6a 65 63 74 3e 3c 66  69 6c 65 30 61 6d 65 3e  |bject><file0ame>|
00000100  3c 2f 66 69 6c 65 30 61  6d 65 3e 3c 6d 74 69 6d  |</file0ame><mtim|
00000110  65 3e 3c 2f 6d 74 69 6d  65 3e 3c 61 74 69 6d 65  |e></mtime><atime|
00000120  3e 3c 2f 61 74 69 6d 65  3e 3c 68 61 73 68 64 69  |></atime><hashdi|
00000130  67 65 73 74 20 65 3d 27  27 3e 3c 2f 68 61 73 68  |gest e=''></hash|
00000140  64 69 67 65 73 74 3e 3c  2f 66 69 6c 65 6f 62 6a  |digest></fileobj|
00000150  65 63 74 3e 3c 66 69 6c  65 30 61 6d 65 3e 3c 2f  |ect><file0ame></|
00000160  66 69 6c 65 30 61 6d 65  3e 3c 66 69 6c 65 73 69  |file0ame><filesi|
00000170  30 65 3e 3c 2f 66 69 6c  65 73 69 30 65 3e 3c 61  |0e></filesi0e><a|
00000180  74 69 6d 65 3e 3c 2f 61  74 69 6d 65 3e 3c 68 61  |time></atime><ha|
00000190  73 68 64 69 67 65 73 74  20 65 3d 27 27 3e 3c 2f  |shdigest e=''></|
000001a0  68 61 73 68 64 69 67 65  73 74 3e                 |hashdigest>|
000001ab

Valgrind:

==23688== Memcheck, a memory error detector
==23688== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==23688== Using Valgrind-3.14.0 and LibVEX; rerun with -h for copyright info
==23688== Command: builds/dfxml_cpp-clean/bin/dfxml_demo crash.1
==23688==
fi.filename:
  pieces: 0
==23688== Invalid read of size 8
==23688==    at 0x484B3F8: lower_bound (stl_tree.h:1203)
==23688==    by 0x484B3F8: lower_bound (stl_map.h:1239)
==23688==    by 0x484B3F8: operator[] (stl_map.h:495)
==23688==    by 0x484B3F8: dfxml::file_object_reader::endElement(void*, char const*) (dfxml_reader.cpp:164)
==23688==    by 0x4867A15: ??? (in /lib/x86_64-linux-gnu/libexpat.so.1.6.8)
==23688==    by 0x48684BB: ??? (in /lib/x86_64-linux-gnu/libexpat.so.1.6.8)
==23688==    by 0x4865F8A: ??? (in /lib/x86_64-linux-gnu/libexpat.so.1.6.8)
==23688==    by 0x4866E7A: ??? (in /lib/x86_64-linux-gnu/libexpat.so.1.6.8)
==23688==    by 0x486AA37: XML_ParseBuffer (in /lib/x86_64-linux-gnu/libexpat.so.1.6.8)
==23688==    by 0x484937F: dfxml::file_object_reader::read_dfxml(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void (dfxml::file_object&)>) (dfxml_reader.cpp:203)
==23688==    by 0x109211: main (dfxml_demo.cpp:25)
==23688==  Address 0x18 is not stack'd, malloc'd or (recently) free'd
==23688==
==23688==
==23688== Process terminating with default action of signal 11 (SIGSEGV)
==23688==  Access not within mapped region at address 0x18
==23688==    at 0x484B3F8: lower_bound (stl_tree.h:1203)
==23688==    by 0x484B3F8: lower_bound (stl_map.h:1239)
==23688==    by 0x484B3F8: operator[] (stl_map.h:495)
==23688==    by 0x484B3F8: dfxml::file_object_reader::endElement(void*, char const*) (dfxml_reader.cpp:164)
==23688==    by 0x4867A15: ??? (in /lib/x86_64-linux-gnu/libexpat.so.1.6.8)
==23688==    by 0x48684BB: ??? (in /lib/x86_64-linux-gnu/libexpat.so.1.6.8)
==23688==    by 0x4865F8A: ??? (in /lib/x86_64-linux-gnu/libexpat.so.1.6.8)
==23688==    by 0x4866E7A: ??? (in /lib/x86_64-linux-gnu/libexpat.so.1.6.8)
==23688==    by 0x486AA37: XML_ParseBuffer (in /lib/x86_64-linux-gnu/libexpat.so.1.6.8)
==23688==    by 0x484937F: dfxml::file_object_reader::read_dfxml(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void (dfxml::file_object&)>) (dfxml_reader.cpp:203)
==23688==    by 0x109211: main (dfxml_demo.cpp:25)
==23688==  If you believe this happened as a result of a stack
==23688==  overflow in your program's main thread (unlikely but
==23688==  possible), you can try to increase the size of the
==23688==  main thread stack using the --main-stacksize= flag.
==23688==  The main thread stack size used in this run was 8388608.
==23688==
==23688== HEAP SUMMARY:
==23688==     in use at exit: 19,730 bytes in 48 blocks
==23688==   total heap usage: 59 allocs, 11 frees, 94,140 bytes allocated
==23688==
==23688== LEAK SUMMARY:
==23688==    definitely lost: 136 bytes in 1 blocks
==23688==    indirectly lost: 790 bytes in 9 blocks
==23688==      possibly lost: 0 bytes in 0 blocks
==23688==    still reachable: 18,804 bytes in 38 blocks
==23688==         suppressed: 0 bytes in 0 blocks
==23688== Rerun with --leak-check=full to see details of leaked memory
==23688==
==23688== For counts of detected and suppressed errors, rerun with: -v
==23688== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Segmentation fault

[simsong]

Neat. Can you post how you found it with afl and post a fix?

[fgeek]

Yes for afl (I'll get back to this later), but unfortunately no plans to provide fix. Feel free to patch this when you have time, thanks.

[simsong]

Okay. I can figure out the fix. I'm thankful for the error report, and I am looking forward to learning how to use afl.

[fgeek]

This should get you up and running in Debian buster. If you have any additional questions feel free to contact me via henri+dfxml@nerv.fi email.

# as root
/usr/sbin/addgroup user sudo

# general + afl
sudo apt install wget make gcc git autoconf libtool g++ libssl-dev libexpat1-dev
wget https://lcamtuf.coredump.cx/afl/releases/afl-latest.tgz
tar xvzf afl-latest.tgz
rm afl-latest.tgz
cd afl-2.52b/
sudo make
sudo make install

# dfxml
cd ~/
git clone https://github.com/dfxml-working-group/dfxml_cpp.git
cd dfxml_cpp/src
autoreconf --install
CC=afl-gcc CXX=afl-g++ ./configure --prefix=/home/user/build/dfxml_cpp
AFL_USE_ASAN=1 make -j
make install

# always fuzz in memory. you should do this in Linux in /dev/shm or similar -
# alternatively you might use harddrives and break them
mkdir -p /dev/shm/fuzz/dfxml_cpp/in
mkdir -p /dev/shm/fuzz/dfxml_cpp/out
cd ~/
cp -R dfxml_cpp/samples /dev/shm/fuzz/dfxml_cpp/
cd /dev/shm/fuzz/dfxml_cpp/
# to minimize one of the example files to get coverage. afl-fuzz works with
# valid test cases and smaller is better, but most of the time smallest input
# file does not "reveal" all the functionality of the program so you probably
# want to introduce few small test cases for afl in the beginning
afl-tmin -m none -t 1000000000 -i samples/simple.xml -o in/1.xml -- ~build/dfxml_cpp/bin/dfxml_demo @@

# you should see something like following as output of tmin minimizing the
# input file:
#     File size reduced by : 37.88% (to 1801 bytes)
#     Characters simplified : 41.14%
#     Number of execs done : 17434
#     Fruitless execs : path=16927 crash=0 hang=0

# start fuzzer in screen/tmux session.  please note that memory limit is set as
# none due to ASan requiring lots of virtualized memory to operate
cd /dev/shm/fuzz/dfxml_cpp
afl-fuzz -m none -i in -o out -- ~/build/dfxml_cpp/bin/dfxml_demo @@

# use timeout for fuzzed paths to make it faster
afl-fuzz -t 5+ -m none -i in -o out -- ~/build/dfxml_cpp/bin/dfxml_demo @@

# run reproducer for issue reported in cli with ASan build binary
wget https://github.com/dfxml-working-group/dfxml_cpp/files/6759534/dfxml-crash-1.txt && ~/build/dfxml_cpp/bin/dfxml_demo dfxml-crash-1.txt

For more information please see:

• https://lcamtuf.coredump.cx/afl/README.txt
• https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html
• https://github.com/google/sanitizers/wiki/AddressSanitizer
• https://github.com/google/sanitizers (other sanitizers)

You can also utilize https://gitlab.com/rc0r/afl-utils/ to run multiple instances simultaneously.

[fgeek]

Additionally, you might want to make build without using afl nor ASan when you want to run Valgrind or other debuggers.