use-of-uninitialized-value in XMPMeta::RegisterNamespace #1223
Assignees
Labels
Comments
|
Thanks. I've built branch 0.27-maintenance with I hope to get the fix for this into exiv2 v0.27.3 RC2 which is scheduled for Sunday 2020-05-31. |
|
henices <notifications@github.com> writes:
Version : commit 356f862
OS : Fedora Linux
How to reproduce:
build exiv2 binary with MemorySanitizer, run following command:
Would you mind sharing how you managed to build exiv2 with MSAN (I'd
really love to include that in our testing pipeline at some point)? As
that requires that all linked libraries (except libc) are instrumented
with MSAN as well, which is not an easy undertaking.
`exiv2 ./use-of-uninitialized-value-RegisterNamespace.tiff `
```
==3642951==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x7fb8b831729e in XMPMeta::RegisterNamespace(char const*, char const*) /home/henices/tests/exiv2/xmpsdk/src/XMPMeta.cpp:1042:7
This is a bug in the XMPSDK then.
… #1 0x7fb8b84151ae in StartNamespaceDeclHandler(void*, char const*, char const*) /home/henices/tests/exiv2/xmpsdk/src/ExpatAdapter.cpp:263:2
#2 0x7fb8b6e1c4e1 (/lib64/libexpat.so.1+0x54e1)
#3 0x7fb8b6e1f424 (/lib64/libexpat.so.1+0x8424)
#4 0x7fb8b6e22752 (/lib64/libexpat.so.1+0xb752)
#5 0x7fb8b6e2376f (/lib64/libexpat.so.1+0xc76f)
#6 0x7fb8b6e20c42 (/lib64/libexpat.so.1+0x9c42)
#7 0x7fb8b6e2210d (/lib64/libexpat.so.1+0xb10d)
#8 0x7fb8b6e25e7f in XML_ParseBuffer (/lib64/libexpat.so.1+0xee7f)
#9 0x7fb8b841a2ab in ExpatAdapter::ParseBuffer(void const*, unsigned long, bool) /home/henices/tests/exiv2/xmpsdk/src/ExpatAdapter.cpp:135:11
#10 0x7fb8b82d3919 in ProcessUTF8Portion(XMLParserAdapter*, unsigned char const*, unsigned long, bool) /home/henices/tests/exiv2/xmpsdk/src/XMPMeta-Parse.cpp:1053:39
#11 0x7fb8b82cdff5 in XMPMeta::ParseFromBuffer(char const*, unsigned long, unsigned long) /home/henices/tests/exiv2/xmpsdk/src/XMPMeta-Parse.cpp:1224:23
#12 0x7fb8b81d4d6e in WXMPMeta_ParseFromBuffer_1 /home/henices/tests/exiv2/xmpsdk/src/WXMPMeta.cpp:1274:9
#13 0x7fb8b7c1f5b8 in TXMPMeta<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >::ParseFromBuffer(char const*, unsigned long, unsigned long) /home/henices/tests/exiv2/xmpsdk/include/client-glue/TXMPMeta.incl_cpp:903:2
#14 0x7fb8b7c1edac in TXMPMeta<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >::TXMPMeta(char const*, unsigned long) /home/henices/tests/exiv2/xmpsdk/include/client-glue/TXMPMeta.incl_cpp:169:9
#15 0x7fb8b7c0b730 in Exiv2::XmpParser::decode(Exiv2::XmpData&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) /home/henices/tests/exiv2/src/xmp.cpp:606:18
#16 0x7fb8b80c74d0 in Exiv2::Internal::TiffDecoder::decodeXmp(Exiv2::Internal::TiffEntryBase const*) /home/henices/tests/exiv2/src/tiffvisitor_int.cpp:406:17
#17 0x7fb8b80c353e in Exiv2::Internal::TiffDecoder::decodeTiffEntry(Exiv2::Internal::TiffEntryBase const*) /home/henices/tests/exiv2/src/tiffvisitor_int.cpp:550:13
#18 0x7fb8b80c2cd4 in Exiv2::Internal::TiffDecoder::visitEntry(Exiv2::Internal::TiffEntry*) /home/henices/tests/exiv2/src/tiffvisitor_int.cpp:314:9
#19 0x7fb8b7f9bceb in Exiv2::Internal::TiffEntry::doAccept(Exiv2::Internal::TiffVisitor&) /home/henices/tests/exiv2/src/tiffcomposite_int.cpp:895:17
#20 0x7fb8b7f9b9d6 in Exiv2::Internal::TiffComponent::accept(Exiv2::Internal::TiffVisitor&) /home/henices/tests/exiv2/src/tiffcomposite_int.cpp:890:50
#21 0x7fb8b7f9cfe9 in Exiv2::Internal::TiffDirectory::doAccept(Exiv2::Internal::TiffVisitor&) /home/henices/tests/exiv2/src/tiffcomposite_int.cpp:918:19
#22 0x7fb8b7f9b9d6 in Exiv2::Internal::TiffComponent::accept(Exiv2::Internal::TiffVisitor&) /home/henices/tests/exiv2/src/tiffcomposite_int.cpp:890:50
#23 0x7fb8b8048eec in Exiv2::Internal::TiffParserWorker::decode(Exiv2::ExifData&, Exiv2::IptcData&, Exiv2::XmpData&, unsigned char const*, unsigned long, unsigned int, void (Exiv2::Internal::TiffDecoder::* (*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, unsigned int, Exiv2::Internal::IfdId))(Exiv2::Internal::TiffEntryBase const*), Exiv2::Internal::TiffHeaderBase*) /home/henices/tests/exiv2/src/tiffimage_int.cpp:1622:22
#24 0x7fb8b7a6ccf4 in Exiv2::TiffParser::decode(Exiv2::ExifData&, Exiv2::IptcData&, Exiv2::XmpData&, unsigned char const*, unsigned long) /home/henices/tests/exiv2/src/tiffimage.cpp:260:16
#25 0x7fb8b7a6a9fd in Exiv2::TiffImage::readMetadata() /home/henices/tests/exiv2/src/tiffimage.cpp:187:24
#26 0x699d46 in Action::Print::printSummary() /home/henices/tests/exiv2/src/actions.cpp:260:16
#27 0x698d48 in Action::Print::run(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) /home/henices/tests/exiv2/src/actions.cpp:215:62
#28 0x4a0c5a in main /home/henices/tests/exiv2/src/exiv2.cpp:80:29
#29 0x7fb8b6e861a2 in __libc_start_main (/lib64/libc.so.6+0x271a2)
#30 0x42369d in _start (/home/henices/tests/exiv2/build_msan/bin/exiv2+0x42369d)
Uninitialized value was created by a heap allocation
#0 0x44fa0d in malloc /tmp/llvm-project/compiler-rt/lib/msan/msan_interceptors.cpp:901:3
#1 0x7fb8b6e1c71f (/lib64/libexpat.so.1+0x571f)
SUMMARY: MemorySanitizer: use-of-uninitialized-value /home/henices/tests/exiv2/xmpsdk/src/XMPMeta.cpp:1042:7 in XMPMeta::RegisterNamespace(char const*, char const*)
Exiting
```
Please download testcase in https://github.com/henices/pocs/raw/master/use-of-uninitialized-value-RegisterNamespace.tiff
Thanks.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
#1223
|
|
Robin Mills <notifications@github.com> writes:
Thanks. I've built branch 0.27-maintenance with `-DEXIV2_TEAM_USE_SANITIZERS=On` on macOS and it doesn't reproduce this. I'll build on Fedora.
That flag does not add `-fsanitize=memory` because you'd have to have
instrumented all dependent libraries. The issue should show up with
valgrind (and no sanitizers) though.
|
|
I'm having a sniff in your file while Fedora's building. Here's what I see: So your file want to register the namespace The current code can handle |
|
This doesn't link on Fedora, and I'm not familiar with the platform. I'm going to ask @D4N to reproduce this. |
|
@D4N Thanks. I've just read your comment about |
|
Robin Mills <notifications@github.com> writes:
This doesn't link on Fedora, and I'm not familiar with the platform. I'm going to ask @D4N to reproduce this.
```
/usr/bin/ld: cannot find /usr/lib64/libasan.so.5.0.0
dnf install libasan
collect2: error: ld returned 1 exit status
make[2]: *** [src/CMakeFiles/exiv2lib.dir/build.make:706: lib/libexiv2.so.0.27.3.20] Error 1
make[1]: *** [CMakeFiles/Makefile2:520: src/CMakeFiles/exiv2lib.dir/all] Error 2
make: *** [Makefile:147: all] Error 2
***@***.*** build]$ pacman -S libasan
warning: database file for 'core' does not exist
error: you cannot perform this operation unless you are root.
Are you sure this is actually Fedora? Fedora uses dnf as the package
manager, packman is for Arch.
… ***@***.*** build]$ sudo pacman -S libasan
warning: database file for 'core' does not exist
error: target not found: libasan
***@***.*** build]$ sudo pacman -S libasan-devel
warning: database file for 'core' does not exist
error: target not found: libasan-devel
***@***.*** build]$ ^C
***@***.*** build]$
```
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#1223 (comment)
|
|
Robin Mills <notifications@github.com> writes:
@D4N Thanks. I've just read your comment about `-fsanitize=memory`. I'll try that on macOS.
That will *not* be enough, you'll have to build all dependencies of
exiv2 with -fsanitize=memory.
…
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#1223 (comment)
|
|
Thanks @D4N for reminding me about dnf. Presumably short for I can't build this with the And we know that's operating with this input: @henices Can you add some print statements to the code and get this nailed down? I'm going to inspect a more recent Adobe/XMPsdk to see if the code has been reinforced. |
|
Robin Mills <notifications@github.com> writes:
Thanks @D4N for reminding me about dnf. Presumably short for `dnf with me`!
I can't build this with the `memory` sanitizer anywhere (macOS, ubuntu 20.04 or Fedora). Let's assume the report is correct and focus on the code in xmpsdk/src/XMPMeta.cpp
Just use an ordinary build (no sanitizer flags, include debug symbols),
but run it with valgrind. That will give you a comparable result (in
most cases at least).
… ```
/* class-static */ void
XMPMeta::RegisterNamespace ( XMP_StringPtr namespaceURI,
XMP_StringPtr prefix )
{
if ( (*namespaceURI == 0) || (*prefix == 0) ) {
XMP_Throw ( "Empty namespace URI or prefix", kXMPErr_BadParam );
}
XMP_VarString nsURI ( namespaceURI );
XMP_VarString prfix ( prefix );
if ( prfix[prfix.size()-1] != ':' ) prfix += ':';
VerifySimpleXMLName ( prefix, prefix+prfix.size()-1 ); // Exclude the colon.
// Set the new namespace in both maps.
(*sNamespaceURIToPrefixMap)[nsURI] = prfix;
(*sNamespacePrefixToURIMap)[prfix] = nsURI;
} // RegisterNamespace
```
And we know that's operating with this input:
```
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 5.1.2">
```
@henices Can you add some print statements to the code and get this nailed down? I'm going to inspect a more recent Adobe/XMPsdk to see if the code has been reinforced.
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#1223 (comment)
|
|
The latest code is here: And boy-oh-boy, he's been beafed up. There is an issue with the XMPsdk 4.4.0 in Exiv2 concerning namespace uniqueness. I can't remember what it's about - however it looks as though that's being fixed by this code. @henices Any additional information you can provide will get this nailed down, otherwise I'm running out of ideas. |
|
@D4N - what a smart guy you are! (I think I've told you that before.). I did a vanilla build on ubuntu and used valgrind. Sadly, it doesn't die. I used I built with the santizers. bin/exiv2 runs OK on the test file. I try to run the sanitised exiv2 with valgrind. No output and this curious statement: Here's the output of ldd: bin/exiv2 outputs similar (by reading something in /proc: Trying LD_PRELOAD without effect: |
|
Robin Mills <notifications@github.com> writes:
@D4N - what a smart guy you are! (I think I've told you that before.). I did a vanilla build on ubuntu and used valgrind.
```
***@***.***:~/gnu/github/exiv2/0.27-maintenance/build$ valgrind bin/exiv2 /media/psf/Home/Downloads/use-of-uninitialized-value-RegisterNamespace.tiff
==256195== Memcheck, a memory error detector
==256195== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==256195== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==256195== Command: bin/exiv2 /media/psf/Home/Downloads/use-of-uninitialized-value-RegisterNamespace.tiff
==256195==
File name : /media/psf/Home/Downloads/use-of-uninitialized-value-RegisterNamespace.tiff
File size : 6925 Bytes
MIME type : image/tiff
Image size : 174 x 38
Camera make :
Camera model :
Image timestamp :
Image number :
Exposure time :
Aperture :
Exposure bias :
Flash :
Flash bias :
Focal length :
Subject distance:
ISO speed :
Exposure mode :
Metering mode :
Macro mode :
Image quality :
Exif Resolution : 174 x 38
White balance :
Thumbnail : None
Copyright :
Exif comment :
==256195==
==256195== HEAP SUMMARY:
==256195== in use at exit: 0 bytes in 0 blocks
==256195== total heap usage: 3,456 allocs, 3,456 frees, 220,536 bytes allocated
==256195==
==256195== All heap blocks were freed -- no leaks are possible
==256195==
==256195== For lists of detected and suppressed errors, rerun with: -s
==256195== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
***@***.***:~/gnu/github/exiv2/0.27-maintenance/build$
```
Sadly, it doesn't die. I used `valgrind -s bin/exiv2 path.....`. Same result.
Hm, memory sanitizer doesn't check the same things that valgrind does,
so that could be an explanation. Or the reporter did not instrument all
their libraries with memory sanitizer, which would mean that they got a
false positive.
I built with the santizers. bin/exiv2 runs OK on the test file.
Not really surprising as ASAN and UBSAN test completely different things
than MSAN.
I try to run the sanitised exiv2 with valgrind. No output and this curious statement:
```
==259193==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.
Running ASAN and valgrind together is not supported/cannot work: both
try check the memory access of the application. Doing this twice for the
same binary simply cannot work.
… ```
Here's the output of ldd:
```
***@***.***:~/gnu/github/exiv2/0.27-maintenance/build$ ldd bin/exiv2
linux-vdso.so.1 (0x00007ffcbe5ae000)
libasan.so.5 => /lib/x86_64-linux-gnu/libasan.so.5 (0x00007f72c5f48000)
libexiv2.so.27 => /home/rmills/gnu/github/exiv2/0.27-maintenance/build/lib/libexiv2.so.27 (0x00007f72c2ccf000)
libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f72c2aee000)
libubsan.so.1 => /lib/x86_64-linux-gnu/libubsan.so.1 (0x00007f72c2181000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f72c2166000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f72c1f74000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f72c1f6c000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f72c1f61000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f72c1f3e000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f72c1def000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f72c1dd3000)
libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f72c1da5000)
/lib64/ld-linux-x86-64.so.2 (0x00007f72c6d36000)
***@***.***:~/gnu/github/exiv2/0.27-maintenance/build$
```
bin/exiv2 outputs similar (by reading something in /proc:
```
***@***.***:~/gnu/github/exiv2/0.27-maintenance/build$ bin/exiv2 -vVg library
exiv2 0.27.3.20
library=/usr/lib/x86_64-linux-gnu/libexpat.so.1.6.11
library=/usr/lib/x86_64-linux-gnu/libz.so.1.2.11
library=/usr/lib/x86_64-linux-gnu/libm-2.31.so
library=/usr/lib/x86_64-linux-gnu/libpthread-2.31.so
library=/usr/lib/x86_64-linux-gnu/librt-2.31.so
library=/usr/lib/x86_64-linux-gnu/libdl-2.31.so
library=/usr/lib/x86_64-linux-gnu/libc-2.31.so
library=/usr/lib/x86_64-linux-gnu/libgcc_s.so.1
library=/usr/lib/x86_64-linux-gnu/libubsan.so.1.0.0
library=/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28
library=/home/rmills/gnu/github/exiv2/0.27-maintenance/build/lib/libexiv2.so.0.27.3.20
library=/usr/lib/x86_64-linux-gnu/libasan.so.5.0.0
library=/usr/lib/x86_64-linux-gnu/ld-2.31.so
***@***.***:~/gnu/github/exiv2/0.27-maintenance/build$
```
Trying LD_PRELOAD without effect:
```
***@***.***:~/gnu/github/exiv2/0.27-maintenance/build$ env LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libasan.so.5.0.0 valgrind bin/exiv2 /media/psf/Home/Downloads/use-of-uninitialized-value-RegisterNamespace.tiff
==262004== Memcheck, a memory error detector
==262004== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==262004== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==262004== Command: bin/exiv2 /media/psf/Home/Downloads/use-of-uninitialized-value-RegisterNamespace.tiff
==262004==
==262004==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.
==262004==
==262004== HEAP SUMMARY:
==262004== in use at exit: 0 bytes in 0 blocks
==262004== total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==262004==
==262004== All heap blocks were freed -- no leaks are possible
==262004==
==262004== For lists of detected and suppressed errors, rerun with: -s
==262004== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
***@***.***:~/gnu/github/exiv2/0.27-maintenance/build$
```
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#1223 (comment)
|
|
I think I know what's wrong here. RegisterNamespace uses two maps namespace->prefix and prefix->namespace. XMPsdk 4.4.0 does not allow two or more prefixes for the same namespace. I suspect that's a bug in XMPsdk. It would be very helpful for @henices to add a couple of debugging statements to his code in XMPMeta.cpp. Something like this, then we would know exactly which of the 70 calls to RegisterNamespace is causing the damage. |
|
@henices Can you try this patch please. Caution: This is work-in-progress and must not escape into any product or production environment. |
|
I've reproduced this on ubuntu by building with the command: However, exiv2 dies long before XMPMeta::RegisterNamespace. I've tried building with that command on macOS. exiv2 ran normally. I'm not convinced that exiv2 (and exiv2.dylib) are linked to the msan libraries! It's Friday afternoon. Exiv2 v0.27.3 RC2 will be released on schedule on Sunday. I'll continue to investigate this matter. When I know more, I'll have to choose between:
|
|
Robin Mills <notifications@github.com> writes:
SUMMARY: MemorySanitizer: use-of-uninitialized-value (/home/rmills/gnu/github/exiv2/0.27-maintenance/build/bin/exiv2+0x4d0ec0) in std::char_traits<char>::assign(char&, char const&)
This is a typical false positive that you get when not all of your
libraries are instrumented with MSAN (in this case libstdc++). To get
anything useful out of a MSAN instrumented library, you have to build
*all* libraries with MSAN as well (yes including libstdc++). That's why
it is hardly done for C++, because essentially rebuilding your whole
distro including libstdc++ is *hard*.
|
|
https://clang.llvm.org/docs/MemorySanitizer.html MemorySanitizer is known to work on large real-world programs (like Clang/LLVM itself) that can be recompiled from source, including all dependent libraries. Exiv2 uses 44 libraries on macOS. I don't know if the source is available for that lot (probably is). I'm not volunteering to do this. |
|
Robin Mills <notifications@github.com> writes:
https://clang.llvm.org/docs/MemorySanitizer.html
_MemorySanitizer is known to work on large real-world programs (like Clang/LLVM itself) that can be recompiled from source, **including all dependent libraries**._
Exiv2 uses 44 libraries on macOS. I don't know if the source is available for that lot (probably is). I'm not volunteering to do this.
Yeah, that's why we haven't done that already…
It is probably doable on Gentoo, but I have never tried.
|
|
I'm going to remove this from milestone v0.27.3. Without input from @henices, we can't make progress. |
|
Saturday. A bright new day and some new ideas. 1 This is caused by the file 2 Build Exiv2 with newer XMPsdk The challenge here is that I haven't reproduced the crash. So, even if I build this and it works, I haven't discovered anything. 3 Build libc++ with MASN The call into RegisterNamespace() takes place almost instantly in exiv2. The very first thing all our sample applications do is: And XmpParser::initialise() is: We know the order in which the libraries are being loaded: If I can build libc++ with MSAN, we can probably avoid a false positive from elsewhere. I'm not sure I can build libSystem. We can build libz and libexpat. However the issue is coming from using RegisterNamespace() with data in the file. By the time we arrive in Image::readMetadata(), we've been opening and reading files and almost certainly have pulled in most of the universe. So building libc++ with MSAN isn't going to help if we use exiv2 as the test harness! So, the brings me to my best idea: 4 Simulate this without libexiv2 In reality, all that exiv2 has done here is:
We know the crash happens in |
|
Progress Update. I'm going off to cut the grass now. @henices Are you willing to work with me and @D4N to solve this? Please understand that we haven't reproduced this, it's occurring in code that we didn't write and has not been touched for about 15 years! The code operates by parsing the XML/xmp with expat. There's a namespace handler |
|
Looks like I'm on my own with this. I've turned on the XMPsdk debugging/trace magic (which involved a little tweak concerning xmpOut and xmpCoreOut. And when it runs: Very puzzling. Not a squeak from XMPsdk about RegisterNamespace. I'll investigate that next. Current Patch: |
|
I've realised that this is another incarnation of the underlying issue in #984 The fix for #984 in 'master' is #1093 and in '0.27-maintenance' is #1125. The fix submitted submitted was "good enough" as it closed the CVE/POC as documented. However it didn't get totally to the bottom of the issue. It has something to do with XMPsdk using delete on data that we passed to him from src/properties.cpp line105. MSAN is incorrectly reporting "uninitialised". As I discovered yesterday, if I throw an XMP_Error() when namespaces are duplicated, it causes trouble in the test suite. There have been big improvements in XMPsdk's namespace handling and it now handles duplication by appending For v0.28, we should consider removing xmpsdk from Exiv2 and get conan to build and link xmpsdk. I haven't had success yet with conan on MinGW and Cygwin. More work ahead. #1224 I've run out of time for v0.27.3 RC2 to work on this. I'm going to revisit #984. I ran out of time in September 2019 while working on Exiv2 v0.27.3 (which was to be released on 2019-09-30). If I can get that "properly" fixed, it's likely the fix for that will also fix this issue. If I'm successful in really fixing #984, I'll give serious thought to releasing Exiv2 v0.27.3 RC3. |
|
Sorry for the delay I was OOO these days. We can build msan with the following steps mkdir build_msan
cd build_msan
export CC=/home/henices/clang/bin/clang
export CXX=/home/henices/clang/bin/clang++
cmake .. -G "Unix Makefiles" -DCMAKE_CXX_FLAGS="-fsanitize=memory -fsanitize-memory-track-origins=2 -fno-omit-frame-pointer -stdlib=libc++ -g" -DCMAKE_C_FLAGS="-fsanitize=memory -fsanitize-memory-track-origins=2 -fno-omit-frame-pointer -stdlib=libc++ -g" -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=memory -stdlib=libc++ -L/home/henices/clang/msan/lib -Wl,-rpath,/home/henices/clang/msan/lib" -DCMAKE_MODULE_LINKER_FLAGS="-fsanitize=memory -stdlib=libc++ -L/home/henices/clang/msan/lib -Wl,-rpath,/home/henices/clang/msan/lib"
make -j4Be aware of the fact, we link with a msan instrument build libc++.so |
|
according the page https://clang.llvm.org/docs/MemorySanitizer.html MSAN only support os: Linux NetBSD and FreeBSD |
|
On June 2, 2020 1:28:01 AM UTC, henices ***@***.***> wrote:
Sorry for the delay I was OOO these days. We can build msan with the following steps
```bash
mkdir build_msan
cd build_msan
export CC=/home/henices/clang/bin/clang
export CXX=/home/henices/clang/bin/clang++
cmake .. -G "Unix Makefiles" -DCMAKE_CXX_FLAGS="-fsanitize=memory -fsanitize-memory-track-origins=2 -fno-omit-frame-pointer -stdlib=libc++ -g" -DCMAKE_C_FLAGS="-fsanitize=memory -fsanitize-memory-track-origins=2 -fno-omit-frame-pointer -stdlib=libc++ -g" -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=memory -stdlib=libc++ -L/home/henices/clang/msan/lib -Wl,-rpath,/home/henices/clang/msan/lib" -DCMAKE_MODULE_LINKER_FLAGS="-fsanitize=memory -stdlib=libc++ -L/home/henices/clang/msan/lib -Wl,-rpath,/home/henices/clang/msan/lib"
make -j4
```
Be aware of the fact, we link with a msan instrument build libc++.so
The more interesting question is: where did you obtain it and would you mind sharing the instructions? Also, I hope you linked exiv2 with an instrumented libexpat, libpng, etc.
…
```
➜ bin ldd ./exiv2 >
linux-vdso.so.1 (0x00007ffe185b3000)
libexiv2.so.27 => /home/henices/tests/exiv2/build_msan/lib/libexiv2.so.27 (0x00007fea0ad11000)
libc++.so.1 => /home/henices/clang/msan/lib/libc++.so.1 (0x00007fea0aadc000)
libm.so.6 => /lib64/libm.so.6 (0x00007fea0a94e000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fea0a92c000)
librt.so.1 => /lib64/librt.so.1 (0x00007fea0a921000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fea0a91a000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fea0a8fe000)
libc.so.6 => /lib64/libc.so.6 (0x00007fea0a735000)
libz.so.1 => /lib64/libz.so.1 (0x00007fea0a71b000)
libexpat.so.1 => /lib64/libexpat.so.1 (0x00007fea0a6ed000)
/lib64/ld-linux-x86-64.so.2 (0x00007fea0c64b000)
```
-- >
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#1223 (comment)
|
|
oss-fuzz provide a shell script: |
|
Thanks for the links! So does that mean that only your libc++ is instrumented?
…On June 2, 2020 5:47:50 AM UTC, henices ***@***.***> wrote:
oss-fuzz provide a shell script:
https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-clang/checkout_build_install_llvm.sh
|
Yes, only libc++ is instrumented. |
|
henices <notifications@github.com> writes:
> Thanks for the links! So does that mean that only your libc++ is instrumented?
> […](#)
> On June 2, 2020 5:47:50 AM UTC, henices ***@***.***> wrote: oss-fuzz provide a shell script: https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-clang/checkout_build_install_llvm.sh
Yes, only libc++ is instrumented.
Ok, have you verified that at **no** point before you get that issue
there is a call into an uninstrumented library? Because if there is,
then this could very well be a false positive (especially as @clanmills
couldn't reproduce this with valgrind).
Could you please try to provide us with the output that @clanmills
requested and preferably try it with a fully instrumented exiv2? If you
want to tackle the later as well, please, please share your build
script! It would be very helpful to us.
|
|
I try to build a msan instrument libexpat.so and then link it to exiv2 binary, the use-of-uninitialized-value error was gone. so it is a false positive, sorry for the mistake. |
|
henices <notifications@github.com> writes:
I try to build a msan instrument libexpat.so and then link it to exiv2 binary, the use-of-uninitialized-value error was gone.
Build script pretty please?
so it is a false positive, sorry for the mistake.
No problem, glad we solved this puzzle!
|
|
Wow! This is incredible. For sure, @D4N has learned some new tricks here. I'm motivated to return to #984. I think that's caused by the namespace duplication issue which is confronted in more recent XMPsdk. It's worth investigating if the "deep down" reason is solved with the latest Adobe code. That would be a very strong motivator to remove the xmpsdk tree in 0.28 and rely on conan to deliver the library. Sometimes doing less is ...... more! I don't know the history behind the xmpsdk source tree in Exiv2. It was there when I joined the project in 2008. Andreas said to me "everything concerning xmpsdk is a pain in the butt!". Great Work, Guys. Thank You. |
Closed
Open
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Version : commit 356f862
OS : Fedora Linux
How to reproduce:
build exiv2 binary with MemorySanitizer, run following command:
exiv2 ./use-of-uninitialized-value-RegisterNamespace.tiffPlease download testcase in https://github.com/henices/pocs/raw/master/use-of-uninitialized-value-RegisterNamespace.tiff
Thanks.
The text was updated successfully, but these errors were encountered: