Stackwalk issue on arm64 #303

ikitayama · 2016-12-21T07:20:10Z

@wrwilliams I'm wondering whether the StackwalkerAPI is fully functional on someone's branch other than master. As of this writing I get:

$ DYNINST_DEBUG_STACKWALK=1 ./a.out 
[walker.C:98] - Creating new stackwalker on current process
[libstate.C:59] - Creating a TrackLibState on pid 873
[linuxbsd-swk.C:266] - Successfully install lib tracker at 0xffff93dcf920
[walker.C:73] - Creating new Walker with proc=0x15992bb0, sym=(nil), step = 1
[steppergroup.C:83] - Constructing new AddrRangeGroup at 0x15b38330
[framestepper.C:50] - Creating FrameStepper 0x15b0e650 with walker 0x15b9cd10
[framestepper_pimple.h:74] - Constructing DebugStepper at 0x15b0e650
[framestepper.C:50] - Creating FrameStepper 0x15bfc390 with walker 0x15b9cd10
[walker.C:649] - Registering stepper DebugStepper with group 0x15b38330
[steppergroup.C:150] - Adding stepper DebugStepper to address ranges 0 -> ffffffffffffffff
[linux-aarch64-swk.C:89] - Stepper 0x15b0e650 is DebugStepper
[symlookup.C:49] - Creating SymbolLookup 0x15bacea0
[walker.C:115] - Successfully created Walker 0x15b9cd10
[frame.C:96] - Created frame at 0xffffd8b61c80
[walker.C:391] - Starting stackwalk on thread 873
Segmentation fault (core dumped)

The text was updated successfully, but these errors were encountered:

wrwilliams · 2016-12-21T23:12:34Z

It should be fully functional on master, and that code hasn't changed other than your proccontrol changes in months (at least in any arm-specific way). Can you get a mutator-side stack trace for this crash?

ikitayama · 2016-12-21T23:31:22Z

@wrwilliams how do I do that?

wrwilliams · 2016-12-22T16:26:35Z

Run the mutator under gdb and get the backtrace from the crash? It shouldn't be different from any other crashing program.

wrwilliams · 2016-12-22T19:04:49Z

I've reproduced what looks like this issue locally. Third party stack walking looks okay from here; it's only first party that's broken. I would guess that we're missing -fno-omit-frame-pointer or that the gcc flag for forcing a frame pointer has changed; the actual logic here should be stable.

I'm checking the build system now but I'd suggest disabling cotire for stackwalker and testing again. The code as written should work if that flag is functional.

ETA: the frame pointer flag is correct, per a clean/make VERBOSE=1. @ssunny7 do you mind taking a look at this one? Reproducible with:

cd testsuite/src/stackwalker
g++ test_basic.C -g -O2 -I$DYNINST_INSTALL/include -L$DYNINST_INSTALL/lib -lstackwalk -lcommon -o test_basic
test_basic -self

Not sure why r29/r30 are not pointing to where the caller frame is saved, but they evidently aren't...

wrwilliams · 2016-12-22T21:17:02Z

Should be fixed by #307. I have no clue how this ever worked except by accident.

ikitayama · 2016-12-23T00:01:20Z

This is observed on Mustang.

$ DYNINST_DEBUG_STACKWALK=1 ./test_basic -self 
[walker.C:98] - Creating new stackwalker on current process
[libstate.C:59] - Creating a TrackLibState on pid 20756
[linuxbsd-swk.C:266] - Successfully install lib tracker at 0xffff9d22f920
[walker.C:73] - Creating new Walker with proc=0x94e2bb0, sym=(nil), step = 1
[steppergroup.C:83] - Constructing new AddrRangeGroup at 0x954fbc0
[framestepper.C:50] - Creating FrameStepper 0x954ee10 with walker 0x95de2a0
[framestepper_pimple.h:74] - Constructing DebugStepper at 0x954ee10
[framestepper.C:50] - Creating FrameStepper 0x95455c0 with walker 0x95de2a0
[walker.C:649] - Registering stepper DebugStepper with group 0x954fbc0
[steppergroup.C:150] - Adding stepper DebugStepper to address ranges 0 -> ffffffffffffffff
[linux-aarch64-swk.C:89] - Stepper 0x954ee10 is DebugStepper
[symlookup.C:49] - Creating SymbolLookup 0x95e2c00
[walker.C:115] - Successfully created Walker 0x95de2a0
walker size 1

20756:
[walker.C:595] - getThreadIds returning 1 values:	
20756 
 Stack for process 20756
[frame.C:96] - Created frame at 0xfffffe6740f0
[walker.C:391] - Starting stackwalk on thread 20756
ARM_DEBUG: ra 0x4022e8
[aarch64-swk.C:95] - Returning value 4022e8 for reg abstract::ReturnAddr
ARM_DEBUG: ra 0x4022e8
[aarch64-swk.C:95] - Returning value fffffe6741b0 for reg abstract::StackTop
ARM_DEBUG: ra 0x4022e8
[aarch64-swk.C:95] - Returning value fffffe6741b0 for reg abstract::FrameBase
[frame.C:176] - Setting ra of frame 0xfffffe6740f0 to 4022e8
[frame.C:183] - Setting fp of frame 0xfffffe6740f0 to fffffe6741b0
[frame.C:189] - Setting sp of frame 0xfffffe6740f0 to fffffe6741b0
[frame.C:199] - Setting frame 0xfffffe6740f0 RA location to register abstract::ReturnAddr
[frame.C:199] - Setting frame 0xfffffe6740f0 SP location to register abstract::StackTop
[frame.C:199] - Setting frame 0xfffffe6740f0 FP location to register abstract::FrameBase
[frame.C:140] - Marking frame 0xfffffe6740f0 as top
[walker.C:425] - walkStackFromFrame called with frame at 4022e8
[frame.C:96] - Created frame at 0xfffffe673f60
[walker.C:437] - Walking single frame from 4022e8
[walker.C:484] - Attempting to walk through frame with RA 0x4022e8
[steppergroup.C:92] - AddrRangeGroup trying to find stepper at 4022e8 (last_tried = <NONE>)
[steppergroup.C:111] - Found FrameStepper DebugStepper at address 4022e8
[walker.C:513] - Attempting to use stepper DebugStepper
[linux-swk.C:128] - Found vsyscall over range ffff9d210000 to ffff9d220000
[dbginfo-stepper.C:226] Dwarf-based stackwalking, using local address 0x4022e8 from 0x4022e8 - 0x0
ARM-debug: dump lib========================
/home/itaru.kitayama/projects/testsuite/src/stackwalker/test_basic
ARM-debug: dump lib========================
[dbginfo-stepper.C:267] - Using DWARF debug file info for /home/itaru.kitayama/projects/testsuite/src/stackwalker/test_basic
[frame.C:176] - Setting ra of frame 0xfffffe673f60 to 402568
[frame.C:183] - Setting fp of frame 0xfffffe673f60 to fffffe6742e0
[frame.C:189] - Setting sp of frame 0xfffffe673f60 to fffffe6742e0
[frame.C:196] - Setting frame 0xfffffe673f60 RA location to address fffffe6741b8
[frame.C:196] - Setting frame 0xfffffe673f60 FP location to address fffffe6741b0
[frame.C:202] - Setting frame 0xfffffe673f60 SP location to unknown
[dbginfo-stepper.C:277] - Success walking with DWARF aux file
[walker.C:517] - Success using stepper DebugStepper on 0x4022e8
[walker.C:524] - Returning frame with RA 402568, SP fffffe6742e0, FP fffffe6742e0
[frame.C:134] - Setting frame 0xfffffe673f60's stepper to 0x954ee10
[frame.C:316] - Destroying frame 0xfffffe673e40
[frame.C:316] - Destroying frame 0x9615e80
[frame.C:316] - Destroying frame 0xfffffe673f60
[frame.C:96] - Created frame at 0xfffffe673f60
[walker.C:437] - Walking single frame from 402568
[walker.C:484] - Attempting to walk through frame with RA 0x402568
[steppergroup.C:92] - AddrRangeGroup trying to find stepper at 402568 (last_tried = <NONE>)
[steppergroup.C:111] - Found FrameStepper DebugStepper at address 402568
[walker.C:513] - Attempting to use stepper DebugStepper
[dbginfo-stepper.C:226] Dwarf-based stackwalking, using local address 0x402568 from 0x402568 - 0x0
ARM-debug: dump lib========================
/home/itaru.kitayama/projects/testsuite/src/stackwalker/test_basic
ARM-debug: dump lib========================
[dbginfo-stepper.C:267] - Using DWARF debug file info for /home/itaru.kitayama/projects/testsuite/src/stackwalker/test_basic
[frame.C:176] - Setting ra of frame 0xfffffe673f60 to 4025d0
[frame.C:183] - Setting fp of frame 0xfffffe673f60 to fffffe674300
[frame.C:189] - Setting sp of frame 0xfffffe673f60 to fffffe674300
[frame.C:196] - Setting frame 0xfffffe673f60 RA location to address fffffe6742e8
[frame.C:196] - Setting frame 0xfffffe673f60 FP location to address fffffe6742e0
[frame.C:202] - Setting frame 0xfffffe673f60 SP location to unknown
[dbginfo-stepper.C:277] - Success walking with DWARF aux file
[walker.C:517] - Success using stepper DebugStepper on 0x402568
[walker.C:524] - Returning frame with RA 4025d0, SP fffffe674300, FP fffffe674300
[frame.C:134] - Setting frame 0xfffffe673f60's stepper to 0x954ee10
[frame.C:316] - Destroying frame 0xfffffe673e40
[frame.C:316] - Destroying frame 0x965c790
[frame.C:316] - Destroying frame 0x965c850
[frame.C:316] - Destroying frame 0xfffffe673f60
[frame.C:96] - Created frame at 0xfffffe673f60
[walker.C:437] - Walking single frame from 4025d0
[walker.C:484] - Attempting to walk through frame with RA 0x4025d0
[steppergroup.C:92] - AddrRangeGroup trying to find stepper at 4025d0 (last_tried = <NONE>)
[steppergroup.C:111] - Found FrameStepper DebugStepper at address 4025d0
[walker.C:513] - Attempting to use stepper DebugStepper
[dbginfo-stepper.C:226] Dwarf-based stackwalking, using local address 0x4025d0 from 0x4025d0 - 0x0
ARM-debug: dump lib========================
/home/itaru.kitayama/projects/testsuite/src/stackwalker/test_basic
ARM-debug: dump lib========================
[dbginfo-stepper.C:267] - Using DWARF debug file info for /home/itaru.kitayama/projects/testsuite/src/stackwalker/test_basic
[frame.C:176] - Setting ra of frame 0xfffffe673f60 to 401cd4
[frame.C:183] - Setting fp of frame 0xfffffe673f60 to fffffe674320
[frame.C:189] - Setting sp of frame 0xfffffe673f60 to fffffe674320
[frame.C:196] - Setting frame 0xfffffe673f60 RA location to address fffffe674308
[frame.C:196] - Setting frame 0xfffffe673f60 FP location to address fffffe674300
[frame.C:202] - Setting frame 0xfffffe673f60 SP location to unknown
[dbginfo-stepper.C:277] - Success walking with DWARF aux file
[walker.C:517] - Success using stepper DebugStepper on 0x4025d0
[walker.C:524] - Returning frame with RA 401cd4, SP fffffe674320, FP fffffe674320
[frame.C:134] - Setting frame 0xfffffe673f60's stepper to 0x954ee10
[frame.C:316] - Destroying frame 0xfffffe673e40
[frame.C:316] - Destroying frame 0xfffffe673f60
[frame.C:96] - Created frame at 0xfffffe673f60
[walker.C:437] - Walking single frame from 401cd4
[walker.C:484] - Attempting to walk through frame with RA 0x401cd4
[steppergroup.C:92] - AddrRangeGroup trying to find stepper at 401cd4 (last_tried = <NONE>)
[steppergroup.C:111] - Found FrameStepper DebugStepper at address 401cd4
[walker.C:513] - Attempting to use stepper DebugStepper
[dbginfo-stepper.C:226] Dwarf-based stackwalking, using local address 0x401cd4 from 0x401cd4 - 0x0
ARM-debug: dump lib========================
/home/itaru.kitayama/projects/testsuite/src/stackwalker/test_basic
ARM-debug: dump lib========================
[dbginfo-stepper.C:267] - Using DWARF debug file info for /home/itaru.kitayama/projects/testsuite/src/stackwalker/test_basic
[frame.C:176] - Setting ra of frame 0xfffffe673f60 to ffff9cc1f3f0
[frame.C:183] - Setting fp of frame 0xfffffe673f60 to fffffe674540
[frame.C:189] - Setting sp of frame 0xfffffe673f60 to fffffe674540
[frame.C:196] - Setting frame 0xfffffe673f60 RA location to address fffffe674328
[frame.C:196] - Setting frame 0xfffffe673f60 FP location to address fffffe674320
[frame.C:202] - Setting frame 0xfffffe673f60 SP location to unknown
[dbginfo-stepper.C:277] - Success walking with DWARF aux file
[walker.C:517] - Success using stepper DebugStepper on 0x401cd4
[walker.C:524] - Returning frame with RA ffff9cc1f3f0, SP fffffe674540, FP fffffe674540
[frame.C:134] - Setting frame 0xfffffe673f60's stepper to 0x954ee10
[frame.C:316] - Destroying frame 0xfffffe673e40
[frame.C:316] - Destroying frame 0x965c920
[frame.C:316] - Destroying frame 0x965c9e0
[frame.C:316] - Destroying frame 0x965caa0
[frame.C:316] - Destroying frame 0x965cb60
[frame.C:316] - Destroying frame 0xfffffe673f60
[frame.C:96] - Created frame at 0xfffffe673f60
[walker.C:437] - Walking single frame from ffff9cc1f3f0
[walker.C:484] - Attempting to walk through frame with RA 0xffff9cc1f3f0
[steppergroup.C:92] - AddrRangeGroup trying to find stepper at ffff9cc1f3f0 (last_tried = <NONE>)
[steppergroup.C:111] - Found FrameStepper DebugStepper at address ffff9cc1f3f0
[walker.C:513] - Attempting to use stepper DebugStepper
[dbginfo-stepper.C:226] Dwarf-based stackwalking, using local address 0x1f3f0 from 0xffff9cc1f3f0 - 0xffff9cc00000
ARM-debug: dump lib========================
/lib64/libc.so.6
ARM-debug: dump lib========================
[dbginfo-stepper.C:267] - Using DWARF debug file info for /lib64/libc.so.6
[frame.C:176] - Setting ra of frame 0xfffffe673f60 to 401e80
[frame.C:183] - Setting fp of frame 0xfffffe673f60 to 0
[frame.C:189] - Setting sp of frame 0xfffffe673f60 to fffffe674670
[frame.C:196] - Setting frame 0xfffffe673f60 RA location to address fffffe674548
[frame.C:196] - Setting frame 0xfffffe673f60 FP location to address fffffe674540
[frame.C:202] - Setting frame 0xfffffe673f60 SP location to unknown
[dbginfo-stepper.C:277] - Success walking with DWARF aux file
[walker.C:517] - Success using stepper DebugStepper on 0xffff9cc1f3f0
[walker.C:524] - Returning frame with RA 401e80, SP fffffe674670, FP 0
[frame.C:134] - Setting frame 0xfffffe673f60's stepper to 0x954ee10
[frame.C:316] - Destroying frame 0xfffffe673e40
[frame.C:316] - Destroying frame 0xfffffe673f60
[frame.C:96] - Created frame at 0xfffffe673f60
[walker.C:437] - Walking single frame from 401e80
[walker.C:484] - Attempting to walk through frame with RA 0x401e80
[steppergroup.C:92] - AddrRangeGroup trying to find stepper at 401e80 (last_tried = <NONE>)
[steppergroup.C:111] - Found FrameStepper DebugStepper at address 401e80
[walker.C:513] - Attempting to use stepper DebugStepper
[dbginfo-stepper.C:226] Dwarf-based stackwalking, using local address 0x401e80 from 0x401e80 - 0x0
ARM-debug: dump lib========================
/home/itaru.kitayama/projects/testsuite/src/stackwalker/test_basic
ARM-debug: dump lib========================
[dbginfo-stepper.C:267] - Using DWARF debug file info for /home/itaru.kitayama/projects/testsuite/src/stackwalker/test_basic
[dbginfo-stepper.C:507] - Couldn't get return debug info at 401e80, error: 16
*** Error in `./test_basic': free(): invalid pointer: 0x0000fffffe673408 ***

wrwilliams · 2016-12-23T16:54:58Z

@mxz297 Does this match a plausible code path for the libdwarf double-frees you've seen?
@ikitayama Can you run this under valgrind and find the specific thing being freed at the crash point?

mxz297 · 2016-12-23T17:09:00Z

This does not look like crashing during opening a file with SymtabAPI, which is what I encountered. But I agree that running valgrind can help further.

ikitayama · 2016-12-23T21:04:58Z

@wrwilliams @mxz297 please have a look at https://gist.github.com/ikitayama/fba4d7f0ff1f913b1a2143b97a5b62f6

ikitayama · 2016-12-23T23:27:41Z

Running it with DYNISINT_DEBUG_TRANSLATE=1
https://gist.github.com/ikitayama/ea76e88c6987440024d8052200930d09

ikitayama · 2016-12-24T06:21:47Z

@wrwilliams @mxz297 does the below look reasonable?

valgrind ./test_basic -self
==23856== Memcheck, a memory error detector
==23856== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==23856== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==23856== Command: ./test_basic -self
==23856== 
walker size 1

23856:
Stack for process 23856
==23856== Conditional jump or move depends on uninitialised value(s)
==23856==    at 0x5809D28: dwarf_dealloc (in /usr/local/lib/libdwarf.so)
==23856==    by 0x55E5F8B: Dyninst::SymtabAPI::DwarfWalker::buildSrcFiles(Dwarf_Debug_s*, Dwarf_Die_s*, boost::shared_ptr<boost::multi_index::multi_index_container<Dyninst::SymtabAPI::StringTableEntry, boost::multi_index::indexed_by<boost::multi_index::random_access<boost::multi_index::tag<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na> >, boost::multi_index::ordered_non_unique<boost::multi_index::member<Dyninst::SymtabAPI::StringTableEntry, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, &Dyninst::SymtabAPI::StringTableEntry::str>, mpl_::na, mpl_::na>, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, std::allocator<Dyninst::SymtabAPI::StringTableEntry> > >) (dwarfWalker.C:263)
==23856==    by 0x55A1CFF: Dyninst::SymtabAPI::Object::fix_global_symbol_modules_static_dwarf() (Object-elf.C:2555)
==23856==    by 0x55A494B: Dyninst::SymtabAPI::Object::load_object(bool) (Object-elf.C:1599)
==23856==    by 0x55A538F: Dyninst::SymtabAPI::Object::Object(MappedFile*, bool, void (*)(char const*), bool, Dyninst::SymtabAPI::Symtab*) (Object-elf.C:2956)
==23856==    by 0x55667B7: Dyninst::SymtabAPI::Symtab::Symtab(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, bool&) (Symtab.C:1270)
==23856==    by 0x5566B2F: Dyninst::SymtabAPI::Symtab::openFile(Dyninst::SymtabAPI::Symtab*&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, Dyninst::SymtabAPI::Symtab::def_t) (Symtab.C:2105)
==23856==    by 0x5594677: Dyninst::SymtabAPI::SymtabReader::SymtabReader(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (SymtabReader.C:53)
==23856==    by 0x559551B: Dyninst::SymtabAPI::SymtabReaderFactory::openSymbolReader(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (SymtabReader.C:288)
==23856==    by 0x49BBEF7: Dyninst::FCNode::parsefile() (addrtranslate-sysv.C:927)
==23856==    by 0x49BCCAB: getSegments (addrtranslate-sysv.C:883)
==23856==    by 0x49BCCAB: Dyninst::LoadedLib::getMappedRegions() (addrtranslate-sysv.C:347)
==23856==    by 0x498158B: Dyninst::AddressTranslate::getLibAtAddress(unsigned long, Dyninst::LoadedLib*&) (addrtranslate.C:108)
==23856== 
==23856== Conditional jump or move depends on uninitialised value(s)
==23856==    at 0x4884F68: free (vg_replace_malloc.c:530)
==23856==    by 0x580A0CF: dwarf_dealloc (in /usr/local/lib/libdwarf.so)
==23856==    by 0x55E5F8B: Dyninst::SymtabAPI::DwarfWalker::buildSrcFiles(Dwarf_Debug_s*, Dwarf_Die_s*, boost::shared_ptr<boost::multi_index::multi_index_container<Dyninst::SymtabAPI::StringTableEntry, boost::multi_index::indexed_by<boost::multi_index::random_access<boost::multi_index::tag<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na> >, boost::multi_index::ordered_non_unique<boost::multi_index::member<Dyninst::SymtabAPI::StringTableEntry, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, &Dyninst::SymtabAPI::StringTableEntry::str>, mpl_::na, mpl_::na>, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, std::allocator<Dyninst::SymtabAPI::StringTableEntry> > >) (dwarfWalker.C:263)
==23856==    by 0x55A1CFF: Dyninst::SymtabAPI::Object::fix_global_symbol_modules_static_dwarf() (Object-elf.C:2555)
==23856==    by 0x55A494B: Dyninst::SymtabAPI::Object::load_object(bool) (Object-elf.C:1599)
==23856==    by 0x55A538F: Dyninst::SymtabAPI::Object::Object(MappedFile*, bool, void (*)(char const*), bool, Dyninst::SymtabAPI::Symtab*) (Object-elf.C:2956)
==23856==    by 0x55667B7: Dyninst::SymtabAPI::Symtab::Symtab(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, bool&) (Symtab.C:1270)
==23856==    by 0x5566B2F: Dyninst::SymtabAPI::Symtab::openFile(Dyninst::SymtabAPI::Symtab*&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, Dyninst::SymtabAPI::Symtab::def_t) (Symtab.C:2105)
==23856==    by 0x5594677: Dyninst::SymtabAPI::SymtabReader::SymtabReader(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (SymtabReader.C:53)
==23856==    by 0x559551B: Dyninst::SymtabAPI::SymtabReaderFactory::openSymbolReader(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (SymtabReader.C:288)
==23856==    by 0x49BBEF7: Dyninst::FCNode::parsefile() (addrtranslate-sysv.C:927)
==23856==    by 0x49BCCAB: getSegments (addrtranslate-sysv.C:883)
==23856==    by 0x49BCCAB: Dyninst::LoadedLib::getMappedRegions() (addrtranslate-sysv.C:347)
==23856== 
==23856== Invalid free() / delete / delete[] / realloc()
==23856==    at 0x4884FB4: free (vg_replace_malloc.c:530)
==23856==    by 0x580A0CF: dwarf_dealloc (in /usr/local/lib/libdwarf.so)
==23856==    by 0x55E5F8B: Dyninst::SymtabAPI::DwarfWalker::buildSrcFiles(Dwarf_Debug_s*, Dwarf_Die_s*, boost::shared_ptr<boost::multi_index::multi_index_container<Dyninst::SymtabAPI::StringTableEntry, boost::multi_index::indexed_by<boost::multi_index::random_access<boost::multi_index::tag<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na> >, boost::multi_index::ordered_non_unique<boost::multi_index::member<Dyninst::SymtabAPI::StringTableEntry, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, &Dyninst::SymtabAPI::StringTableEntry::str>, mpl_::na, mpl_::na>, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, std::allocator<Dyninst::SymtabAPI::StringTableEntry> > >) (dwarfWalker.C:263)
==23856==    by 0x55A1CFF: Dyninst::SymtabAPI::Object::fix_global_symbol_modules_static_dwarf() (Object-elf.C:2555)
==23856==    by 0x55A494B: Dyninst::SymtabAPI::Object::load_object(bool) (Object-elf.C:1599)
==23856==    by 0x55A538F: Dyninst::SymtabAPI::Object::Object(MappedFile*, bool, void (*)(char const*), bool, Dyninst::SymtabAPI::Symtab*) (Object-elf.C:2956)
==23856==    by 0x55667B7: Dyninst::SymtabAPI::Symtab::Symtab(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, bool&) (Symtab.C:1270)
==23856==    by 0x5566B2F: Dyninst::SymtabAPI::Symtab::openFile(Dyninst::SymtabAPI::Symtab*&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, Dyninst::SymtabAPI::Symtab::def_t) (Symtab.C:2105)
==23856==    by 0x5594677: Dyninst::SymtabAPI::SymtabReader::SymtabReader(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (SymtabReader.C:53)
==23856==    by 0x559551B: Dyninst::SymtabAPI::SymtabReaderFactory::openSymbolReader(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (SymtabReader.C:288)
==23856==    by 0x49BBEF7: Dyninst::FCNode::parsefile() (addrtranslate-sysv.C:927)
==23856==    by 0x49BCCAB: getSegments (addrtranslate-sysv.C:883)
==23856==    by 0x49BCCAB: Dyninst::LoadedLib::getMappedRegions() (addrtranslate-sysv.C:347)
==23856==  Address 0xfff00df98 is on thread 1's stack
==23856==  in frame #3, created by Dyninst::SymtabAPI::Object::fix_global_symbol_modules_static_dwarf() (Object-elf.C:2477)
==23856== 
   Stack for thread 23856
   walk_stack3(Dyninst::Stackwalker::Walker*) @ 0x4022f0, FP = 0xfff00ed40    
      RALoc: Register: 2, FPLoc: Register: 1, SPLoc: Register: 3
   walk_stack2(Dyninst::Stackwalker::Walker*) @ 0x402570, FP = 0xfff00ee70    
      RALoc: Addr: 0xfff00ed48 (0x402570), FPLoc: Addr: 0xfff00ed40 (0xfff00ee70), SPLoc: Unknown
   walk_stack1(Dyninst::Stackwalker::Walker*) @ 0x4025d8, FP = 0xfff00ee90    
      RALoc: Addr: 0xfff00ee78 (0x4025d8), FPLoc: Addr: 0xfff00ee70 (0xfff00ee90), SPLoc: Unknown
   main @ 0x401cd4, FP = 0xfff00eeb0    
      RALoc: Addr: 0xfff00ee98 (0x401cd4), FPLoc: Addr: 0xfff00ee90 (0xfff00eeb0), SPLoc: Unknown
   __libc_start_main @ 0x4d2f3f0, FP = 0xfff00f0d0    
      RALoc: Addr: 0xfff00eeb8 (0x4d2f3f0), FPLoc: Addr: 0xfff00eeb0 (0xfff00f0d0), SPLoc: Unknown
   _start @ 0x401e88, FP = 0x0    
      RALoc: Addr: 0xfff00f0d8 (0x401e88), FPLoc: Addr: 0xfff00f0d0 (0x0), SPLoc: Unknown

23856:
Stack for process 23856
   Stack for thread 23856
   walk_stack3(Dyninst::Stackwalker::Walker*) @ 0x4022f0, FP = 0xfff00ed40    
      RALoc: Register: 2, FPLoc: Register: 1, SPLoc: Register: 3
   walk_stack2(Dyninst::Stackwalker::Walker*) @ 0x402570, FP = 0xfff00ee70    
      RALoc: Addr: 0xfff00ed48 (0x402570), FPLoc: Addr: 0xfff00ed40 (0xfff00ee70), SPLoc: Unknown
   walk_stack1(Dyninst::Stackwalker::Walker*) @ 0x4025d8, FP = 0xfff00ee90    
      RALoc: Addr: 0xfff00ee78 (0x4025d8), FPLoc: Addr: 0xfff00ee70 (0xfff00ee90), SPLoc: Unknown
   main @ 0x401cd4, FP = 0xfff00eeb0    
      RALoc: Addr: 0xfff00ee98 (0x401cd4), FPLoc: Addr: 0xfff00ee90 (0xfff00eeb0), SPLoc: Unknown
   __libc_start_main @ 0x4d2f3f0, FP = 0xfff00f0d0    
      RALoc: Addr: 0xfff00eeb8 (0x4d2f3f0), FPLoc: Addr: 0xfff00eeb0 (0xfff00f0d0), SPLoc: Unknown
   _start @ 0x401e88, FP = 0x0    
      RALoc: Addr: 0xfff00f0d8 (0x401e88), FPLoc: Addr: 0xfff00f0d0 (0x0), SPLoc: Unknown

23856:
Stack for process 23856
   Stack for thread 23856
   walk_stack3(Dyninst::Stackwalker::Walker*) @ 0x4022f0, FP = 0xfff00ed40    
      RALoc: Register: 2, FPLoc: Register: 1, SPLoc: Register: 3
   walk_stack2(Dyninst::Stackwalker::Walker*) @ 0x402570, FP = 0xfff00ee70    
      RALoc: Addr: 0xfff00ed48 (0x402570), FPLoc: Addr: 0xfff00ed40 (0xfff00ee70), SPLoc: Unknown
   walk_stack1(Dyninst::Stackwalker::Walker*) @ 0x4025d8, FP = 0xfff00ee90    
      RALoc: Addr: 0xfff00ee78 (0x4025d8), FPLoc: Addr: 0xfff00ee70 (0xfff00ee90), SPLoc: Unknown
   main @ 0x401cd4, FP = 0xfff00eeb0    
      RALoc: Addr: 0xfff00ee98 (0x401cd4), FPLoc: Addr: 0xfff00ee90 (0xfff00eeb0), SPLoc: Unknown
   __libc_start_main @ 0x4d2f3f0, FP = 0xfff00f0d0    
      RALoc: Addr: 0xfff00eeb8 (0x4d2f3f0), FPLoc: Addr: 0xfff00eeb0 (0xfff00f0d0), SPLoc: Unknown
   _start @ 0x401e88, FP = 0x0    
      RALoc: Addr: 0xfff00f0d8 (0x401e88), FPLoc: Addr: 0xfff00f0d0 (0x0), SPLoc: Unknown

23856:
Stack for process 23856
   Stack for thread 23856
   walk_stack3(Dyninst::Stackwalker::Walker*) @ 0x4022f0, FP = 0xfff00ed40    
      RALoc: Register: 2, FPLoc: Register: 1, SPLoc: Register: 3
   walk_stack2(Dyninst::Stackwalker::Walker*) @ 0x402570, FP = 0xfff00ee70    
      RALoc: Addr: 0xfff00ed48 (0x402570), FPLoc: Addr: 0xfff00ed40 (0xfff00ee70), SPLoc: Unknown
   walk_stack1(Dyninst::Stackwalker::Walker*) @ 0x4025d8, FP = 0xfff00ee90    
      RALoc: Addr: 0xfff00ee78 (0x4025d8), FPLoc: Addr: 0xfff00ee70 (0xfff00ee90), SPLoc: Unknown
   main @ 0x401cd4, FP = 0xfff00eeb0    
      RALoc: Addr: 0xfff00ee98 (0x401cd4), FPLoc: Addr: 0xfff00ee90 (0xfff00eeb0), SPLoc: Unknown
   __libc_start_main @ 0x4d2f3f0, FP = 0xfff00f0d0    
      RALoc: Addr: 0xfff00eeb8 (0x4d2f3f0), FPLoc: Addr: 0xfff00eeb0 (0xfff00f0d0), SPLoc: Unknown
   _start @ 0x401e88, FP = 0x0    
      RALoc: Addr: 0xfff00f0d8 (0x401e88), FPLoc: Addr: 0xfff00f0d0 (0x0), SPLoc: Unknown

23856:
Stack for process 23856
   Stack for thread 23856
   walk_stack3(Dyninst::Stackwalker::Walker*) @ 0x4022f0, FP = 0xfff00ed40    
      RALoc: Register: 2, FPLoc: Register: 1, SPLoc: Register: 3
   walk_stack2(Dyninst::Stackwalker::Walker*) @ 0x402570, FP = 0xfff00ee70    
      RALoc: Addr: 0xfff00ed48 (0x402570), FPLoc: Addr: 0xfff00ed40 (0xfff00ee70), SPLoc: Unknown
   walk_stack1(Dyninst::Stackwalker::Walker*) @ 0x4025d8, FP = 0xfff00ee90    
      RALoc: Addr: 0xfff00ee78 (0x4025d8), FPLoc: Addr: 0xfff00ee70 (0xfff00ee90), SPLoc: Unknown
   main @ 0x401cd4, FP = 0xfff00eeb0    
      RALoc: Addr: 0xfff00ee98 (0x401cd4), FPLoc: Addr: 0xfff00ee90 (0xfff00eeb0), SPLoc: Unknown
   __libc_start_main @ 0x4d2f3f0, FP = 0xfff00f0d0    
      RALoc: Addr: 0xfff00eeb8 (0x4d2f3f0), FPLoc: Addr: 0xfff00eeb0 (0xfff00f0d0), SPLoc: Unknown
   _start @ 0x401e88, FP = 0x0    
      RALoc: Addr: 0xfff00f0d8 (0x401e88), FPLoc: Addr: 0xfff00f0d0 (0x0), SPLoc: Unknown

23856:
Stack for process 23856
   Stack for thread 23856
   walk_stack3(Dyninst::Stackwalker::Walker*) @ 0x4022f0, FP = 0xfff00ed40    
      RALoc: Register: 2, FPLoc: Register: 1, SPLoc: Register: 3
   walk_stack2(Dyninst::Stackwalker::Walker*) @ 0x402570, FP = 0xfff00ee70    
      RALoc: Addr: 0xfff00ed48 (0x402570), FPLoc: Addr: 0xfff00ed40 (0xfff00ee70), SPLoc: Unknown
   walk_stack1(Dyninst::Stackwalker::Walker*) @ 0x4025d8, FP = 0xfff00ee90    
      RALoc: Addr: 0xfff00ee78 (0x4025d8), FPLoc: Addr: 0xfff00ee70 (0xfff00ee90), SPLoc: Unknown
   main @ 0x401cd4, FP = 0xfff00eeb0    
      RALoc: Addr: 0xfff00ee98 (0x401cd4), FPLoc: Addr: 0xfff00ee90 (0xfff00eeb0), SPLoc: Unknown
   __libc_start_main @ 0x4d2f3f0, FP = 0xfff00f0d0    
      RALoc: Addr: 0xfff00eeb8 (0x4d2f3f0), FPLoc: Addr: 0xfff00eeb0 (0xfff00f0d0), SPLoc: Unknown
   _start @ 0x401e88, FP = 0x0    
      RALoc: Addr: 0xfff00f0d8 (0x401e88), FPLoc: Addr: 0xfff00f0d0 (0x0), SPLoc: Unknown

23856:
Stack for process 23856
   Stack for thread 23856
   walk_stack3(Dyninst::Stackwalker::Walker*) @ 0x4022f0, FP = 0xfff00ed40    
      RALoc: Register: 2, FPLoc: Register: 1, SPLoc: Register: 3
   walk_stack2(Dyninst::Stackwalker::Walker*) @ 0x402570, FP = 0xfff00ee70    
      RALoc: Addr: 0xfff00ed48 (0x402570), FPLoc: Addr: 0xfff00ed40 (0xfff00ee70), SPLoc: Unknown
   walk_stack1(Dyninst::Stackwalker::Walker*) @ 0x4025d8, FP = 0xfff00ee90    
      RALoc: Addr: 0xfff00ee78 (0x4025d8), FPLoc: Addr: 0xfff00ee70 (0xfff00ee90), SPLoc: Unknown
   main @ 0x401cd4, FP = 0xfff00eeb0    
      RALoc: Addr: 0xfff00ee98 (0x401cd4), FPLoc: Addr: 0xfff00ee90 (0xfff00eeb0), SPLoc: Unknown
   __libc_start_main @ 0x4d2f3f0, FP = 0xfff00f0d0    
      RALoc: Addr: 0xfff00eeb8 (0x4d2f3f0), FPLoc: Addr: 0xfff00eeb0 (0xfff00f0d0), SPLoc: Unknown
   _start @ 0x401e88, FP = 0x0    
      RALoc: Addr: 0xfff00f0d8 (0x401e88), FPLoc: Addr: 0xfff00f0d0 (0x0), SPLoc: Unknown

23856:
Stack for process 23856
   Stack for thread 23856
   walk_stack3(Dyninst::Stackwalker::Walker*) @ 0x4022f0, FP = 0xfff00ed40    
      RALoc: Register: 2, FPLoc: Register: 1, SPLoc: Register: 3
   walk_stack2(Dyninst::Stackwalker::Walker*) @ 0x402570, FP = 0xfff00ee70    
      RALoc: Addr: 0xfff00ed48 (0x402570), FPLoc: Addr: 0xfff00ed40 (0xfff00ee70), SPLoc: Unknown
   walk_stack1(Dyninst::Stackwalker::Walker*) @ 0x4025d8, FP = 0xfff00ee90    
      RALoc: Addr: 0xfff00ee78 (0x4025d8), FPLoc: Addr: 0xfff00ee70 (0xfff00ee90), SPLoc: Unknown
   main @ 0x401cd4, FP = 0xfff00eeb0    
      RALoc: Addr: 0xfff00ee98 (0x401cd4), FPLoc: Addr: 0xfff00ee90 (0xfff00eeb0), SPLoc: Unknown
   __libc_start_main @ 0x4d2f3f0, FP = 0xfff00f0d0    
      RALoc: Addr: 0xfff00eeb8 (0x4d2f3f0), FPLoc: Addr: 0xfff00eeb0 (0xfff00f0d0), SPLoc: Unknown
   _start @ 0x401e88, FP = 0x0    
      RALoc: Addr: 0xfff00f0d8 (0x401e88), FPLoc: Addr: 0xfff00f0d0 (0x0), SPLoc: Unknown

23856:
Stack for process 23856
   Stack for thread 23856
   walk_stack3(Dyninst::Stackwalker::Walker*) @ 0x4022f0, FP = 0xfff00ed40    
      RALoc: Register: 2, FPLoc: Register: 1, SPLoc: Register: 3
   walk_stack2(Dyninst::Stackwalker::Walker*) @ 0x402570, FP = 0xfff00ee70    
      RALoc: Addr: 0xfff00ed48 (0x402570), FPLoc: Addr: 0xfff00ed40 (0xfff00ee70), SPLoc: Unknown
   walk_stack1(Dyninst::Stackwalker::Walker*) @ 0x4025d8, FP = 0xfff00ee90    
      RALoc: Addr: 0xfff00ee78 (0x4025d8), FPLoc: Addr: 0xfff00ee70 (0xfff00ee90), SPLoc: Unknown
   main @ 0x401cd4, FP = 0xfff00eeb0    
      RALoc: Addr: 0xfff00ee98 (0x401cd4), FPLoc: Addr: 0xfff00ee90 (0xfff00eeb0), SPLoc: Unknown
   __libc_start_main @ 0x4d2f3f0, FP = 0xfff00f0d0    
      RALoc: Addr: 0xfff00eeb8 (0x4d2f3f0), FPLoc: Addr: 0xfff00eeb0 (0xfff00f0d0), SPLoc: Unknown
   _start @ 0x401e88, FP = 0x0    
      RALoc: Addr: 0xfff00f0d8 (0x401e88), FPLoc: Addr: 0xfff00f0d0 (0x0), SPLoc: Unknown

23856:
Stack for process 23856
   Stack for thread 23856
   walk_stack3(Dyninst::Stackwalker::Walker*) @ 0x4022f0, FP = 0xfff00ed40    
      RALoc: Register: 2, FPLoc: Register: 1, SPLoc: Register: 3
   walk_stack2(Dyninst::Stackwalker::Walker*) @ 0x402570, FP = 0xfff00ee70    
      RALoc: Addr: 0xfff00ed48 (0x402570), FPLoc: Addr: 0xfff00ed40 (0xfff00ee70), SPLoc: Unknown
   walk_stack1(Dyninst::Stackwalker::Walker*) @ 0x4025d8, FP = 0xfff00ee90    
      RALoc: Addr: 0xfff00ee78 (0x4025d8), FPLoc: Addr: 0xfff00ee70 (0xfff00ee90), SPLoc: Unknown
   main @ 0x401cd4, FP = 0xfff00eeb0    
      RALoc: Addr: 0xfff00ee98 (0x401cd4), FPLoc: Addr: 0xfff00ee90 (0xfff00eeb0), SPLoc: Unknown
   __libc_start_main @ 0x4d2f3f0, FP = 0xfff00f0d0    
      RALoc: Addr: 0xfff00eeb8 (0x4d2f3f0), FPLoc: Addr: 0xfff00eeb0 (0xfff00f0d0), SPLoc: Unknown
   _start @ 0x401e88, FP = 0x0    
      RALoc: Addr: 0xfff00f0d8 (0x401e88), FPLoc: Addr: 0xfff00f0d0 (0x0), SPLoc: Unknown

23856:
Stack for process 23856
   Stack for thread 23856
   walk_stack3(Dyninst::Stackwalker::Walker*) @ 0x4022f0, FP = 0xfff00ed40    
      RALoc: Register: 2, FPLoc: Register: 1, SPLoc: Register: 3
   walk_stack2(Dyninst::Stackwalker::Walker*) @ 0x402570, FP = 0xfff00ee70    
      RALoc: Addr: 0xfff00ed48 (0x402570), FPLoc: Addr: 0xfff00ed40 (0xfff00ee70), SPLoc: Unknown
   walk_stack1(Dyninst::Stackwalker::Walker*) @ 0x4025d8, FP = 0xfff00ee90    
      RALoc: Addr: 0xfff00ee78 (0x4025d8), FPLoc: Addr: 0xfff00ee70 (0xfff00ee90), SPLoc: Unknown
   main @ 0x401cd4, FP = 0xfff00eeb0    
      RALoc: Addr: 0xfff00ee98 (0x401cd4), FPLoc: Addr: 0xfff00ee90 (0xfff00eeb0), SPLoc: Unknown
   __libc_start_main @ 0x4d2f3f0, FP = 0xfff00f0d0    
      RALoc: Addr: 0xfff00eeb8 (0x4d2f3f0), FPLoc: Addr: 0xfff00eeb0 (0xfff00f0d0), SPLoc: Unknown
   _start @ 0x401e88, FP = 0x0    
      RALoc: Addr: 0xfff00f0d8 (0x401e88), FPLoc: Addr: 0xfff00f0d0 (0x0), SPLoc: Unknown

23856:
Stack for process 23856
   Stack for thread 23856
   walk_stack3(Dyninst::Stackwalker::Walker*) @ 0x4022f0, FP = 0xfff00ed40    
      RALoc: Register: 2, FPLoc: Register: 1, SPLoc: Register: 3
   walk_stack2(Dyninst::Stackwalker::Walker*) @ 0x402570, FP = 0xfff00ee70    
      RALoc: Addr: 0xfff00ed48 (0x402570), FPLoc: Addr: 0xfff00ed40 (0xfff00ee70), SPLoc: Unknown
   walk_stack1(Dyninst::Stackwalker::Walker*) @ 0x4025d8, FP = 0xfff00ee90    
      RALoc: Addr: 0xfff00ee78 (0x4025d8), FPLoc: Addr: 0xfff00ee70 (0xfff00ee90), SPLoc: Unknown
   main @ 0x401cd4, FP = 0xfff00eeb0    
      RALoc: Addr: 0xfff00ee98 (0x401cd4), FPLoc: Addr: 0xfff00ee90 (0xfff00eeb0), SPLoc: Unknown
   __libc_start_main @ 0x4d2f3f0, FP = 0xfff00f0d0    
      RALoc: Addr: 0xfff00eeb8 (0x4d2f3f0), FPLoc: Addr: 0xfff00eeb0 (0xfff00f0d0), SPLoc: Unknown
   _start @ 0x401e88, FP = 0x0    
      RALoc: Addr: 0xfff00f0d8 (0x401e88), FPLoc: Addr: 0xfff00f0d0 (0x0), SPLoc: Unknown

23856:
Stack for process 23856
   Stack for thread 23856
   walk_stack3(Dyninst::Stackwalker::Walker*) @ 0x4022f0, FP = 0xfff00ed40    
      RALoc: Register: 2, FPLoc: Register: 1, SPLoc: Register: 3
   walk_stack2(Dyninst::Stackwalker::Walker*) @ 0x402570, FP = 0xfff00ee70    
      RALoc: Addr: 0xfff00ed48 (0x402570), FPLoc: Addr: 0xfff00ed40 (0xfff00ee70), SPLoc: Unknown
   walk_stack1(Dyninst::Stackwalker::Walker*) @ 0x4025d8, FP = 0xfff00ee90    
      RALoc: Addr: 0xfff00ee78 (0x4025d8), FPLoc: Addr: 0xfff00ee70 (0xfff00ee90), SPLoc: Unknown
   main @ 0x401cd4, FP = 0xfff00eeb0    
      RALoc: Addr: 0xfff00ee98 (0x401cd4), FPLoc: Addr: 0xfff00ee90 (0xfff00eeb0), SPLoc: Unknown
   __libc_start_main @ 0x4d2f3f0, FP = 0xfff00f0d0    
      RALoc: Addr: 0xfff00eeb8 (0x4d2f3f0), FPLoc: Addr: 0xfff00eeb0 (0xfff00f0d0), SPLoc: Unknown
   _start @ 0x401e88, FP = 0x0    
      RALoc: Addr: 0xfff00f0d8 (0x401e88), FPLoc: Addr: 0xfff00f0d0 (0x0), SPLoc: Unknown

23856:
Stack for process 23856
   Stack for thread 23856
   walk_stack3(Dyninst::Stackwalker::Walker*) @ 0x4022f0, FP = 0xfff00ed40    
      RALoc: Register: 2, FPLoc: Register: 1, SPLoc: Register: 3
   walk_stack2(Dyninst::Stackwalker::Walker*) @ 0x402570, FP = 0xfff00ee70    
      RALoc: Addr: 0xfff00ed48 (0x402570), FPLoc: Addr: 0xfff00ed40 (0xfff00ee70), SPLoc: Unknown
   walk_stack1(Dyninst::Stackwalker::Walker*) @ 0x4025d8, FP = 0xfff00ee90    
      RALoc: Addr: 0xfff00ee78 (0x4025d8), FPLoc: Addr: 0xfff00ee70 (0xfff00ee90), SPLoc: Unknown
   main @ 0x401cd4, FP = 0xfff00eeb0    
      RALoc: Addr: 0xfff00ee98 (0x401cd4), FPLoc: Addr: 0xfff00ee90 (0xfff00eeb0), SPLoc: Unknown
   __libc_start_main @ 0x4d2f3f0, FP = 0xfff00f0d0    
      RALoc: Addr: 0xfff00eeb8 (0x4d2f3f0), FPLoc: Addr: 0xfff00eeb0 (0xfff00f0d0), SPLoc: Unknown
   _start @ 0x401e88, FP = 0x0    
      RALoc: Addr: 0xfff00f0d8 (0x401e88), FPLoc: Addr: 0xfff00f0d0 (0x0), SPLoc: Unknown

23856:
Stack for process 23856
   Stack for thread 23856
   walk_stack3(Dyninst::Stackwalker::Walker*) @ 0x4022f0, FP = 0xfff00ed40    
      RALoc: Register: 2, FPLoc: Register: 1, SPLoc: Register: 3
   walk_stack2(Dyninst::Stackwalker::Walker*) @ 0x402570, FP = 0xfff00ee70    
      RALoc: Addr: 0xfff00ed48 (0x402570), FPLoc: Addr: 0xfff00ed40 (0xfff00ee70), SPLoc: Unknown
   walk_stack1(Dyninst::Stackwalker::Walker*) @ 0x4025d8, FP = 0xfff00ee90    
      RALoc: Addr: 0xfff00ee78 (0x4025d8), FPLoc: Addr: 0xfff00ee70 (0xfff00ee90), SPLoc: Unknown
   main @ 0x401cd4, FP = 0xfff00eeb0    
      RALoc: Addr: 0xfff00ee98 (0x401cd4), FPLoc: Addr: 0xfff00ee90 (0xfff00eeb0), SPLoc: Unknown
   __libc_start_main @ 0x4d2f3f0, FP = 0xfff00f0d0    
      RALoc: Addr: 0xfff00eeb8 (0x4d2f3f0), FPLoc: Addr: 0xfff00eeb0 (0xfff00f0d0), SPLoc: Unknown
   _start @ 0x401e88, FP = 0x0    
      RALoc: Addr: 0xfff00f0d8 (0x401e88), FPLoc: Addr: 0xfff00f0d0 (0x0), SPLoc: Unknown

23856:
Stack for process 23856
   Stack for thread 23856
   walk_stack3(Dyninst::Stackwalker::Walker*) @ 0x4022f0, FP = 0xfff00ed40    
      RALoc: Register: 2, FPLoc: Register: 1, SPLoc: Register: 3
   walk_stack2(Dyninst::Stackwalker::Walker*) @ 0x402570, FP = 0xfff00ee70    
      RALoc: Addr: 0xfff00ed48 (0x402570), FPLoc: Addr: 0xfff00ed40 (0xfff00ee70), SPLoc: Unknown
   walk_stack1(Dyninst::Stackwalker::Walker*) @ 0x4025d8, FP = 0xfff00ee90    
      RALoc: Addr: 0xfff00ee78 (0x4025d8), FPLoc: Addr: 0xfff00ee70 (0xfff00ee90), SPLoc: Unknown
   main @ 0x401cd4, FP = 0xfff00eeb0    
      RALoc: Addr: 0xfff00ee98 (0x401cd4), FPLoc: Addr: 0xfff00ee90 (0xfff00eeb0), SPLoc: Unknown
   __libc_start_main @ 0x4d2f3f0, FP = 0xfff00f0d0    
      RALoc: Addr: 0xfff00eeb8 (0x4d2f3f0), FPLoc: Addr: 0xfff00eeb0 (0xfff00f0d0), SPLoc: Unknown
   _start @ 0x401e88, FP = 0x0    
      RALoc: Addr: 0xfff00f0d8 (0x401e88), FPLoc: Addr: 0xfff00f0d0 (0x0), SPLoc: Unknown

23856:
Stack for process 23856
   Stack for thread 23856
   walk_stack3(Dyninst::Stackwalker::Walker*) @ 0x4022f0, FP = 0xfff00ed40    
      RALoc: Register: 2, FPLoc: Register: 1, SPLoc: Register: 3
   walk_stack2(Dyninst::Stackwalker::Walker*) @ 0x402570, FP = 0xfff00ee70    
      RALoc: Addr: 0xfff00ed48 (0x402570), FPLoc: Addr: 0xfff00ed40 (0xfff00ee70), SPLoc: Unknown
   walk_stack1(Dyninst::Stackwalker::Walker*) @ 0x4025d8, FP = 0xfff00ee90    
      RALoc: Addr: 0xfff00ee78 (0x4025d8), FPLoc: Addr: 0xfff00ee70 (0xfff00ee90), SPLoc: Unknown
   main @ 0x401cd4, FP = 0xfff00eeb0    
      RALoc: Addr: 0xfff00ee98 (0x401cd4), FPLoc: Addr: 0xfff00ee90 (0xfff00eeb0), SPLoc: Unknown
   __libc_start_main @ 0x4d2f3f0, FP = 0xfff00f0d0    
      RALoc: Addr: 0xfff00eeb8 (0x4d2f3f0), FPLoc: Addr: 0xfff00eeb0 (0xfff00f0d0), SPLoc: Unknown
   _start @ 0x401e88, FP = 0x0    
      RALoc: Addr: 0xfff00f0d8 (0x401e88), FPLoc: Addr: 0xfff00f0d0 (0x0), SPLoc: Unknown

23856:
Stack for process 23856
   Stack for thread 23856
   walk_stack3(Dyninst::Stackwalker::Walker*) @ 0x4022f0, FP = 0xfff00ed40    
      RALoc: Register: 2, FPLoc: Register: 1, SPLoc: Register: 3
   walk_stack2(Dyninst::Stackwalker::Walker*) @ 0x402570, FP = 0xfff00ee70    
      RALoc: Addr: 0xfff00ed48 (0x402570), FPLoc: Addr: 0xfff00ed40 (0xfff00ee70), SPLoc: Unknown
   walk_stack1(Dyninst::Stackwalker::Walker*) @ 0x4025d8, FP = 0xfff00ee90    
      RALoc: Addr: 0xfff00ee78 (0x4025d8), FPLoc: Addr: 0xfff00ee70 (0xfff00ee90), SPLoc: Unknown
   main @ 0x401cd4, FP = 0xfff00eeb0    
      RALoc: Addr: 0xfff00ee98 (0x401cd4), FPLoc: Addr: 0xfff00ee90 (0xfff00eeb0), SPLoc: Unknown
   __libc_start_main @ 0x4d2f3f0, FP = 0xfff00f0d0    
      RALoc: Addr: 0xfff00eeb8 (0x4d2f3f0), FPLoc: Addr: 0xfff00eeb0 (0xfff00f0d0), SPLoc: Unknown
   _start @ 0x401e88, FP = 0x0    
      RALoc: Addr: 0xfff00f0d8 (0x401e88), FPLoc: Addr: 0xfff00f0d0 (0x0), SPLoc: Unknown
q
END
==23856== 
==23856== HEAP SUMMARY:
==23856==     in use at exit: 103,477,739 bytes in 1,146,849 blocks
==23856==   total heap usage: 4,584,099 allocs, 3,437,251 frees, 526,198,003 bytes allocated
==23856== 
==23856== LEAK SUMMARY:
==23856==    definitely lost: 1,119,216 bytes in 4,806 blocks
==23856==    indirectly lost: 5,702,206 bytes in 139,459 blocks
==23856==      possibly lost: 23,368,517 bytes in 350,229 blocks
==23856==    still reachable: 73,287,800 bytes in 652,355 blocks
==23856==                       of which reachable via heuristic:
==23856==                         newarray           : 632 bytes in 1 blocks
==23856==         suppressed: 0 bytes in 0 blocks
==23856== Rerun with --leak-check=full to see details of leaked memory
==23856== 
==23856== For counts of detected and suppressed errors, rerun with: -v
==23856== Use --track-origins=yes to see where uninitialised values come from
==23856== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 0 from 0)

ikitayama · 2016-12-24T07:27:37Z

Even PR #308 applied, running just ./test_basic -self ends up wth this message:

*** Error in `./test_basic': free(): invalid pointer: 0x000003ffe3e47988 ***

ikitayama · 2016-12-25T23:01:24Z

The uses of dwarf_dealloc() in DwarfWalker::buildSrcFiles() just look ok to me.
I'm wondering as well if it might be a bug in the libdwarf.

ikitayama · 2016-12-26T00:32:14Z

PR #309 fixes this issue.

wrwilliams closed this as completed Dec 22, 2016

wrwilliams added the bug label Mar 2, 2017

wrwilliams mentioned this issue Mar 2, 2017

Fix test_basic test error #309

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stackwalk issue on arm64 #303

Stackwalk issue on arm64 #303

ikitayama commented Dec 21, 2016

wrwilliams commented Dec 21, 2016

ikitayama commented Dec 21, 2016

wrwilliams commented Dec 22, 2016

wrwilliams commented Dec 22, 2016 •

edited

wrwilliams commented Dec 22, 2016

ikitayama commented Dec 23, 2016 •

edited

wrwilliams commented Dec 23, 2016

mxz297 commented Dec 23, 2016

ikitayama commented Dec 23, 2016

ikitayama commented Dec 23, 2016

ikitayama commented Dec 24, 2016

ikitayama commented Dec 24, 2016

ikitayama commented Dec 25, 2016

ikitayama commented Dec 26, 2016

Stackwalk issue on arm64 #303

Stackwalk issue on arm64 #303

Comments

ikitayama commented Dec 21, 2016

wrwilliams commented Dec 21, 2016

ikitayama commented Dec 21, 2016

wrwilliams commented Dec 22, 2016

wrwilliams commented Dec 22, 2016 • edited

wrwilliams commented Dec 22, 2016

ikitayama commented Dec 23, 2016 • edited

wrwilliams commented Dec 23, 2016

mxz297 commented Dec 23, 2016

ikitayama commented Dec 23, 2016

ikitayama commented Dec 23, 2016

ikitayama commented Dec 24, 2016

ikitayama commented Dec 24, 2016

ikitayama commented Dec 25, 2016

ikitayama commented Dec 26, 2016

wrwilliams commented Dec 22, 2016 •

edited

ikitayama commented Dec 23, 2016 •

edited