support msvcr*d.dll (/MDd)

[derekbruening]

From bruen...@google.com on September 27, 2011 22:08:51

for issue #606 the decision is to not support /MDd at all until we have early injection in DR.

once we do there is still an issue: how get msvcr90d.pdb in order to
intercept _calloc_dbg_impl, etc.? xref issue #388 .

Original issue: http://code.google.com/p/drmemory/issues/detail?id=607

[derekbruening]

From bruen...@google.com on April 09, 2012 10:59:48

some users are unable to avoid the debug crt dll:

"However, my real app is a dll dynamically loaded by an exe. I don't
have control on that exe, it links against the the debug crt dll so it
can not run with DrMemory."

[derekbruening]

From bruen...@google.com on July 02, 2012 20:47:06

note that there are multiple challenges with supporting /MDd: in addition to LFH ( issue #606 ) there are operator issues ( issue #832 , issue #26 , issue #500 ). replacing malloc ( issue #794 ), if it works, and if it can also replace the operators, can solve several of the issues, but not all. we have to give up on reporting invalid frees on pre-us allocs, or support earliest injection. attach exacerbates the problem.

Status: Accepted
Owner: bruen...@google.com

[derekbruening]

From bruen...@google.com on August 06, 2012 20:29:55

raising priority. this has ties into -replace_malloc even w/o debug crt: xref issue #960 where -replace_malloc wants to use libc arena but misses the
libc layer b/c of the internal alloc. solving part A of the internal alloc-exported free would solve that, whether done via symbols, symbol-less analysis, or retroactive handling (will paste in notes on those options)

Labels: -Priority-Medium Priority-Critical

[derekbruening]

From bruen...@google.com on August 06, 2012 21:32:20

dividing the work here into two issues (ignoring problems solved previously in other ways like eliminating redzones, namely issue #26 and issue #500 ; issue #832 covers re-enabling those):

part A: internal alloc + redzone vs exported free
part B: LFH pre-us chunks

A is by far the worst as it leads to app crashes

based on tests w/ small apps (not yet tested on chrome component build), I can get MDd working in controlled environments (like chrome buildbots). still working on more general solution.

[derekbruening]

From bruen...@google.com on August 06, 2012 22:39:07

pasting in more notes:

** TODO solving issue #606 part A: internal alloc + redzone vs exported free

xref issue #960 where -replace_malloc wants to use libc arena but misses the
libc layer b/c of the internal alloc.

here's the internal-alloc point w/ release msvcrt (which has the same
mismatch, just no extra redzone so not as big of a deal):

ChildEBP RetAddr

00 00d7dc94 693a75e1 ntdll!NtRaiseHardError+0x12
01 00d7dcdc 6938fdc8 dynamorio!nt_messagebox+0x71 [d:\derek\dr\git\src\core\win32\ntdll.c @ 3479]
02 00d7f4f0 73951510 dynamorio!dr_messagebox+0x88 [d:\derek\dr\git\src\core\x86\instrument.c @ 3524]
03 00d7f524 739514e6 drmemorylib!handle_Rtl_alloc_failure+0x10 [d:\derek\drmemory\git\src\common\alloc_replace.c @ 1675]
04 00d7f6ac 6d5f09ee drmemorylib!replace_RtlAllocateHeap+0x236 [d:\derek\drmemory\git\src\common\alloc_replace.c @ 1710]
05 00d7f6c4 6d5f1e32 MSVCR100!_calloc_impl+0x49
06 00d7f6e0 6d6175bc MSVCR100!_calloc_crt+0x16
07 00d7f6f4 6d5f0733 MSVCR100!_getptd_noexit+0x2b
08 00d7f6fc 6d5f076a MSVCR100!_getptd+0x8
09 00d7f708 6d5fb305 MSVCR100!_LocaleUpdate::_LocaleUpdate+0x18
0a 00d7f724 6d5fb34c MSVCR100!_ismbcalpha+0x2b
0b 00d7f73c 6d5fd0cc MSVCR100!_ismbblead+0x13
0c 00d7f754 6d5fd16e MSVCR100!_lock+0x92
0d 00d7f780 6d5fd1f0 MSVCR100!_setargv+0x4f
0e 00d7f788 000e17cf MSVCR100!__getmainargs+0x21
0f 00d7f7a0 6d5f263d mallocMD!pre_cpp_init+0x36 [f:\dd\vctools\crt_bld\self_x86\crt\src\crtexe.c @ 298]
10 00d7f7ac 000e18a8 MSVCR100!_initterm+0x13
11 00d7f7ec 75b2339a mallocMD!__tmainCRTStartup+0xc4 [f:\dd\vctools\crt_bld\self_x86\crt\src\crtexe.c @ 473]
12 00d7f7f8 77ae9ef2 kernel32!BaseThreadInitThunk+0xe
13 00d7f838 77ae9ec5 ntdll!__RtlUserThreadStart+0x70

if we had symbols we would intercept _calloc_impl (release) (added for issue #940) and
_calloc_dbg_impl (debug).

possible solutions:

*** TODO require msvcr* symbols and have frontend download them?

xref issue #143 and issue #388 which have some discussion of auto-downloading
symbols. it's messy though: tradeoffs of perf overhead vs requiring user
to manually run things. not a road I want to go down if there are nicer
alternatives.

*** TODO locate w/o symbols?

it calls _callnewh so do many routines

probably best bet is to find callers of HeapAlloc. only a few in crt.

but then have to work backward to find start of function. ugh.

*** TODO identify at HeapAlloc callee and retroactively fix up?

ideally would either have syms or analyze ahead of time. but if had to
intercept:

see whether caller is in msvcr* and we didn't intercept it:

• for wrap: check prior levels
• for replace, mere fact of reaching replace_RtlAllocateHeap.

add the libc layer retroactively.

• for wrap, we want our redzone to be inside the dbgcrt redzone, which is
  what would happen if we intercepted the dbgcrt layer.
  but we can't do that!
  so instead we add to the outside and we add an
  extra malloc table entry pointing at the app data (inside dbgcrt redzone)
  marked as "treat as non-adjusting layer on free or query"
• to get dbgcrt redzone size:
  could look on app stack perhaps and find args but for dbg need to go back
  several callers.
• for replace, analyze stack frame, call replace_calloc, and return
  directly to caller of _calloc_impl? but for _calloc_dbg_impl there are
  layers in between:
  MSVCR90D!_heap_alloc_base
  MSVCR90D!_heap_alloc_dbg_impl <=== adds own redzone to size
  MSVCR90D!_nh_malloc_dbg_impl
  MSVCR90D!_calloc_dbg_impl
  could just hardcode it as skipping one caller (_calloc_impl) for release
  and 2 (_heap_alloc_base and _heap_alloc_dbg_impl) for debug.

*** INFO disasm, etc.: cut for brevity

** TODO solving issue #606 part B: LFH pre-us chunks

the heap iteration seems to treat LFH chunks as mallocs rather than
iterating inside them. not sure whether it's really what the Rtl interface
I'm using does or whether my interpretation of the flags is off.

note that the invalid args and app crashes we've seen (issue #606, or just
helloMDd.exe) are not pre-us: they're part of part A. the part B invalid
args are different and far less fatal.

potential solutions:

*** TODO can we look inside LFH during heap walk?

*** TODO early injection: or just earlier injection.

quoting from issue #960:
early injection: or just earlier injection.
could impl that in drinject.
except we'll hit issue #699 which is why currently using -no_early_inject!
need a custom entry point after kernel32 but before msvcrt.

*** TODO give up on distinguishing during heap walk and instead handle later

if see invalid heap args (for ptr to free or query) that are inside pre-us
arenas:

• for wrap, let it run w/o adjustment
• for replace: turn into nop

** TODO testing: even helloMDd.exe crashes

w/ symbols and issue #960/issue #607 private sym search for libc which fixes part A => runs w/ just 4 invalid heap args from pre_us which is part B

** TODO test chrome component build

** TODO evaluate and test on other than win7

[derekbruening]

From bruen...@google.com on August 09, 2012 06:48:18

** TODO solving issue #606 part B: LFH pre-us chunks => no, just dbgcrt chunks

*** TODO can we look inside LFH during heap walk? => it's not LFH: it's dbgcrt

helloMDd's 4 invalid heap args:

Dr.M Error #1: INVALID HEAP ARGUMENT to _free_dbg() 0x00611b10
Dr.M Error #2: INVALID HEAP ARGUMENT to _free_dbg() 0x00611a90
Dr.M Error #3: INVALID HEAP ARGUMENT to _free_dbg() 0x0068bf78
Dr.M Error #4: INVALID HEAP ARGUMENT to _free_dbg() 0x00611260

% grep -A 1 walking ls -1td logs/D*|head -1/g*
walking 3 heaps
walking heap 0 0x00420000
adding heap region 0x00420000-0x00520000 arena

walking heap 1 0x00610000
adding heap region 0x00610000-0x00620000 arena

walking heap 2 0x00890000
skipping private heap 0x00890000

walking memory looking for images
mmap_walk add 0x00010000: alloc base is 0x00010000

% grep 'adding heap reg' ls -1td logs/D*|head -1/g*
adding heap region 0x00420000-0x00520000 arena
adding heap region 0x00610000-0x00620000 arena
adding heap region 0x00620000-0x00720000 arena

0x00620000-0x00720000 is part of 0x00420000 Heap

heap 1 0x00611240-0x0061124c-0x00611a70 0 0x00610590,0x00620000 824 824 0
set range 0x00611240-0x00611a64 => 0x0
MALLOC 0x00611240-0x00611a64
heap 1 0x00611a70-0x00611a78-0x00611af0 0 0x00610590,0x00620000 78 78 0
set range 0x00611a70-0x00611ae8 => 0x0
MALLOC 0x00611a70-0x00611ae8
heap 1 0x00611af0-0x00611af8-0x00611b28 0 0x00610590,0x00620000 30 30 0
set range 0x00611af0-0x00611b20 => 0x0
MALLOC 0x00611af0-0x00611b20

heap 1 0x0068bf58-0x0068bf64-0x0068bfa0 0 0x00620048,0x00720000 3c 3c 0
set range 0x0068bf58-0x0068bf94 => 0x0
MALLOC 0x0068bf58-0x0068bf94

so all 4 are +0x20 == 32 inside == sizeof(_CrtMemBlockHeader)

so these aren't LFH chunks (doesn't really make sense for API-exposed
iterator to not walk inside those): these are simply dbgcrt redzones!

how know at walk time? if have symbols could look up _crtheap.

could use heuristics to see if looks like _CrtMemBlockHeader:
0:000> dt _CrtMemBlockHeader 0x002e1b10-20
+0x000 pBlockHeaderNext : 0x002e1a70
+0x004 pBlockHeaderPrev : 0x004bbf58
+0x008 szFileName : 0x688f1614 "f:\dd\vctools\crt_bld\self_x86\crt\prebuild\eh\excptptr.cpp"
+0x00c nLine : 470
+0x010 nDataSize : 0xc
+0x014 nBlockUse : 2
+0x018 lRequest : 123
+0x01c gap : [4] "???"

[derekbruening]

From bruen...@google.com on August 09, 2012 06:49:02

** TODO solving issue #606 part C: need private syms for issue #722
*** DONE showed up as alloc mismatches on googleurl_unittests
CLOSED: [2012-08-08 Wed 00:39]
- State "DONE" from "TODO" [2012-08-08 Wed 00:39]

Dr.M ERRORS FOUND:
Dr.M 0 unique, 0 total unaddressable access(es)
Dr.M 0 unique, 0 total uninitialized access(es)
Dr.M 159 unique, 210 total invalid heap argument(s)
Dr.M 0 unique, 0 total warning(s)
Dr.M 0 unique, 0 total, 0 byte(s) of leak(s)
Dr.M 0 unique, 0 total, 0 byte(s) of possible leak(s)
Dr.M ERRORS IGNORED:
Dr.M 474 still-reachable allocation(s)
Dr.M (re-run with "-show_reachable" for details)
Dr.M Details: D:\derek\drmemory\git\build_x86_dbg/logs/DrMemory-googleurl_unittests.exe.9300.000/results.txt

% grep 'allocated with' ls -1td logs/D*|head -1/r*
Error #1: INVALID HEAP ARGUMENT: allocated with operator new, freed with free
Error #2: INVALID HEAP ARGUMENT: allocated with operator new, freed with free
Error #3: INVALID HEAP ARGUMENT: allocated with operator new, freed with free
...

Error #1: INVALID HEAP ARGUMENT: allocated with operator new, freed with free

0 MSVCP100D.dll!std::basic_streambuf<char,std::char_traits >::setp +0x22e (0x6f6a800f <MSVCP100D.dll+0x1800f>)

1 MSVCP100D.dll!std::basic_streambuf<char,std::char_traits >::~basic_streambuf<char,std::char_traits >+0x43 (0x6f6a5274 <MSVCP100D.dll+0x15274>)

2 std::basic_stringbuf<char,std::char_traits,std::allocator >::~basic_stringbuf<char,std::char_traits,std::allocator > [c:\program files (x86)\microsoft visual studio 10.0\vc\include\sstream:77]

3 std::basic_stringstream<char,std::char_traits,std::allocator >::~basic_stringstream<char,std::char_traits,std::allocator > [c:\program files (x86)\microsoft visual studio 10.0\vc\include\sstream:705]

4 std::basic_stringstream<char,std::char_traits,std::allocator >::`vbase destructor'

# 5 testing::Message::GetString [d:\derek\chromium\src\testing\gtest\include\gtest\gtest-message.h:191]
# 6 testing::internal::FlagToEnvVar [d:\derek\chromium\src\testing\gtest\src\gtest-port.cc:693]
# 7 testing::internal::BoolFromGTestEnv [d:\derek\chromium\src\testing\gtest\src\gtest-port.cc:749]
# 8 testing::`dynamic initializer for 'FLAGS_gtest_also_run_disabled_tests'' [d:\derek\chromium\src\testing\gtest\src\gtest.cc:186]

9 __tmainCRTStartup [f:\dd\vctools\crt_bld\self_x86\crt\src\crtexe.c:473]

#10 mainCRTStartup [f:\dd\vctools\crt_bld\self_x86\crt\src\crtexe.c:370]
#11 ntdll.dll!_RtlUserThreadStart
Note: @0:00:01.240 in thread 12960
Note: memory was allocated here:
Note: # 0 MSVCP100D.dll!std::basic_streambuf<char,std::char_traits >::basic_streambuf<char,std::char_traits >+0x58 (0x6f6a6dc9 <MSVCP100D.dll+0x16dc9>)
Note: # 1 std::basic_stringbuf<char,std::char_traits,std::allocator >::basic_stringbuf<char,std::char_traits,std::allocator > [c:\program files (x86)\microsoft visual studio 10.0\vc\include\sstream:26]
Note: # 2 std::basic_stringstream<char,std::char_traits,std::allocator >::basic_stringstream<char,std::char_traits,std::allocator > [c:\program files (x86)\microsoft visual studio 10.0\vc\include\sstream:658]
Note: # 3 testing::internal::StringStreamToString [d:\derek\chromium\src\testing\gtest\src\gtest.cc:1732]
Note: # 4 testing::Message::GetString [d:\derek\chromium\src\testing\gtest\include\gtest\gtest-message.h:191]
Note: # 5 testing::internal::FlagToEnvVar [d:\derek\chromium\src\testing\gtest\src\gtest-port.cc:693]
Note: # 6 testing::internal::BoolFromGTestEnv [d:\derek\chromium\src\testing\gtest\src\gtest-port.cc:749]
Note: # 7 testing::dynamic initializer for 'FLAGS_gtest_also_run_disabled_tests'' [d:\derek\chromium\src\testing\gtest\src\gtest.cc:186] Note: # 8 __tmainCRTStartup [f:\dd\vctools\crt_bld\self_x86\crt\src\crtexe.c:473] Note: # 9 mainCRTStartup [f:\dd\vctools\crt_bld\self_x86\crt\src\crtexe.c:370] Note:#10KERNEL32.dll!BaseThreadInitThunk Note:#11` ntdll.dll!__RtlUserThreadStart

in event_basic_block(tag=0x77ae2c7c)
in event_basic_block(tag=0x6f70479f)
in event_basic_block(tag=0x6f6f3468)
in event_basic_block(tag=0x6f6a6dae)
in event_basic_block(tag=0x6f6a6dc1)
symcache_symbol_add: MSVCP100D.dll "__DrMemory_post_call" @ 0x16dc9
entering alloc routine 0x6f6f2ef0 operator new type=10 rec=0 adj=0
entering alloc routine 0x688a6660 _malloc_dbg type=22 rec=1 adj=0 (recursive)
malloc-pre heap=0x00000004 size=0x4 flags=0x0
entering alloc routine 0x688a66a0 _nh_malloc_dbg type=22 rec=2 adj=2 (recursive)
_nh_malloc_dbg recursive call: helper, so no adjustments
entering alloc routine 0x77ade046 RtlAllocateHeap type=30 rec=3 adj=2 (recursive)
check_recursive RtlAllocateHeap: 0x00000004 vs 0x00000008
RtlAllocateHeap recursive call: helper, so no adjustments
leaving alloc routine 0x77ade046 RtlAllocateHeap rec=4 adj=2
recursive post-alloc routine 0x77ade046 RtlAllocateHeap: no adjustments; eax=0x005bd6d8
leaving alloc routine 0x688a66a0 _nh_malloc_dbg rec=3 adj=2
recursive post-alloc routine 0x688a66a0 _nh_malloc_dbg: no adjustments; eax=0x005bd6f8
leaving alloc routine 0x688a6660 _malloc_dbg rec=2 adj=2
_malloc_dbg-post 0x005bd6f8-0x005bd6fc = 0x4 (really 0x005bd6f8-0x005bd6fc 0x4)
MALLOC 0x005bd6f8-0x005bd6fc
set range 0x005bd6f8-0x005bd6fc => 0x3
in event_basic_block(tag=0x6f6a6dc9)

0:000> U 0x6f6a6dc1
MSVCP100D!std::basic_streambuf<char,std::char_traits >::basic_streambuf<char,std::char_traits >+0x51:
6f6a6dc1 50 push eax
6f6a6dc2 6a04 push 0x4
6f6a6dc4 e827c10400 call MSVCP100D!operator new (6f6f2ef0)

0:000> U 0x688a9c70
MSVCR100D!free:
688a9c70 8bff mov edi,edi
688a9c72 55 push ebp
688a9c73 8bec mov ebp,esp
688a9c75 6a01 push 0x1
688a9c77 8b4508 mov eax,[ebp+0x8]
688a9c7a 50 push eax
688a9c7b e890d8ffff call MSVCR100D!_free_dbg (688a7510)

w/o syms:
0:000> U 0x6f6a8005
MSVCP100D!std::basic_streambuf<char,std::char_traits >::setp+0x225:
6f6a8005 8b4508 mov eax,[ebp+0x8]
6f6a8008 50 push eax
6f6a8009 ff156412696f call dword ptr [MSVCP100D+0x1264 (6f691264)]

w/ syms:
0:000> U 0x6f6a8005
MSVCP100D!std::_DebugHeapDeletestd::locale+0x15:
6f6a8005 8b4508 mov eax,[ebp+0x8]
6f6a8008 50 push eax
6f6a8009 ff156412696f call dword ptr [MSVCP100D!_imp__free (6f691264)]

=> it's issue #722

and it needs private syms:
% bin/symquery.exe -e C:/Windows/system32/MSVCP100D.dll --search -s "std::DebugHeapDelete<>"
% bin/symquery.exe -e C:/Windows/system32/MSVCP100D.dll --searchall -s "std::DebugHeapDelete<>"
std::_DebugHeapDeletestd::_Fac_node +0x62bb0
std::_DebugHeapDeletestd::locale +0x17ff0
std::_DebugHeapDeletestd::ios_base::_Iosarray +0x142e0
std::_DebugHeapDeletestd::ios_base::_Fnarray +0x14300
std::_DebugHeapDeletestd::locale::facet +0x13d20
std::_DebugHeapDelete +0x46c00
std::_DebugHeapDelete<_RTL_CRITICAL_SECTION> +0x635a0

hmm perhaps a symbol-based solution to part A should be considered after
all since part C needs symbols too

note that -replace_malloc does not solve this as it also needs to find the
symbol and in fact it can't replace it directly and must do the hacky
decode => issue #965

*** DONE std::_DebugHeapDelete looks different: indirect call to free
CLOSED: [2012-08-08 Wed 00:38]
- State "DONE" from "TODO" [2012-08-08 Wed 00:38]

even w/ syms and doing full lookup for msvcp*.dll:
WARNING: unable to suppress std::_DebugHeapDelete mismatch errors

so a maintenance issue for issue #722's decoding hack for VS2010 I guess:

0:000> U 0x66055261 L10
base!std::_DebugHeapDeletestd::locale::facet [c:\program files (x86)\microsoft visual studio 10.0\vc\include\xdebug @ 56]:
66055261 55 push ebp
66055262 8bec mov ebp,esp
66055264 56 push esi
66055265 8b7508 mov esi,[ebp+0x8]
66055268 85f6 test esi,esi
6605526a 7410 jz base!std::_DebugHeapDeletestd::locale::facet+0x1b (6605527c)
6605526c 8b06 mov ...

[derekbruening]

From bruen...@google.com on August 09, 2012 06:49:02

... eax,[esi]
6605526e 6a00 push 0x0
66055270 8bce mov ecx,esi
66055272 ff10 call dword ptr [eax]
66055274 56 push esi
66055275 ff15e8d21566 call dword ptr [base!_imp__free (6615d2e8)]
6605527b 59 pop ecx
6605527c 5e pop esi
6605527d 5d pop ebp
6605527e c3 ret

0:000> U 0x660dda60 L10
base!std::_DebugHeapDeletestd::_Fac_node [f:\dd\vctools\crt_bld\self_x86\crt\src\xdebug @ 56]:
660dda60 8bff mov edi,edi
660dda62 55 push ebp
660dda63 8bec mov ebp,esp
660dda65 837d0800 cmp dword ptr [ebp+0x8],0x0
660dda69 7417 jz base!std::_DebugHeapDeletestd::_Fac_node+0x22 (660dda82)
660dda6b 6a00 push 0x0
660dda6d 8b4d08 mov ecx,[ebp+0x8]
660dda70 e81b000000 call base!std::_Fac_node::`scalar deleting destructor' (660dda90)
660dda75 8b4508 mov eax,[ebp+0x8]
660dda78 50 push eax
660dda79 ff15e8d21566 call dword ptr [base!_imp__free (6615d2e8)]
660dda7f 83c404 add esp,0x4
660dda82 5d pop ebp
660dda83 c3 ret

=> need to handle indirect call via IAT to free

also need to look up possible_crtdbg_routines in msvcrp*.dll even though it
has no _malloc_dbg routine, or (better) move std::_DebugHeapDelete<> to the
possible_cpp_routines

[derekbruening]

From bruen...@google.com on August 09, 2012 06:49:33

** TODO part D: MSVCP100D.dll pdb does not have type info for operators

operator new[] in MSVCP100D.dll @0x66352f80 generic type=11 => drsyms res=1, 3472380 args
WARNING: unable to determine args for operator operator new[] in MSVCP100D.dll @0x66352f80

0:000> x msvcp100d!operator*
66352f80 MSVCP100D!operator new[] =
66352fd0 MSVCP100D!operator delete[] =
66308470 MSVCP100D!operator new =
66364dd0 MSVCP100D!operator delete[] =
663643f0 MSVCP100D!operator new[] =
66364fa4 MSVCP100D!operator new =
66364bfe MSVCP100D!operator delete =
66308480 MSVCP100D!operator delete =
663420d0 MSVCP100D!operator delete =
663420a0 MSVCP100D!operator new =
66352ef0 MSVCP100D!operator new =
66352fb0 MSVCP100D!operator delete =
66365136 MSVCP100D!operator new =

0:000> lm m msvcp100d
start end module name
662f0000 663a7000 MSVCP100D (pdb symbols) d:\derek\symbols\msvcp100d.i386.pdb\1263D030053C444391846A6F1ECB31281\msvcp100d.i386.pdb

this is an issue for -replace_malloc operator replacement (from issue #882) for
now but if we add operators as an adjusting layer for wrapping it will
also matter there.

need to look at mangled syms. fortunately these are exports so that should
be doable.

[derekbruening]

From bruen...@google.com on August 09, 2012 10:00:01

** TODO symbol solution discussion
for part A and part C we would like msvcrt and msvcpt symbols for simplest
solution

xref DRi#450: symsrv.dll is redistributable.
but can't use it easily inside drmemorylib.
so, if drmemorylib sees msvc*.dll and it can't find the syms it needs (or
just do pdb presence query), it gives a fatal error and aborts w/ message
to manually run a command.
that command is also run at installation.
frontend sets _NT_SYMBOL_PATH env var to point at logs/symbols or sthg?

but, we can use hacky solutions for part A, and for part C we can just
disable mismatches: so we can make the lack of symbols non-fatal which
seems good. so instead of fatal error, have warning about false negatives,
and have drmem write a file or sthg which frontend checks on next run (so
user doesn't have to run manual cmd, and frontend doesn't have to check
dlls vs pdbs on every run: wait for drmemorylib to identify problem).
also works for both interactive users and bots.
so it's seeming a frontend option to get syms isn't needed, but could have
it for manual sym download (could run at install time)

once we have the symsrv in place we can use it to get system lib syms which
will help quality of callstacks in general, though IMHO it's still best to
have default suppressions work w/o symbols (just like for above: better w/
syms as in fewer false neg or clearer error reports, but syms not required
to avoid false pos)

OTOH, implementing the hack solns requires extra code at runtime: is that
really better than extra time in frontend? though we already have retaddr
and libc bounds are cached, so runtime cost should be much smaller than
frontend opening up files.

[derekbruening]

From bruen...@google.com on August 13, 2012 21:20:17

splitting automatically retrieving symbols into issue #143

[derekbruening]

From bruen...@google.com on August 15, 2012 08:06:02

for part A nosyms wrap we hit an unaddr from internal alloc routine touching headers:

heap 1 0x003fbec8-0x003fbed1-0x003fbf10 0 0x00390048,0x00490000 3f 3f 0
skipping dbgcrt header => 0x003fbee8-0x003fbf03
set range 0x003fbee8-0x003fbf03 => 0x0
memset 0x003fbee8-0x003fbf00
malloc_add_common: type=0
MALLOC 0x003fbee8-0x003fbf03

0 (0x00000000) modid:0

heap 1 0x003fbf10-0x003fbf1c-0x003fbf58 0 0x00390048,0x00490000 3c 3c 0
skipping dbgcrt header => 0x003fbf30-0x003fbf48
set range 0x003fbf30-0x003fbf48 => 0x0
memset 0x003fbf30-0x003fbf48
malloc_add_common: type=0
MALLOC 0x003fbf30-0x003fbf48

0 (0x00000000) modid:0

heap 1 0x003fbf58-0x003fbf64-0x003fbfa0 0 0x00390048,0x00490000 3c 3c 0
skipping dbgcrt header => 0x003fbf78-0x003fbf90
set range 0x003fbf78-0x003fbf90 => 0x0
memset 0x003fbf78-0x003fbf90
malloc_add_common: type=0
MALLOC 0x003fbf78-0x003fbf90

0 (0x00000000) modid:0

heap 0 0x003fbfa0-0x003fbfa8-0x003fbfe8 0 0x00390048,0x00490000 40 0 0
heap 0 0x003fbfe8-0x003fbff0-0x003fc030 0 0x00390048,0x00490000 40 0 0

Error #1: UNADDRESSABLE ACCESS: writing 0x003fbf5c-0x003fbf60 4 byte(s)

0 MSVCR100D.dll!malloc_dbg (0x68726a5b <MSVCR100D.dll+0x116a5b>) modid:3

1 MSVCR100D.dll!malloc_dbg (0x6872671f <MSVCR100D.dll+0x11671f>) modid:3

2 MSVCR100D.dll!malloc_dbg (0x68726ba4 <MSVCR100D.dll+0x116ba4>) modid:3

3 MSVCR100D.dll!initptd (0x6865a801 <MSVCR100D.dll+0x4a801>) modid:3

4 MSVCR100D.dll!getptd (0x6865a88b <MSVCR100D.dll+0x4a88b>) modid:3

5 MSVCR100D.dll!isalpha_l (0x686c808f <MSVCR100D.dll+0xb808f>) modid:3

6 MSVCR100D.dll!ismbbkana (0x686cfe73 <MSVCR100D.dll+0xbfe73>) modid:3

7 MSVCR100D.dll!ismbblead (0x686cfd54 <MSVCR100D.dll+0xbfd54>) modid:3

8 MSVCR100D.dll!unlock (0x686598f9 <MSVCR100D.dll+0x498f9>) modid:3

9 MSVCR100D.dll!unlock (0x686597b7 <MSVCR100D.dll+0x497b7>) modid:3

#10 MSVCR100D.dll!_getmainargs (0x68658d96 <MSVCR100D.dll+0x48d96>) modid:3
#11 pre_cpp_init [f:\dd\vctools\crt_bld\self_x86\crt\src\crtexe.c:298](0x00f610fd <helloMDd.exe+0x10fd) modid:1
Note: @0:00:00.716 in thread 10100
Note: next higher malloc: 0x003fbf78-0x003fbf90
Note: prev lower malloc: 0x003fbf30-0x003fbf48
Note: instruction: mov %ecx -> 0x04(%eax)

I'm just going to suppress it

[derekbruening]

From bruen...@google.com on August 15, 2012 08:24:47

so in the end we see that replacing doesn't make correctly handling /MDd
easier. in fact, replacing is harder to get to work with /MDd b/c part A
nosyms is harder (though I guess more complete b/c we get our redzone) and
it needs part D while wrapping does not. although that last bit is not fair: if we
implemented operators as a fully adjusting layer for issue #882 we would have
the same problem. replacing does help w/ getting our redzones there but that's
issue #832 and isn't necessarily a correctness issue.

[derekbruening]

From bruen...@google.com on April 25, 2013 19:07:22

I just discovered that the calls to _calloc{,_dbg}_impl that kept triggering all of these /MD and /MDd problems only show up due to transparency bugs that were fixed in DR r1728 (DRi#985), pulled into DrMem in r1115 . Normally the FLS slot for _getptd_noexit() is set up during MSVCR*.dll initialization. Although we'd see the same problems anyway if we did earlier injection.

[derekbruening]

From derek.br...@gmail.com on April 30, 2013 20:33:44

This issue was closed by revision r1310 .

Status: Fixed