drmemory crashes on calc on recently-updated win8.1 due to kernel checking TEB stack fields

[derekbruening]

% bin32/drrun -t drmemory_light -- calc
~~Dr.M~~ Dr. Memory version 1.8.0
~~Dr.M~~ Running "calc"
~~Dr.M~~ WARNING: application exited with abnormal code 0xc0000409

I tried 5.0.0 -- crashes too.
In my notes, I tested win8.1 64 and 32 kernels: certainly it used to work,
so this must be due to a recent Windows Update?
Indeed, Qin's win8.1 x64 VM works fine, and his ntdll doesn't have the new
call in DR i#1599.

define STATUS_STACK_BUFFER_OVERRUN ((NTSTATUS)0xC0000409L) // winnt

(8dc.3f8): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=00000000 ebx=0003100c ecx=21eb1258 edx=00c30000 esi=21e2ef58 edi=00000001
eip=739d0001 esp=21e2e314 ebp=21e2e434 iopl=0 nv up ei pl zr na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010246
739d0001 0fb642fb movzx eax,byte ptr [edx-5] ds:002b:00c2fffb=??
0:000> g
(8dc.3f8): Security check failure or stack buffer overrun - code c0000409 (!!! second chance !!!)
eax=00000000 ebx=0003100c ecx=21eb1258 edx=00c30000 esi=21e2ef58 edi=00000001
eip=739d0001 esp=21e2e314 ebp=21e2e434 iopl=0 nv up ei pl zr na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010246
739d0001 0fb642fb movzx eax,byte ptr [edx-5] ds:002b:00c2fffb=??

Or if I have windbg ignore access violation:

(9c8.3ec): Access violation - code c0000005 (first chance)
(9c8.3ec): Security check failure or stack buffer overrun - code c0000409 (!!! second chance !!!)
eax=00000000 ebx=0003100c ecx=245f1258 edx=00c30000 esi=2456ef58 edi=00000001
eip=739d0001 esp=2456e314 ebp=2456e434 iopl=0 nv up ei pl zr na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010246
739d0001 0fb642fb movzx eax,byte ptr [edx-5] ds:002b:00c2fffb=??

0:000> kn
 # ChildEBP RetAddr  
00 2456e434 739d36b1 drmemorylib!is_retaddr+0xb1 [d:\drmemory_package\common\callstack.c @ 1087]
01 2456e670 739cdd88 drmemorylib!find_next_fp+0x1691 [d:\drmemory_package\common\callstack.c @ 1390]
02 2456ea2c 739d694a drmemorylib!print_callstack+0x2f18 [d:\drmemory_package\common\callstack.c @ 1703]
03 2456eb00 73a5cff7 drmemorylib!packed_callstack_record+0xcaa [d:\drmemory_package\common\callstack.c @ 1921]
04 2456eb44 73a5bc2e drmemorylib!gdicheck_dc_alloc+0x227 [d:\drmemory_package\drmemory\gdicheck.c @ 213]
05 2456ebb8 73a5b42b drmemorylib!syscall_check_gdi+0x49e [d:\drmemory_package\drmemory\syscall_wingdi.c @ 206]
06 2456ebdc 73a59bc4 drmemorylib!wingdi_shared_process_syscall+0x7b [d:\drmemory_package\drmemory\syscall_wingdi.c @ 259]
07 2456ec44 739fbbde drmemorylib!os_shared_post_syscall+0x5f4 [d:\drmemory_package\drmemory\syscall_windows.c @ 573]
08 2456edcc 73a87ea8 drmemorylib!event_post_syscall+0x154e [d:\drmemory_package\drmemory\syscall.c @ 544]
09 2456ef34 73f062d7 drmemorylib!drmgr_postsyscall_event+0x88 [d:\drmemory_package\dynamorio\ext\drmgr\drmgr.c @ 1232]
0a 2456ef68 73f4b615 dynamorio!instrument_post_syscall+0x97 [d:\dynamorio_package\core\lib\instrument.c @ 1918]
0b 2456ef98 73ee2db5 dynamorio!post_system_call+0x675 [d:\dynamorio_package\core\win32\syscall.c @ 4045]
0c 2456efa0 73ee36a2 dynamorio!handle_post_system_call+0x25 [d:\dynamorio_package\core\dispatch.c @ 1987]
0d 2456efbc 73ee37a2 dynamorio!dispatch_enter_dynamorio+0x3c2 [d:\dynamorio_package\core\dispatch.c @ 798]
0e 2456eff4 24581be2 dynamorio!dispatch+0x12 [d:\dynamorio_package\core\dispatch.c @ 154]

0:000> !vprot esp
BaseAddress:       2456e000
AllocationBase:    244e0000
AllocationProtect: 00000001  PAGE_NOACCESS
RegionSize:        00001000
State:             00001000  MEM_COMMIT
Protect:           00000004  PAGE_READWRITE
Type:              00020000  MEM_PRIVATE
0:000> ? esp
Evaluate expression: 609674004 = 2456e314
0:000> dds esp
2456e314  00000005
2456e318  73f025ce dynamorio!our_vsnprintf+0x40e [d:\dynamorio_package\core\iox.h @ 508]
2456e31c  00000001
2456e320  73f07200 dynamorio!dr_unload_aux_x64_library [d:\dynamorio_package\core\lib\instrument.c @ 3163]
2456e324  00000001
2456e328  2456e53c

My theory: I think that there is a new Windows kernel security feature
where on any fault if the stack pointer doesn't match what the app's stack
should be, it throws an uncatchable 0xc0000409. So on the first
is_retaddr fault, kernel kills us.

To solve we'd have to write into TEB stack fields on cxt sw I guess and
hope kernel reads those and hasn't cached sthg. Or have only dr_try and
safe_read do that?

[derekbruening]

Indeed, if on 1st chance AV in windbg I write to the 2 TEB fields, we
continue w/o the 0xc0000409.
It also works if I write to just StackBase such that esp is inside there:
esp=1f17eba8
StackBase: 00950000
StackLimit: 00948000
=>
StackBase: 1f17f000
StackLimit: 00948000

Unfortunately the cxt sw is not enough for DrMem:

 # ChildEBP RetAddr  
00 21deeed8 73839b3a drmemorylib!find_next_fp+0x301 [d:\drmemory_package\common\callstack.c @ 1390]
01 21deef2c 73839d31 drmemorylib!print_callstack+0x4ba [d:\drmemory_package\common\callstack.c @ 1740]
02 21deef64 7383ae32 drmemorylib!packed_callstack_record+0xf1 [d:\drmemory_package\common\callstack.c @ 1921]
03 21deef94 73831899 drmemorylib!client_malloc_data_to_free_list+0x72 [d:\drmemory_package\drmemory\alloc_drmem.c @ 963]
04 21deefd8 72e2c2cc drmemorylib!replace_free_common+0x209 [d:\drmemory_package\common\alloc_replace.c @ 1880]
05 21deeff8 00000000 dynamorio!dr_call_on_clean_stack+0x51 [D:\derek\dr\git\build_x86_dbg\core\CMakeFiles\dynamorio.dir\arch\x86\x86.asm.obj.s @ 1432]

dr_call_on_clean_stack() is static so we have to pay the cost on all
Windows versions, unless we want dr_call_on_clean_stack_win81 or sthg and
dispatch.

Is the cost measurable:
06:56 PM ~/extsw/heaplayers-351/benchmarks/cfrac
% for ((i=0; i<3; i++)); do /usr/bin/time ~/drmemory/git/build_x86_rel/bin/drmemory.exe -light -dr d:/derek/dr/git/exports -batch -- ./cfrac.exe 41757646344123832613190542166099121; done
dr_call_on_clean_stack w/o swaps:
0.00user 0.01system 0:30.12elapsed 0%CPU (0avgtext+0avgdata 262144maxresident)k
0.01user 0.00system 0:28.43elapsed 0%CPU (0avgtext+0avgdata 262400maxresident)k
0.00user 0.00system 0:30.38elapsed 0%CPU (0avgtext+0avgdata 262400maxresident)k
dr_call_on_clean_stack w/ swaps:
0.01user 0.00system 0:31.93elapsed 0%CPU (0avgtext+0avgdata 262144maxresident)k
0.00user 0.01system 0:27.61elapsed 0%CPU (0avgtext+0avgdata 262400maxresident)k
0.01user 0.00system 0:27.91elapsed 0%CPU (0avgtext+0avgdata 262144maxresident)k
So maybe not a big deal.

[derekbruening]

Xref DR i#1102

An alternative for dr_call_on_clean_stack() is to change DR_STATE_TO_SWAP in DrMem to include
DR_STATE_STACK_BOUNDS. The call to dr_switch_to_dr_state_ex() is prior to
the stack swap though, and it seems best to ensure we can handle faults anywhere.