Crash When Using EasyHook With VLD in 64 bit application - Stack Alignment issue

[xiangrl]

I try to use easyhook to hook Windows API ShowWindow while VLD is presenting.My Version is
The Code like blow(to simply, I have omitted the implementation details in my hook function)。

BOOL WINAPI hookShowWindow(HWND hWnd, int nCmd)
{
return true;
}

int main(int argc, char *argv[])
{
QApplication a(argc, argv);

static HOOK_TRACE_INFO info; 
LhInstallHook(ShowWindow, hookShowWindow, NULL, &info);

QWidget w;
w.show();	
return a.exec();

}

I try to debug this problem,and find the vlaue of rsp register is 0x000000000023f328 which is not 16-byte aligned. So I modify the calling code of LhInstallHook in file HookSpecific_x64.asm like below.
I sub the value of rsp by 8 before calling and add it back after called.
It works.

; call NET intro
lea rcx, [IsExecutedPtr + 8] ; Hook handle (only a position hint)
mov rdx, qword ptr [rsp + 32 + 4 * 16 + 4 * 8] ; push return address
lea r8, qword ptr [rsp + 32 + 4 * 16 + 4 * 8] ; push address of return address
sub rsp, 8
call qword ptr [NETIntro] ; Hook->NETIntro(Hook, RetAddr, InitialRSP);
add rsp, 8

I don't sure whether other internal call have similar issue.
Please tell me if any mistake or ambiguous.

[justinstenning]

Can you test what the stack alignment is when it enters the trampoline? I think you will find it is incorrect on the way in.

I’ve checked the code in EasyHook and it is maintaining a 16-byte stack alignment within the trampoline.

Are you able to provide a disassembly of the start of ShowWindow on your system? Perhaps the instructions leading up to the call to showwindow too?

I’m not sure what could be causing this.

[xiangrl]

I think the problem is caused by this:
Rsp is 16-byte aligned before calling to ShowWindow.Then the call instruction will make rip pushed which means rsp sub 8 byte. Now the rsp is not 16-byte aligned.
I guess any function which calls other function must deal with rsp in its function prologue,to make sure rsp is 16-byte aligned after the function prologue.

I have tested the rsp=0x00000000002af7b8 when enter ShowWindow without EasyHook.
The disassembly of ShowWindow on my system is list as below for your information:

NtUserShowWindow:
00000000771F18E0 mov r10,rcx
00000000771F18E3 mov eax,1058h
00000000771F18E8 syscall
00000000771F18EA ret
00000000771F18EB nop

[justinstenning]

We could add code around the call that ensures it is 16-byte aligned, 2 or 3 instructions before the call and one after to restore RSP. There are a few examples of how to do this on stackoverflow.

[justinstenning]

This is how I was thinking we can ensure alignment of the stack: https://stackoverflow.com/a/9592712/323899

push %rbp          ;save rbp for stack pointer
mov  %rsp, %rbp    ;move old sp to rbp
and  $-0x10, %rsp  ;align stack
...                
...                ;if you want to use %rbp here, save it on the stack before
...  
mov  %rbp, %rsp    ;old stack pointer
pop  %rbp

or https://stackoverflow.com/a/9594334/323899

    push %rsp
    test $0xf, %rsp
    jz aligned
    push (%rsp)   // duplicate the top of the stack
aligned:
    // now have 16-byte alignment with the original stack pointer
    // on the top of the stack, either once or twice
         :
    pop %rsp

[RobertN]

I have also run into this issue and it would be very nice if it could be fixed. :)

I found that it is possible to reproduce the issue consistently (100%) if you use the "Application Verifier" which is included in Windows. Simply add the binary to it which is using EasyHook and then enable the "Basics" group and then run the binary. This will cause a crash because a 'movaps' instruction will be invoked on a non-16-byte-aligned address, when allocating memory in LhBarrierIntro.

(I also found another user that had problems with EasyHook + Application Verifier, so seems like it might not be something new:
https://stackoverflow.com/questions/23523483/application-verifier-causes-hooked-x64-application-with-easyhook-to-crash )

I tried to figure out what the alignment problem is caused by and I have a theory at least, but x64 assembly is not something I commonly use, so I could very well be wrong. :)

Normally, the stack pointer is aligned on 16 bytes before call is invoked. After the call instruction, the stack pointer is no longer aligned to 16 bytes because it pushes the return address on the stack (8 bytes). Usually, functions seems to compensate for this to make it aligned, but this is not done before calling the intro function in the EasyHook assembler.

@xiangrl's fix seems to solve the issue for me.

[justinstenning]

@RobertN thanks for the details. I'll be adding some assembly (similar to what I posted above) to ensure alignment around the call.

[starius]

I created a patch to ensure alignment of the stack: #234
Please take a look.

[justinstenning]

@starius thanks for doing that