GetTrampoline() Returns 0 If Hooking WINAPI

[YaseenTwati]

It works well with normal functions but when hooking something from WIN32 API it return 0 .
Lets say we hooked MessageBox the only to call the original function would be using a ScopedRemove or a Remove.

[4zv4l]

I also have that issue, any solution yet ?

[Zeex]

I have just tried to do the same.

In my case the trampoline could not be created because the offset between the trampoline's code buffer address and a memory address used in one of the leading instructions inside MessageBoxA (those that we copy to the trampoline) was more than could be stored in 32 bits, i.e. size of the memory operand in that instruction (this is the branch where subhook_make_trampoline returns -EOVERFLOW), therefore it was not possible to safely relocate the code to the trampoline (that would change the address referenced by the original code). See my screenshot below.

I think that it's most likely what happened in your case too. This one of the limitations of the trampoline approach implemented in subhook.

[Zeex]

Now there is a new flag SUBHOOK_TRAMPOLINE_ALLOC_NEARBY that will use a new experimental trampoline allocation strategy. I was able to make my MessageBox test work with this flag. Note: to create trampolines, you now need to also add SUBHOOK_TRAMPOLINE flag (they are off by default).

This approach may be pretty slow though (hence experimental), as there is a chance it will make a million calls to VirtualAlloc inside a loop.