2.9 Hook: Runtime Callsite Logging

Usage

Sometimes during plugin development, you may want to track down callers to specific function during runtime, and especially when it's a virtual function/member function, that makes it even harder to find.

Here's a common example of intercepting a virtual function at runtime and log its callers(more specifically, its return address):

Target function:

void RE::TESObjectREFR::SetStartingPosition(const RE::NiPoint3& a_pos); // 0x54 in vtbl
// this function is a virtual member function of class RE::TESObjectREFR, 
// first we need to translate it to normal function:
void SetStartingPosition(RE::TESObjectREFR* a_this, const RE::NiPoint3& a_pos);
// per x64 calling convention, *this pointer is implicitly passed as first argument

Target function assembly starts with:

40:56                    | push rsi
57                       | push rdi
41:56                    | push r14

We only need 5 bytes minimum to setup a cave hook, this three lines are sufficient.

Hook

using namespace DKUtil::Alias;

// assume we acquired an instance of the class RE::TESObjectREFR
RE::TESObjectREFR* refr_instance = 0x123456;

class SetStartingPositionEx : 
    public Xbyak::CodeGenerator
{
    // because vptr is at 0x0 of class instance, so a pointer to class is a pointer to vptr
    inline static std::uintptr_t* vtbl{ *std::bit_cast<std::uintptr_t**>(refr_instance) };
    // the index of the target virtual function we want to log in vtbl
    inline static std::size_t index{ 0x54 };

    // actual logging function, we have added third argument, it's the return address/callsite
    static void Intercept_SSP(RE::TESObjectREFR* a_this, const RE::NiPoint3& a_pos, std::uintptr_t a_caller = 0)
    {
        INFO("caller 0x{:X}, [{:X}] {}", dku::Hook::GetRawAddress(a_caller), a_this->formID, a_this->GetName());
    }

public:
    SetStartingPositionEx()
    {
        // we move the return address on stack[rsp] to third argument, as per x64 calling convention
        mov(r8, qword[rsp]);
    }

    static void Install() 
    {
        SetStartingPositionEx sse{};
        sse.ready();

        // generates prolog & epilog patches that preserve register values, 
        // so our logging function won't break anything
        auto [ppatch, epatch] = dku::Hook::JIT::MakeNonVolatilePatch(
            {
                dku::Hook::Register::ALL
            });

        // actual prolog patch
        Patch prolog;
        // first we move the third argument
        prolog.Append(sse);
        // then apply the preserve patch
        prolog.Append(ppatch);
        // nothing extra to add for epilog patch, because we didn't break stack balance

        // target function is at index 0x54 in vtbl
        auto addr = vtbl[index];
        auto hook = dku::Hook::AddCaveHook(
            addr,
            { 0, 5 },
            FUNC_INFO(Intercept_SSP), 
            &prolog, 
            &epatch, 
            // important that we restore original function prolog(the 5 bytes we took) after returning from logger
            HookFlag::kRestoreAfterEpilog);
        hook->Enable();
    }
};