VEH Debugger Detector

An educational Rust detector for studying user-mode VEH debuggers on native x64 Windows processes. GhostDebug is the primary positive test sample.

For the complete design, implementation, testing process, and results, read the full VEH debugger detection blog post.

The detector correlates:

• conventional debugger state;
• the undocumented x64 PEB ProcessUsingVEH flag;
• decoded VEH callback addresses and their owning memory/module;
• module-load transitions observed after a baseline;
• committed private executable regions;
• active INT3 candidates found by comparing in-memory executable sections with their PE files, excluding relocation and IAT ranges;
• state transitions captured by continuous watch mode.

VEH, private executable memory, and 0xCC bytes all have legitimate uses. The tool therefore reports evidence and confidence instead of claiming perfect detection. Its strongest mode captures a baseline before the debugger attaches.

Build on Windows

Install Rust, open a terminal in the project directory, and build the release:

cargo build --release

Output:

target/release/veh-detector.exe

Run on Windows 11

Use an elevated terminal when inspecting a process at a higher integrity level:

.\veh-detector.exe --pid 1234
.\veh-detector.exe --name TestTarget.exe
.\veh-detector.exe scan --pid 1234
.\veh-detector.exe scan --pid 1234 --json scan.json

For stronger evidence, start watch mode before attaching GhostDebug:

.\veh-detector.exe watch --pid 1234 --interval-ms 1000 --json watch.json

Press Ctrl+C to stop. For repeatable lab runs, --samples N stops after N post-baseline samples.

Exit codes:

• 0: no indicators observed;
• 1: suspicious or high-confidence evidence;
• 2: fatal usage or collection error;
• 3: inconclusive or partially unsupported result.

Limitations

This is a research proof of concept, not a production endpoint sensor. Windows does not expose a supported API for enumerating registered VEH callbacks. The tool experimentally resolves the private handler list through a validated RIP-relative reference reached through RtlRemoveVectoredExceptionHandler's public wrapper, translates the result to the target's ntdll image, and resolves RtlDecodeRemotePointer dynamically from ntdll.dll to decode the callbacks. KernelBase and Kernel32 exports are compatibility fallbacks. Dynamic resolution avoids relying on an import that is not consistently available. The walker rejects broken links, invalid pages, unstable reads, and ambiguous resolver results.

Windows requires the target handle to include PROCESS_VM_WRITE when querying the process cookie used by remote pointer decoding. The detector requests that access right but never writes to the process or calls WriteProcessMemory.

The list and entry layouts are private implementation details and may change in any Windows update. A failed resolver is reported as unknown; it is not evidence that the process has no callbacks. The PEB field is also undocumented, while PE comparison may observe legitimate hotpatching or security instrumentation.

GhostDebug's DLL and pipe names are intentionally not detection signals. Both are trivial to rename, and the pipe name cannot reliably be attributed to the selected PID from this snapshot scanner.

Run it in an isolated lab and review the raw evidence behind every verdict.