Debugging Reading NT binaries

Reading NT binaries

This page owns: answering a question by reading Microsoft's code instead of guessing — and the line between reading and copying.

A great deal of what this project knows was not deduced. It was read, out of binaries that were sitting on the installation CD the whole time.

This turns out to be the highest-yield debugging technique available here, for a simple reason: the interface between NT and a HAL is barely documented, but it is fully determined, and the determination is in the binaries.

---

1. The clean-room line, first

Before any of the technique: the leaked Windows NT source trees are not consulted and must not be. If you have seen them, you cannot contribute to files they would inform.

What is legitimate, and what this project uses:

• the DDK-documented Hal* contract,
• the export tables and symbol tables of shipped binaries,
• observed behaviour under a debugger,
• hardware documentation.

Reading a binary to learn an interface is normal engineering and is what every provenance row here records. Copying code out of one is not, and would change the project's licence situation entirely. → CONTRIBUTING.md

None of these binaries is in the repository, and none may be added. → Media you must supply

---

2. Two kinds of image, two sets of tools

This is the first thing to get right, because a PE tool will simply fail on half the interesting files.

Kind	Examples	Tools
PE — MZ stub, PE\0\0 header	NTOSKRNL.EXE, HALEAGLE.DLL, VIDEOPRT.SYS, every .SYS	pe-exports.py, pe-dis.py
Raw COFF — no MZ stub, file begins at the COFF header	VENEER.EXE, OSLOADER.EXE	coffsyms.py, coffdis.py

A PE disassembler looks for e_lfanew at offset 0x3C and gets nonsense from a raw COFF image. That is why this project has two sets rather than one.

The gift nobody expected

VENEER.EXE and OSLOADER.EXE both shipped with their COFF symbol tables intact — 1,512 and 2,135 symbols respectively, including a named symbol for every string constant.

That is the difference between reverse-engineering and reading. Function names like VrOpen, BlLoadSystemHive, IsFatFileStructure, choose_args and FindInLocalEnv are the originals. Whole pages of this wiki exist because of it.

---

3. The tools

# what does this image export?
python3 tools/pe-exports.py NTOSKRNL.EXE | grep -i bugcheck

# disassemble a PE, with imports and TOC loads annotated
TOC=182c0 python3 tools/pe-dis.py VIDEOPRT.SYS 0x12b60 0x12c40

# symbols of a raw COFF image
python3 tools/coffsyms.py VENEER.EXE VrOpen

# disassemble one, labelled from its symbol table
python3 tools/coffdis.py VENEER.EXE 0x54494 0x546c0

pe-exports.py	machine, sections, exports, import counts
pe-dis.py	PowerPC disassembly, annotating branch targets, imports and TOC-relative loads
coffsyms.py	header, sections and symbol table of a raw COFF image; importable as Coff
coffdis.py	the same disassembly, labelled from those symbols

The TOC

pe-dis.py needs the module's TOC anchor to resolve lwz rN, -disp(r2) into a name. Take it from the entry point's function descriptor — its second word:

ent, toc = struct.unpack_from('<II', data, rva_to_offset(entry_rva))

Then TOC=<hex> python3 tools/pe-dis.py … annotates every global reference with what it points at. Without it, the same output is an unreadable wall of displacements. → The NT PowerPC ABI

---

4. Resources: message tables

OSLOADER.EXE carries an RT_MESSAGETABLE resource with ids 9000–10014 — every error string the loader can print. Extracting it turns a bare number into a sentence, which matters because the loader falls back to printing raw ids when its own resource loading has failed.

0x232E (9006)  Windows NT could not start because of a computer disk hardware
               configuration problem.
0x2333 (9011)  Could not access disk partition tables

Walking the .rsrc directory to type 11 and decoding the block structure is about forty lines of Python. → Decoding a bugcheck

---

5. What reading actually settles

Three kinds of question, with real examples from this project.

Structure layouts, from the code that uses them. IoAllocateAdapterChannel writes the wait context block; IopCreateArcNames walks the ARC signature list; MmGetPhysicalAddress reads its argument from r4. Each of those pins down an offset or a convention that no header available here states.

Calling conventions, from the caller. Twice the caller answered what no declaration could: SCSIPORT.SYS passes a stack slot in r3 and reads the result out of it afterwards, proving that eight-byte returns come back through a hidden pointer.

Why a specific failure happened. Wall 49's whole diagnosis was reading: the faulting address was the first instruction of an import glue stub, and the stub's shape said immediately that r2 was wrong rather than the memory.

---

6. Identifying an unknown module

A technique worth knowing on its own, because "which driver is this code in?" comes up whenever a fault lands outside the modules you know.

Take a run of instructions at the faulting address, pick ones with no branch displacements — so they are position-independent — and search every driver image for that byte sequence:

words = [0x63a40000, 0x63c50000, 0x3be10038, 0x63e60000, 0x38600004]
pat = b''.join(struct.pack('<I', w) for w in words)
for f in glob.glob('drivers/*'):
    if (i := open(f,'rb').read().find(pat)) >= 0:
        print(f, hex(i))

Seven instructions were enough to identify MGA_MIL.SYS — a Matrox miniport NT was merely probing — uniquely among 53 drivers, and the file offset then gives the module's load base. → Debugging recipes

---

7. Where the binaries are

Everything above comes off your own media:

VENEER.EXE, SETUPLDR, HAL*.DLL	the NT CD's \PPC
NTOSKRNL.EXE, drivers	an installed system — extract with tools/fatcat.py

python3 tools/fatcat.py tmp/nt-installed.img 2 \
        '\WINNT\SYSTEM32\NTOSKRNL.EXE' /tmp/ntoskrnl.exe

---

Next

• Decoding a bugcheck
• Debugging recipes
• Provenance and credits — how what is learned this way gets recorded.