mmiotic

An exploratory tool for MMIO timing — time any physical address, and dissect the hardware off the latency.

mmiotic maps physical MMIO through /dev/mem and times each access with RDTSC, reporting the minimum cycle count over repeated samples. That number is a signal. A read from DRAM costs tens of cycles; a read from a device register across the PCIe fabric costs thousands — and the spread, register to register, is a fingerprint of the silicon underneath. Point mmiotic at a range you know nothing about and structure falls out of the timing:

• Reverse-engineer hardware topology. Sweep a device region (--iomem + --binary) and the latencies draw the map — controllers, bridges, and register banks separate out even when nothing documents them. plot_mmio.py renders the scan as a heatmap.
• Discover mailbox / doorbell registers. Registers that hand work to a controller read slow and often carry side effects — they stand out sharply against ordinary config space, no datasheet required.
• Fingerprint the hypervisor. Emulated MMIO traps to a VM-exit — a huge, characteristic latency spike that betrays virtualization, often which one.
• Watch device activity. A busy device — DMA in flight, a queue draining — perturbs its register latency; a passive side channel, no driver required.
• Type registers by behavior. Read vs. write latency (--write-read), stdev, and the dynamic flag sort a bank into IDs, status, counters, and side-effecting doorbells.
• Find and build long-latency instructions. Some single accesses stall the core for milliseconds. -F/-G search for one that reaches a target time. Useful for creating deterministic, attacker-placed stalls: a race window pried open on demand, or a scheduling quantum spent inside one opcode.

Build

make

Quick start

# Time one PCI config register (bus 0, dev 2, func 0, offset 0)
sudo ./mmiotic -b 0 -d 2 -f 0 -r 0

# Time one raw physical address, 8-byte read, 1000 samples
sudo ./mmiotic -A 0xf0100000 -s 8 -c 1000

# Scan a physical range and show only slow addresses (>= 500 cycles)
sudo ./mmiotic -a 0xf0000000 -z 0xf0100000 -t 0x100 -C 500

# Hunt for a single access that takes >= 1 ms, then escalate access width
sudo ./mmiotic -a 0xf0000000 -z 0xff000000 -F 0.001 -T

Numeric arguments take decimal or 0x hex. mmiotic --help prints the full, authoritative option list — the tables below cover the ones you reach for.

Targeting

Four ways to name an address, from structured to raw:

Option	Names
-b/-d/-f/-r	PCI --bus/--device/--function/--register — decoded against the ECAM base
-o, --offset <n>	offset from the MMIO base (skips B/D/F decode)
-A, --address <n>	one physical address; sets the region base implicitly
-a/-z	--start-address/--end-address — a physical range; implies --scan

The ECAM base and size are auto-detected from ACPI MCFG (/sys/firmware/acpi/tables/MCFG). Override with -M, --mmio-base / -Z, --mmio-size.

Scanning

Option	Effect
-S, --scan	walk all addresses in the target region
-t, --stride <n>	step between addresses (default: 4)
-n, --limit <n>	stop after this many addresses
--binary	probe in bit-reversal order — after k probes the range has uniform coverage at ~range/k granularity (see the warning above)
-x, --limited	limited register range (0x00–0xff vs 0x000–0xfff)
-e, --skip	skip functions that read 0xffffffff at offset 0 (absent devices)
-I, --iomem	scan every non–System-RAM top-level region in /proc/iomem
-R, --ioregion <name>	scan /proc/iomem regions whose label matches <name>, any depth

--find-target/--find-longest turn scanning into search:

Option	Effect
-F, --find-target <s>	find an address whose access time reaches s seconds, then escalate width (unaligned 4b → 8/16/32/64/512b) to push it higher
-G, --find-longest	same search, but exhaust every candidate instead of bailing on the first hit
-B, --fallbacks <n>	candidates carried into the escalation phase (default: 10)

Access method

Option	Effect
-s, --size <n>	bytes per access: 1 / 2 / 4 (default) / 8 / 16 (XMM) / 32 (YMM) / 64 (ZMM) / 512 (fxrstor). Sizes > 4 are outside spec but work on tested hardware; AVX widths need AVX / AVX-512F
-c, --count <n>	samples per address (default: 1); the minimum is reported
-L, --lock	time a locked RMW: lock xadd (sizes 1/2/4/8) or lock cmpxchg16b (size 16). Writes the value back
-g, --gather	time a vectored gather; --size 16/32/64 picks XMM/YMM/ZMM (AVX-512)
-w, --write-read	capture write latency by timing a read that must observe a prior posted write — see below
-E, --enter (--enter-2, --enter-3)	drive a burst of up to 30 accesses from one enter $0, $31 — see below
-k, --continue	on an fxrstor fault, drop that sample and keep scanning

Output / execution

Option	Effect
-C, --min-cycles <n>	only print results with min_cycles >= n
-T, --time	show nanoseconds alongside cycles (detects TSC frequency)
-N, --nanosecond	with --time, print raw nanoseconds, no unit scaling
-P, --progress	live scan progress on stderr
-p, --processor <n>	pin to logical CPU n (default: 0) — reduces timer noise
-u, --unbound	don't pin to a core
--lazy-mmap	map in 2 GB windows on demand; required for regions > 2 GB
--quiet-mmap	suppress "mmap failed" noise from inaccessible regions

Result lines

b/d/f.o.l: 00:02:00.00.4  address: 00000000f0100000  min:      312  max:      480  stdev:      2.1  dynamic: 0  value: 8086abcd

Field	Meaning
b/d/f.o.l	bus/device/function, register offset, access size
offset	offset from the MMIO base — replaces b/d/f.o.l when the target came from -o/-A/-a+-z
address	full 64-bit physical address
min/max	fastest and slowest access seen, in cycles
stdev	standard deviation of the sampled cycle counts
dynamic	1 if the value changed at any point across samples
value	value read — for sizes over 8 bytes, the first 32 bytes as space-separated qwords
faults	appended only if an access faulted

-T suffixes the cycle counts with cy and inserts a scaled min/max time pair (-N prints those as raw nanoseconds):

b/d/f.o.l: 00:02:00.00.4  address: 00000000f0100000  min:      312 cy  max:      480 cy  (   104 ns/   160 ns)  stdev:      2.1  dynamic: 0  value: 8086abcd

Parsing note: the time column contains spaces and changes width with magnitude (ns/us/ms/s), and faults is conditional — so field positions shift from line to line. Match on the labels, not on column index. plot_mmio.py does this.

Setup

Runs as root (for /dev/mem and the MCFG table). Two kernel details bite:

• CONFIG_STRICT_DEVMEM blocks /dev/mem access to device ranges — the symptom is mmap ... Operation not permitted even as root. Boot with iomem=relaxed to lift it.
• nopat — without it the kernel may map the region write-back cached, and reads return stale cached values instead of live MMIO.

# /etc/default/grub
GRUB_CMDLINE_LINUX="iomem=relaxed nopat"

Then sudo update-grub and reboot.

⚠️ Scanning can hard-lock and/or reset the machine

Reading memory should be harmless. It is not. On some platforms, touching the wrong MMIO address — or touching the right one too fast — will hang the box or trigger a reset, with no warning and no log. There is no allowlist that saves you. Expect it, scan on a machine you can power-cycle, and work around dead zones iteratively: narrow the range, --skip empty functions, raise --stride, and re-run. --binary (below) exists partly for this — it spreads coverage across the whole range early, so a lock-up on probe 10,000 still leaves you a usable map.

In-the-wild

mmiotic has been used to advance a variety of internal and external research, sometimes in unexpected ways. Some examples of mmiotic applications:

• MCHAMMER: mmiotic calibrates a delayed machine check exception for precision delivery of MC# signals into protected environments.

• smiiiiiiiiiiiiiiii: mmiotic resolves the high-latency instruction for breaking the SMM rendezvous.

• The Assembly Hall of Shame: mmiotic is used extensively for the important problem of performance deoptimization.

Author

mmiotic is a research effort from Christopher Domas (@xoreaxeaxeax).