Testing Strategy

Testing Strategy

Define the layered testing approach that takes the project from "compiles" to "passes the cycle-accuracy bar set by Mesen2 and ares". Tests are the primary specification — when written-down rules conflict, the test ROM behavior wins.

Layer 1 — Unit Tests (Per Crate)

Each crate has its own #[test] coverage in src/ and tests/:

• rustynes-cpu: every opcode × every addressing mode (~600 cases). Property tests via proptest for arithmetic flag correctness.
• rustynes-ppu: register read/write semantics; OAMADDR rules; sprite evaluation FSM.
• rustynes-apu: register layout; frame counter sequence per mode; mixer table values (compared against the closed-form formula within 0.1%).
• rustynes-mappers: per-mapper bank-resolution tables; MMC1 serial protocol; MMC3 IRQ counter (deterministic toggle test).

Layer 2 — Golden-Log Compare (CPU Only)

nestest.nes is the canonical 6502 golden master. The test harness forces PC to $C000, captures state (PC, A, X, Y, P, SP, CYC, PPU dot, scanline) after each instruction, and diffs against the bundled Nintendulator log. Target: 0-diff.

Layer 3 — Test ROM Corpus (Subsystem Coverage)

The full blargg + kevtris + community test ROM suite. Each ROM is run headlessly until complete, and its status code/string is asserted.

Category	ROMs	Pass target
CPU instruction	instr_test_v5/* (16)	All
CPU timing	cpu_timing_test6, instr_timing (2)	All strict-pass on full Nes
CPU branches	branch_timing_tests/* (3)	All
CPU interrupts	cpu_interrupts_v2/* (5)	All except 3 C1-axis residuals
PPU VBL/NMI	ppu_vbl_nmi/* (10)	All
PPU sprite	sprite_overflow_tests/* (5), sprite_hit_tests	All
APU	apu_test/* (8), apu_mixer/* (4)	All
DMC DMA	dmc_dma_during_read4/* (4)	All
MMC3	mmc3_test_2/* (5), mmc3_irq_tests/* (6)	All except known ADR-0002 residuals
Accuracy battery	AccuracyCoin (single ROM)	100% (139/139, 0 fail)

Layer 4 — Golden Framebuffer / Audio Comparison

For a curated corpus of freely-distributable demos, the harness runs for 600 frames deterministically, captures frames 60/180/300/600, and compares pixel-exact against a stored reference generated from Mesen2. Audio computes PSNR against a stored reference WAV.

Layer 4.5 — Commercial-ROM Oracle

A 99-title regression gate covering the commercial library.

• ROM SHA-256 + framebuffer FNV-1a 64-bit hash + cumulative CPU cycle count + audio hash + sample count, all asserted against a committed insta snapshot.
• Discoverable coverage for .nes, .unf, .fds, and .zip archives.

Layer 5 — Fuzz Testing

cargo-fuzz harnesses for:

• Cartridge parser: arbitrary &[u8]. Must not panic.
• CPU step: arbitrary RAM contents + opcode sequence.
• Mapper writes: arbitrary write sequences.

Layer 6 — CI Gating

GitHub Actions workflow enforces linting, no_std compilation on embedded targets, full test suites, and regression checks across Linux, macOS, and Windows.