DSP Capture Replay and Differential Testing

DSP capture, replay and differential testing

Reviewed: 2026-11-02 · Source: 8f51d237 · Availability: PR #384, HOLD, ready for review. These tools are not merged into main or included in a released download.

Issue #383 tracks a standalone MCPX GP/EP capture/replay kit. The current branch can run production DSP routines in a headless worker, compare checkpoints, PCM and ordered transfers, and reduce certain seeded failures without booting a game for each test. Capture requires the emulator; replay and synthetic comparisons run separately.

The source manual is the detailed format/CLI reference. HLE implementation is follow-on work. A/V synchronization #382 has separate guest-clock and host-output tests.

Available abilities

The follow-on HLE readiness and public XISO proof demonstrates guest GP/DMA input/output validation, 100 isolated replays, three isolated native gain routes and intentional mismatch detection. #385 adds exact-program capture selection for that workload; xemu-perf-tests #69 contains its public source. Live game routing remains unqualified.

Concurrent DSP shadow and OpenAL evaluation documents the subsequent private-worker implementation, route coverage, partial fallback, candidate-only fault detection and isolated OpenAL gain parity. These remain experimental follow-on branches with DSP authority preserved.

Tool / mechanism	Current scope
Native checkpoints	Separate JIT and interpreter state; export avoids peripheral reads with side effects; restore invalidates instruction caches
Opt-in recorder	APU-owned boundaries, preallocated bounded ring, separate writer, exclusive private file, drop/loss accounting
XCR1 v3 stream	Checksummed metadata and frame records; xemu commit/version, timestamp and fallback DSP archive identity
dsp_capture_stream.py	Inspect bounded structure, hashes, footer, frame flags and mapping observations
dsp_capture_reconstruct.py	Build private RAM/transcript replay inputs, known-byte bitmap and per-frame expected checkpoints
dsp-replay.py + dsp-replay-worker	Timed production-core, GP/EP frame/sequence, mapped-trace and synthetic engine comparison
dsp-boundary-fuzz.py	Seeded mapped scratch/FIFO scenarios, failure preservation, reduction and replay
dsp-engine-fuzz.py	JIT/interpreter comparisons for scratch, FIFO and EP-sink programs; reduction and replay
dsp_corpus.py	Preliminary metadata/payload hash admission; INTEGRITY_PASS does not perform replay or seal a corpus
dsp_upstream_goldens.py	Separate pinned upstream step/block silicon-golden checks; shipping DSP backend is not upgraded

Build and smoke tests

Use a separate checkout so the experimental branch does not replace a normal installation. Install the repository's usual Linux build dependencies first.

git clone --branch issue383-dsp-capture-replay https://github.com/Mainkill1/xemu.git xemu-dsp-tests
cd xemu-dsp-tests
git rev-parse HEAD
./configure --target-list=i386-softmmu --disable-werror --extra-cflags=-DXBOX=1
./build/pyvenv/bin/meson configure build -Dxemu_dsp_capture=true
ninja -C build qemu-system-i386 dsp-replay-worker
./build/pyvenv/bin/meson test -C build '*dsp*' --print-errorlogs

Record the actual commit and executable SHA-256. This page describes the pinned revision above; later branch changes need their own evidence. The recorder option defaults false. Replay helpers are developer test targets.

Compare production engines without a game boot

python3 tests/xbox/mcpx-apu/dsp-replay.py verify-engines \
  --worker build/dsp-replay-worker --fixture scratch \
  --frames 9 --value 0x800000 --signal deterministic-noise --repeat 100

Supported fixtures are scratch, fifo, and ep-sink. Signals are impulse, silence, alternating, ramp, left, right, extremes, near-zero, deterministic-noise, and swept-sine. EP-sink disables GP and accepts only impulse selection. The noise and sine patterns use deterministic integer generation.

Default reference/candidate are JIT/interpreter. Select --reference jit --candidate jit or the corresponding interpreter pair for same-engine checks. Only supported fixed programs are compared; arbitrary backend snapshots cannot be translated between engines.

Seeded campaigns and injected failures

Choose new output directories; existing files are never overwritten.

python3 tests/xbox/mcpx-apu/dsp-engine-fuzz.py run \
  --worker build/dsp-replay-worker --seed 89 --count 150 --output engine-results
python3 tests/xbox/mcpx-apu/dsp-boundary-fuzz.py run \
  --worker build/dsp-replay-worker --seed 47 --count 24 --output boundary-results

Reports include requested/attempted/completed work, per-status counts, coverage and executable/tool identities. Engine campaigns vary 24-bit values, signal patterns and 1/8/9/128/1024-subframe lengths. Boundary campaigns exercise the declared mapped scratch/FIFO model. Counts do not imply complete MCPX coverage.

To demonstrate failure detection and reduction:

python3 tests/xbox/mcpx-apu/dsp-engine-fuzz.py run \
  --worker build/dsp-replay-worker --seed 89 --count 1 \
  --inject-fault --output injected-failure
python3 tests/xbox/mcpx-apu/dsp-engine-fuzz.py replay \
  --worker build/dsp-replay-worker --case injected-failure/minimized.json

Both commands are expected to exit nonzero and report the injected mismatch. The reducer retains the failure location while trying shorter sequences, simpler values and impulse input. Replay checks recorded worker/wrapper/runner hashes. Cancellation and missing context remain nonpasses. POSIX resource limits and process timeouts are implemented; Windows execution remains unqualified.

Record a bounded private game sequence

Enable the JIT in the emulator settings, configure the intended DSP mode, and launch the capture-enabled binary with your usual local firmware, disk and game settings:

XEMU_DSP_CAPTURE_PATH=/path/to/new-private-capture.xcr \
XEMU_DSP_CAPTURE_FRAMES=8 \
XEMU_DSP_CAPTURE_SKIP_FRAMES=25000 \
./build/qemu-system-i386

The frame count is 1–1024 (default 32); skip is 0–1,000,000 subframes. The file must not exist. The writer creates owner-only output and finishes the bounded capture while emulation continues. Shut down normally before inspecting the final footer. Proprietary microcode, audio and guest data stay in private storage.

python3 tests/xbox/mcpx-apu/dsp_capture_stream.py /path/to/new-private-capture.xcr
python3 tests/xbox/mcpx-apu/dsp_capture_reconstruct.py \
  /path/to/new-private-capture.xcr --ram-size 67108864 --output private-replay-inputs

A structurally valid capture still reports INCOMPLETE and exits nonzero. Reconstruction can create inputs while retaining that status. Unknown RAM is initialized to zero, with a bitmap recording what was observed; that is an explicit model assumption. Version 1 requires an explicit legacy opt-in. Non-RAM descriptors, lost frames and missing mapping markers prevent plain-RAM reconstruction.

For replay, the documented verify-mapped-trace interface consumes the generated GP/EP initial/expected states, mixbins, prior/expected PCM, APU registers, RAM, control and events files. Supply both --gp-frame-states and --ep-frame-states to compare every boundary; the source manual lists every required flag and size. Use --repeat 100 for the repeatability check.

Interpreting results and current evidence

BASELINE_MATCH means equality within the selected emulator fixture and observation contract. INTEGRITY_PASS means hashes pass preliminary admission. Neither establishes hardware authority. MISMATCH, NONDETERMINISTIC, TIMEOUT, CRASH, INVALID, INCOMPLETE, UNSUPPORTED and INCONCLUSIVE must remain visible in evidence.

Recorded check on November 2	Observed result
Focused DSP suites after new-file formatting	11/11 in capture-enabled and default/system-DSP builds
Earlier seed-89 engine campaign	150/150 matches, 92 input tuples, ten signals, zero nonpasses
Private Deck v3 capture	8 frames, 1,477 events, no reported drops/losses
Private modeled Deck replay	100/100 matches; unknown RAM zero-filled
Deck noise/sine checks	Nine-subframe scratch noise and FIFO sweep matched
Independent pinned upstream ISA harness	Historical step and block each 562/562
Hosted campaigns / performance qualification	Not yet verified / not measured

Full historical hashes and limitations are in the pinned source manual and PR description. The private capture observed 28,528 bytes and left 67,080,336 RAM bytes unknown; title/scene provenance was not retained publicly and dummy audio was used. Source formatting was followed by fresh focused tests; earlier Deck/campaign binary hashes remain historical.

The issue remains open for public immutable corpus sealing, complete external context and provenance, reset/reload/overlay/IRQ and persistent-state mutations, 10k CI/100k nightly campaigns, Windows/full sanitizer coverage and runtime/XISO/game/audio regression qualification. APU register arrays and concurrent RAM/mappings are not captured as one coherent snapshot. No game performance or audio-device result follows from these checks.

Related work

#377 repairs legacy migration export. #378 is the earlier single-GP replay prototype. The new kit overlaps checkpoint/replay concepts and expands GP/EP sequencing, asynchronous recording and engine comparisons; reconcile interfaces during review. Audio architecture and development methodology provide context.

Leads from the DSP56300 corpus

The local dsp56300-corpus.7z contains an investigation of one EP firmware image and a different xemu branch. Its results are unverified leads, not measurements of our current HLE. The archive did not include the firmware or an independently validated audio output.

Reported result	Possible work
Six 256-sample blocks, 222 DMA transfers and separate stereo and encoded-output FIFOs	Reproduce with legal input; compare DMA order, FIFO bytes, cursor movement and output across block boundaries.
Five-channel matrix, mono and stereo downmix modes; six-channel encoder input	Test each channel separately, including LFE, then combinations; compare PCM, channel order and persistent filter state.
Dynamic program overlays and saved filter/encoder state	Test program replacement, cache invalidation, state restoration, reset and interrupts across blocks.
DMA offset 0x401f described as zero-filled output	Verify against an independent hardware or trusted reference. Our DMA currently rejects the offset; its correct hardware behavior is not established.
The report claims EP interrupt gating prevents frame progress	Does not match our shared peripheral read handler; do not treat this branch-specific claim as a confirmed defect.

Reported DSP cycle profile

The archive attributes the following inclusive simulated DSP cycles to one six-block encoder run. These are useful profiling leads only. Its tracer changes execution behavior and uses an enlarged cycle budget; the report's percentages have an unclear denominator. The figures are not host CPU usage, elapsed time, memory use or proof of HLE speedup.

Proposed processing path	Reported cycles	Next validation
Matrix downmix and filters	529,350	Compare channel-isolated output and filter state.
Six-channel transform and exponents	1,402,202	Compare intermediate values and overlap history.
Masking and bit allocation	2,208,652	Verify bit-allocation decisions and output independently.
Quantization and bitstream packing	519,432	Compare FIFO bytes and decode the resulting stream.

The report also claims alternate mono and stereo downmix paths took 12,586 and 15,162 simulated cycles per block respectively. None of these numbers has been reproduced in our test suite. Once a trusted reference is available, run matching offline input through both paths and record output differences, operation counts, host CPU time, wall time and peak memory. Measure startup and persistent-state cases separately.

Do not publish firmware, private game data or the report's disassembly. DSP56367 manual excerpts are background, not proof that every peripheral exists in the Xbox MCPX. Existing replay matches do not establish Dolby or 5.1 correctness.