Release Qualification and Testing

Release Qualification and Testing

A recorder that works for ten minutes and fails at minute forty is worse than useless. This page describes how Flashback is tested before a release: the automated checks every build goes through, the hardware gate a release is meant to pass on a real gaming PC, and what that gate does and doesn't prove.

What every build is checked for

• Automated tests cover settings, naming, game recognition rules, storage-limit decisions, update verification, A/V sync logic at segment joins, the installer contents and much more, without needing a display or a GPU.
• The FFmpeg build is checked against the list of components Flashback relies on, so an image change can't quietly add or drop a feature. See FFmpeg Build.
• Packaging checks confirm signatures, the full-package-only update feed, the checksum file, and that every native file ships with the exact Microsoft Visual C++ runtime it needs beside it. The hardware gates run the recorder on that same shipped runtime, not on whatever copy Windows has, and fail if Windows loads any other.
• Install and update smoke tests install the new setup on a clean Windows user or in Windows Sandbox, update from the previous public release, and check that settings, the startup entry and the Start menu come through intact and nothing is left behind on uninstall.

The hardware gate

Some things can only be proven on real hardware with a real game. The release gate is a long capture session on a gaming PC:

What it requires	Why
A Flashback-off control first	Every measurement is compared with the same PC and game with no recorder running.
At least 60 minutes of capture	Long enough to catch slow leaks and drift that short tests miss.
At least 20 real segment rollovers at the production interval, while a game supplies constant motion	Rollovers are where recorders usually break. These are the same rollovers the installed app does, not sped-up ones.
One capture worker for the whole run	Any worker replacement, restart, recovery or quarantine fails the gate. Recovery exists for users, but the gate demands it was never needed.
Every segment and the saved replay fully decoded	Frame cadence, continuous audio across every boundary, and sustained motion are checked in the decoded media, not just in file metadata.
System and game measurements against the control	CPU, GPU engines, DPC and ISR load, memory, and the game's own frame-time percentiles and outliers (with PresentMon, input tracking off).

The gate doesn't launch or control the game, doesn't change display or audio settings, and never opens Flashback's interface during the run.

The audio-sync gate

A separate controlled check plays visible flashes together with tone pulses on known, isolated audio routes, records them through Flashback in both the single mixed-track mode and the editable-tracks mode, then decodes every saved audio stream and measures where each pulse lands against its flash. The transport delay of the test setup itself is measured and removed, so only the offset Flashback adds is judged. See Audio and Video Sync.

The audio fidelity gate (0.8.0)

Sync isn't the only thing that matters: a clip can be perfectly in time and still crackle. From 0.8.0 the release gate also runs an audio fidelity check:

• Recorded device timing, replayed. Real packet timing captured from audio devices, jitter included, is played back through Flashback's capture timeline without any hardware, to check that clean audio comes out continuous.
• A real run on a virtual audio cable. Test audio plays into a virtual cable while Flashback records it through its real capture, segment rotation and join path, and an independent capture records the same sound as a control. Flashback's Microphone track and its joined export are compared with the control sample for sample. Any inserted or dropped sample, click, inserted silence, clipping, or difference above -80 dBFS fails the gate.
• The mixed track too. A second run records the single mixed track with jittered segment rotations and measures every join, which may show only the recorder's own short start-up silence, faded so it doesn't click.

The check refuses to run if any other app is recording the same cable, so no other program receives its test audio. A fidelity failure stops a release; a PC that simply lacks the cable gets a warning instead.

The worker recovery checks

Recovery is tested separately from normal rollover, by deliberately breaking things:

1. The capture worker is killed before a rollover is accepted, after it's accepted, and while the previous file is being finished.
2. Start, rollover, snapshot and stop are each stalled on purpose, to confirm the deadline stops only the worker.
3. The app, tray, shortcuts, settings, library and already-confirmed segments must all survive.
4. A saved replay must never combine footage from both sides of an unplanned break.
5. Restarts must back off, and stopping the buffer or preparing an update must never start another worker.
6. Normal and forced shutdowns must leave no worker process and no locked file.

Memory growth is tested by deliberately growing memory in the app (which must not trigger recorder recovery) and then in the worker (which must). See Resource Guards and Recovery.

The test matrix

The test plan calls for:

• NVIDIA, AMD and Intel graphics
• 1080p60, 1440p60 and 4K60, plus 240 FPS at 720p and 1080p
• Borderless and exclusive fullscreen games, Java and Bedrock Minecraft, shader-heavy scenes
• Single and mixed-refresh multi-monitor setups, SDR and HDR desktops
• Changing the default audio device while recording
• Display sleep and wake, and a GPU driver reset where practical
• Shutting down or preparing an update while the worker is recording or recovering
• An uncapped game saturating the encoder

What this does and doesn't prove

• The automated release gate runs on one NVIDIA PC. Published measurements are NVIDIA-only. AMD and Intel coverage comes from the wider test plan and from tester reports, such as the AMD encoder fix in 0.8.0.
• A gate pass describes that PC and that game. It's strong evidence the recorder is stable and in sync, not a guarantee for every GPU, driver or game.
• The gate is required, but a release can be published without it. A few releases (0.7.36 to 0.7.39) were published without the full hardware run, by an explicit decision noted in their release notes; signing, package smoke tests, checksums and update-feed validation still applied. Flashback says so rather than implying every release passed.

Related pages

Benchmarks · Resource Guards and Recovery · Audio and Video Sync · Why Flashback Is Light