Testing and QA

Testing and QA

What a TrayIconGuard change has to pass before it ships. Every gate below is executed, not aspirational; the evidence quoted is from 8.6.3 (2026-10-16, Windows 11 build 26300 + CI).

The gate ladder

#	Gate	How it is executed	What it catches
1	Warning-free build	./scripts/build.sh — GCC -Wall -Wextra; MSVC /W4 /permissive- in CI	new diagnostics, implicit conversions
2	Manifest gate	resources/app.manifest parses, supportedOS GUIDs unique, exactly one RT_MANIFEST(id 1) (checked with pefile)	the Win11 "side-by-side configuration is incorrect" failure class
3	Dependency audit	x86_64-w64-mingw32-objdump -p dist/TrayIconGuard.exe → only the OS allow-list	accidental runtime dependencies (the product promise is a single exe)
4	Static analysis	cppcheck --enable=warning,performance,portability src	shadowed members, missed initialisers
5	CLI matrix	every verb with its exit code — and with redirection: --status --json > state.json, --help > help.txt, --bogus → error: …, exit 1	output written to the console only, silently discarded when stdout was a pipe or a file. A console-only check cannot see it, and wine does not reproduce it
6	GUI smoke	TrayIconGuard.exe --ui-test <page> writes %TEMP%\tig-uitest-<page>.png; all 7 pages × light/dark, plus both wizard variants	layout and painting regressions (e.g. the 8.4.1 About z-order bug)
7	Updater end-to-end	loopback feed: download → SHA-256 → rename chain (.old) → relaunch, exit 0; live feed: --check-update → latest=<ver> newer=no	feed/asset drift, broken self-replacement
8	Publication check	release assets exist, and scripts/make_feed.py pins the feed from the served bytes	a feed whose sha256 does not match the asset the client downloads
9	Guard event-driven E2E	run --watch with the default pollMs: 0, force IsPromoted = 0 on a real icon, expect it restored within a few seconds	the 8.3.0–8.6.2 defect where the armed registry notification was never awaited and the loop never ran a pass

What stays manual

• Real-Windows behaviours wine cannot model: tray menu interaction, notification balloons, MSIX registration, per-monitor DPI, SmartScreen reputation.
• Code signing and reputation: see Installation and Deployment.

Continuous integration

.github/workflows/ci.yml runs on every push and every v* tag:

1. Cross-build (ubuntu, mingw-w64, static): gates 1–3;
2. Native build (windows-2022, MSVC): compile parity;
3. Release (tags only): normalize + upload the three assets, then run the feed job that regenerates and commits update/stable.json from those bytes.

Regression tests worth keeping green

• --status --json piped through ConvertFrom-Json (see the 8.6.2 fix).
• --check-update after an update prints newer=no — no update loop.
• Unknown argument exits 1 with a message.
• --version, --status, About diagnostics and the PE resource all agree on the version.
• --watch with pollMs: 0 re-promotes an icon whose IsPromoted was forced to 0 (8.6.3+; on 8.6.2 and earlier the value simply stays 0 — see The Guard Watcher).

See also

• Building from Source — the exact toolchain recipes
• Self-Updater Guide — the feed contract and its E2E test
• Independent-Audits — what was audited and what was not

Lesson: test redirection on the shipping subsystem

The redirected-output defect survived one release because it was verified on the wrong kind of build. TrayIconGuard.exe ships as a GUI-subsystem binary (add_executable(… WIN32 …) → WINDOWS_GUI); a hand-rolled g++ build without -mwindows produces a console binary (WINDOWS_CUI), where stdout is a real console/pipe handle and the defect simply does not appear. The GUI binary attaches to the parent console with AttachConsole, which replaces the process's standard handles — so a redirected stdout has to be saved before the call and restored with SetStdHandle afterwards (8.6.2).

Check the subsystem before trusting a CLI test:

# Subsystem field of the PE optional header: 2 = WINDOWS_GUI, 3 = WINDOWS_CUI
$b=[IO.File]::ReadAllBytes('.\\dist\\TrayIconGuard.exe'); $pe=[BitConverter]::ToInt32($b,0x3C)
[BitConverter]::ToUInt16($b,$pe+24+68)