Test Suite Audit

Test Suite Audit — Coverage Analysis

Audience: Programmers | Mixed

Thread Context: N/A (audit/reference)

Platform/Backend Scope: All platforms (tests run headless / NullRHI; many use standalone reimplementations for CI)

Overview

A coverage analysis of SparkEngine's test suite: how many tests exist, which subsystems they cover, and where gaps remain. The suite has grown substantially since the original audit (April 2026): from 297 test files / 3,647 tests to 485 files / ~5,991 tests as of 2026-06-08. Most of the original audit's "critical missing test suite" and "medium priority" gaps have since been filled.

---

Current Metrics (re-measured 2026-06-08)

Metric	Value
C++ test files under Tests/	485
Total TEST / TEST_F macros	~5,991
Engine headers (SparkEngine/Source)	523
Editor headers (SparkEditor/Source)	126
GameModule headers	115

Measurement commands:

find Tests -name '*.cpp' | wc -l
grep -rcE '\b(TEST|TEST_F)\s*\(' Tests --include='*.cpp' | awk -F: '{s+=$2} END{print s}'

---

Status of the Original Audit's Critical Gaps

The original audit listed Priority-1 systems with zero dedicated tests. Most have since gained orchestration tests; the audit's own "Phase 1/2" recommendations were largely executed.

Original gap (Priority 1/2)	Status
NetworkManager orchestration	Addressed — networking has expanded coverage (e.g. TestNetworkManager*, plus existing TestNetBuffer, TestReliableChannel, TestNetworkEncryption)
ECSystems ordering / SystemManager	Addressed
MaterialSystem validation	Addressed
AssetPipeline cache (LRU)	Addressed
EngineSettings parser	Addressed
PerceptionSystem math	Addressed
InputManager state	Addressed
GamepadInput processing	Addressed
Camera transforms	Addressed
WorldServer / AreaServer routing	Partial → expanded

The audit's "MEDIUM" tier was already marked resolved in the source document at the time of writing: TestAudioMixerBus, TestMusicManager (24 tests), TestAngelScriptEngine (14), TestScriptHotReload (16), TestConsoleVariables, TestVRSystem (12), TestWeaponMechanics, TestGameObjectTransforms, TestSeamlessAreaManager (12) all exist.

---

Remaining Coverage Gaps

These categories were the original audit's standing weak spots and remain the most likely places to find untested code:

• Editor systems (SparkEditor/Source, 126 headers): Historically ~4% tested. The most testable-without-GPU targets remain UndoRedoManager, CommandPalette, PrefabManager (now covered by Tests/TestEditorSubsystemsReal.cpp), SceneSerializer, EditorLayoutManager, VersionControlSystem (Tests/TestVersionControlSystemGitReal.cpp), LevelStreamingSystem, and gizmo transform math (Tests/TestEditorGizmoTransformReal.cpp); AssetDatabase was deleted (no production caller).
• Game modules (115 headers across 10 modules): Per-module mechanic tests are uneven. FPS and MMO have coverage; RPG/ARPG/RTS/Racing/Platformer/OpenWorld/VisualScript modules are lighter.
• RHI backend implementations (D3D11Device, D3D12Device, VulkanDevice, MetalDevice, OpenGLDevice): require hardware; tested implicitly via NullRHI and standalone logic tests.

---

Acceptably-Untested Categories

These do not need dedicated unit tests:

• Platform stubs (PlatformDirectXMathStubs, PlatformD3DStubs, PlatformAudioStubs) — exercised implicitly.
• Type / enum headers (AnimationTypes, PhysicsTypes, RHITypes, GraphicsEnums) — data-only.
• Trivial helpers (MakeDesc, DrawSortKey, LogMacros).
• Diagnostic debug tools (CpuDebugger, ThreadDebugger, CacheDebugger, IODebugger).

---

Test Conventions

• Framework: custom TestFramework.h with EXPECT/ASSERT macros (not GoogleTest).
• Naming: Test<SystemName>.cpp.
• A large majority of tests use standalone reimplementations (no engine/DirectXMath headers) for CI/Linux portability.
• All test files are registered in Tests/CMakeLists.txt and run via CTest:

cd build && ctest --output-on-failure --no-tests=error

See also the maintained AUTO: test inventory in wiki/.../Testing.md, regenerated by docs/sync-wiki.sh sync.

---

Source & Freshness

• Original audit: .claude/knowledge/test-suite-audit.md, last updated 2026-04-04.
• Re-measured against codebase 2026-06-08.
• OLD → NEW numbers updated:
  • Test files OLD 297 → NEW 485.
  • Total TEST/TEST_F OLD 3,647 → NEW ~5,991 (consistent with CLAUDE.md's "~5,962 across ~484 files").
  • Engine headers OLD 405 → NEW 523.
  • Editor headers OLD 109 → NEW 126.
  • GameModule headers OLD 102 → NEW 115.
• Findings now resolved since the original audit:
  • The five Priority-1 "critical, zero-test" systems (NetworkManager, ECSystems, MaterialSystem, AssetPipeline, EngineSettings) and the Priority-2 set (Perception, Input, Gamepad, Camera, WorldServer) now have coverage — the audit's Phase 1/2 plan was executed.
  • The entire MEDIUM tier was already resolved in-source (audio, scripting, CVars, VR, weapons, transforms, seamless streaming).
• Still open: editor and game-module coverage remain the primary gaps.

Related Pages

• Testing
• Benchmark Framework
• Golden Image Testing
• Codebase Health
• Load Test Baseline