Quality Assurance

Quality Assurance

"Quality is not a phase at the end. It is a discipline embedded in every commit."

---

The Problem

Game framework code has a unique quality challenge: it runs in every project that adopts it. A bug in your game code affects your game. A bug in your framework code affects every game built on it. The cost of a defect scales with adoption.

UE5 amplifies this challenge with its macro-heavy reflection system, garbage collector, delegate lifecycle, and cross-DLL linking. The failure modes are not just logic bugs — they are dangling pointers from unbalanced delegate bindings, GC-collected objects from missing UPROPERTY markers, cross-DLL symbol resolution failures, and header collision errors from the Unity Build system. These are not theoretical risks. They are the actual bugs that cost hours of debugging in production UE5 projects.

Most UE5 projects address quality reactively: bugs are found in testing, reported, and fixed. A framework cannot afford this. By the time a framework bug reaches a user's project, the damage to trust is already done. Quality must be proactive, automated, and baked into the development process.

PGX addresses this with a multi-layer QA pipeline that combines static analysis, architectural enforcement, standardized testing, simulation-based validation, and formal auditing.

---

Layer 1: Static Analysis (clang-tidy)

Configuration

PGX uses clang-tidy with a carefully tuned configuration across three layers:

Layer 1 — Code Smells: Modernization checks (missing override keywords, unnecessary copies, boolean expression simplification, else-after-return). These catch the patterns that accumulate into maintenance debt.

Layer 2 — Potential Errors: Narrowing conversions (implicit float to int32, int64 to int32), function size limits, unused parameters. These catch the patterns that become bugs under specific conditions.

Layer 3 — Framework Patterns: Custom checks for patterns specific to UE5 and PGX conventions. These enforce the engineering standards that static analysis tools do not know about by default.

30 checks are active. UE-generated noise (engine headers, auto-generated code) is filtered out. Only PGX source files are analyzed.

Orchestration

A Python orchestrator runs clang-tidy per-plugin with:

• Incremental cache: Only re-analyzes files that changed since the last run. A full 13-plugin analysis takes minutes; incremental runs take seconds.
• Markdown + JSON reports: Human-readable reports for review, machine-readable reports for CI.
• CI mode: Exit code 0/1 for pass/fail integration into GitHub Actions.

Results

Two formal remediation rounds have been completed:

Round	Warnings Fixed	Key Categories
Round 1	684	489 missing override, 26 narrowing conversions, 150 unused parameters, 11 assorted
Round 2	273	63 mixed (destructors, overrides), 207 unused parameters, 3 assorted
Total	957	0 remaining (excluding 15 intentional skips)

The 15 intentional skips are large construction/registration functions that exceed the line-count threshold by design — splitting them would harm readability without improving safety.

---

Layer 2: Audit Doctrine (DA0-DA12)

Static analysis catches syntax-level issues. The Audit Doctrine catches architecture-level issues that no static analyzer can detect.

13 rules, each addressing a specific category of defect that has historically caused real bugs in PGX:

Memory Safety

DA0 — UPROPERTY Guard: Every UObject pointer stored as a class member must be marked with the UPROPERTY macro. Without it, the garbage collector does not track the reference, and the pointed-to object can be collected while the pointer is still live. This is the single most common crash cause in UE5 C++ projects.

DA1 — Delegate Lifecycle Symmetry: Every delegate binding must have a corresponding unbinding in the appropriate teardown function. Bindings in initialization must be unbound in deinitialization. Bindings in begin-play must be unbound in end-play. An asymmetric delegate is a dangling pointer waiting to fire.

Architecture

DA2 — Star Topology: No runtime module may depend on another L2 runtime module (with three documented, justified exceptions). This rule prevents the dependency graph from becoming a web where changing one system breaks another.

DA3 — Build Dependency Integrity: Every header include implies a build dependency. If a file includes a header from module X, module X must appear in the build configuration. Missing dependencies compile locally but fail on clean builds or CI.

Convention Compliance

DA4 — Logging Hygiene: No temporary log categories in production code. Every system uses its own named log category. Appropriate verbosity levels (Log for normal operations, Warning for recoverable issues, Error for failures).

DA5 — Tag Pattern: All framework tags use the declared-in-header, defined-in-source pattern. No static tag definitions in headers (which fail under certain build configurations). Editor-only modules use runtime tag requests instead of native tag declarations.

DA6 — Factory-Registry Sync: Every Data Asset type must have a factory, a registry entry, and a type action — all referencing the same metadata. Drift between these three locations means assets appear in some places but not others.

DA7 — Blueprint Conventions: Blueprint libraries are the sole entry point for Blueprint-callable functions. Subsystems do not duplicate Blueprint-exposed functions. Categories follow the uniform pattern.

UX

DA8 — Progressive Disclosure: Data Assets show only essential properties by default. Advanced properties are collapsed under the standard UE "Advanced" section. No Data Asset should overwhelm a new user with 30 properties when they only need 3 to get started.

DA9 — Editor Wiring Completeness: Every system must complete the 13-item editor wiring checklist. No system ships without an icon, a factory, a registry entry, a toolbar entry, a hub card, and an inspector panel.

Documentation

DA10 — Truth Sync: Documentation and source code must agree. When code changes, documentation must be updated in the same operation.

DA11 — Deliverable Completeness: Every system must have all 15 per-system deliverables (or documented justifications for N/A items).

DA12 — Version Consistency: Version numbers in manifests, plugin descriptors, and documentation must match. Drift between version references creates confusion about what is actually deployed.

---

Layer 3: Test Utilities

Every implemented system provides a standardized test utility with a minimum of 7 test functions. The standard set:

1. Initialization test: Verify the subsystem initializes correctly with default configuration
2. Configuration test: Verify Data Asset auto-discovery and configuration application
3. Core API test: Exercise the 3-4 most critical public functions
4. State test: Verify state transitions and state queries
5. Delegate test: Verify delegate binding/firing/unbinding lifecycle
6. Error handling test: Verify graceful behavior with invalid inputs
7. Cleanup test: Verify proper teardown and resource release

Systems with additional complexity provide additional test functions. The total across all 13 systems exceeds 95 test functions.

All test utilities follow the same pattern: a setup phase that creates test fixtures, an execution phase that exercises the system, and a teardown phase that cleans up. Test fixtures use the same public API as game code — there are no backdoor test-only interfaces.

The Test Dashboard (an editor panel) runs all test utilities and presents results with per-system color coding, pass/fail KPIs, and JSON export for CI integration.

---

Layer 4: Simulation Harness

The Simulation Harness (documented in its own wiki page) exercises approximately 160 API calls across all 13 systems, covering roughly 90% of the public surface area. This is not unit testing — it is integration testing at the system interaction level.

The harness catches the bugs that individual test utilities miss: cross-system timing issues, cascade failures when one system's state change triggers another system's delegate, and data consistency problems that only appear when multiple systems are operating simultaneously.

---

Layer 5: Multi-Perspective Panel Audits

PGX pioneered a formal audit process for editor UI quality:

1. Audit guide: A structured evaluation criteria across 5 dimensions (layout, information density, interactivity, visual consistency, accessibility)
2. Four independent reviewers: Different perspectives and strengths audit all panels simultaneously
3. Per-panel scoring: Each panel receives a score out of 20 on each dimension
4. Consolidated report: Cross-referencing all four audits, deduplicating findings, prioritizing by severity

Results from the initial audit:

Metric	Value
Panels audited	22
Systemic issues identified	12
Critical findings	8
Major findings	42
Minor findings	28
Suggestions	15
Average UX score (pre-remediation)	11.9 / 20
Implementation plans generated	9
Estimated UX score (post-remediation)	16.5+ / 20

The remediation was executed across 9 implementation plans touching all 22 panels: shared infrastructure utilities, standardized badges and tags, section headers, KPI chips, table formatting, footers, and empty states.

---

Layer 6: Definition of Done

No system is marked as complete without verifying:

• Code compiles with 0 errors, 0 warnings (warnings treated as errors)
• All 15 per-system deliverables completed (or justified N/A)
• Documentation created and verified (Architecture + Usage + Testing guides)
• 13-item editor wiring checklist completed
• Console commands registered and functional
• All engineering standards (S0-S7) compliance verified

This checklist is not aspirational. It is a blocking gate. A system with working code but missing documentation is not done. A system with an inspector panel but no hub card is not done. The checklist exists because each of these items has been forgotten at least once and caught only through formal verification.

---

Lifetime Safety

Beyond the standard QA layers, PGX underwent a dedicated lifetime safety audit addressing a specific class of bug: raw this pointer captures in asynchronous callbacks and delegate bindings.

In Slate panels and async operations, capturing this in a lambda creates a dangling pointer risk — if the widget is destroyed before the callback fires, the program crashes. The audit converted approximately 190 lines across 24 files:

• Async task callbacks: converted to weak pointer captures with validity checks
• Slate delegate bindings: converted from raw to shared-pointer-aware binding
• Timer callbacks: converted to weak lambda pattern
• Named methods replaced inline lambdas where capture semantics were unclear

---

CI Integration

The QA pipeline integrates with GitHub Actions:

• Validation workflow: Runs the workspace validation script (directory structure, version consistency, topology, module dependencies)
• Package workflow: Builds release packages with UE version tagging
• Sync workflow: Curated sync to the public repository

The clang-tidy orchestrator runs in CI mode with exit code 0/1, making it suitable for PR checks.

---

The Philosophy

Quality in PGX is not a separate activity performed by a separate team. It is a set of automated checks, enforced conventions, and formal gates embedded in the development process. Every layer catches a different class of defect:

• Static analysis catches syntax and style issues
• Audit doctrine catches architecture violations
• Test utilities catch functional regressions
• Simulation harness catches integration issues
• Panel audits catch UX degradation
• Definition of Done catches incompleteness

No single layer is sufficient. Together, they form a pipeline where each stage's output feeds the next stage's input, and the final output is a system that is correct, complete, and polished.