Capturing the stated rationale behind game studio operations.
This system is an AI-powered infrastructure designed to bridge the provenance gap in live-service game development. While version control (Git) captures what changed, the reasoning behind those changes is often lost in fragmented Jira tickets and Slack threads. This project explores how AI can passively monitor developer discussions, identify consequential decisions, and structure their reasoning into a queryable institutional memory.
High-velocity game studios suffer from Reasoning Decay. Decisions made during live incidents or balance tweaks evaporate as teams move to new tasks. This leads to institutional memory loss, circular regressions, and inefficient post-mortems. This project provides infrastructure for capturing auditable rationale for operational choices.
- Passive Ingestion: Monitors Jira/Linear tickets for decision-level signals.
- Signal Detection: Uses LLMs to identify consequential choices (e.g., balance tweaks, rollbacks) based on reasoning language.
- Human-in-the-Loop (HitL): A human authority reviews and canonizes candidates to mitigate interpretation risk and hallucination.
- Deep Extraction & Provenance: Structures stated rationale with Extraction Spans (exact text citations) and lineage tracking.
- State-Driven Lifecycle: Manages decisions through Epistemic and Lifecycle State Machines (e.g., Proposed -> Confirmed -> Superseded).
This system is built on a foundation of Governed AI Operations:
- Augmentation, Not Autonomy: AI proposes candidates for the institutional memory; human experts provide the final "source of truth."
- Auditability by Default: Every extracted rationale is bound to a specific source artifact via inspectable citations (Extraction Spans).
- Confidence Context: AI confidence scores are used for queue prioritization and "Skeptical Auditor" flags, never for automated bypass of human review.
- Traceable Lineage: Decisions are treated as immutable events in an append-only log to support temporal reconstruction.
The following metrics represent infrastructure validation using synthetic fixtures. They prove the efficacy of the signal detection pipeline and the "surface-ability" of decision patterns within a controlled game studio dataset.
| Metric | Value | Rationale |
|---|---|---|
| Accuracy | 94.3% | Overall classification performance on curated studio scenarios. |
| Precision | 92.3% | Optimizing for a clean confirmation queue to minimize reviewer fatigue. |
| Recall | 96.0% | Prioritized to ensure critical operational signals are not lost before human review. |
Note: These benchmarks use Grade: BRONZE (Synthetic) evidence. They validate that the system correctly identifies reasoning structures, not that the model is "solving" game design. See evals/RESULTS.md for full context.
Validated through 5 representative studio scenarios to ensure operational relevance:
- Critical Exploit Response
- Emergency Server Rollback
- Event Economy Correction
- Community Visibility Escalation
- Meta Rebalancing
For a detailed analysis of the project's architecture, governance, and utility, see the Portfolio Documentation:
- Executive Summary - High-level overview for leadership.
- docs/portfolio/01-problem.md - The "Reasoning Decay" challenge.
- docs/portfolio/02-solution.md - Technical pillars and architecture.
- docs/portfolio/03-evaluation.md - Benchmark results and performance metrics.
- docs/portfolio/04-governance.md - AI integrity, trust models, and audit trails.
- docs/portfolio/05-operational-utility.md - Scenario-based value validation.
- docs/portfolio/06-limitations.md - Technical constraints and known risks.
- docs/portfolio/07-why-this-matters.md - Strategic value for AI PQM and Governance.
- Sequential 5-stage ingestion pipeline.
- Multi-dimensional confidence model (Signal, Rationale, Temporal).
- Extraction span citation system for auditability.
- Lifecycle and Epistemic state machines.
- End-to-end integration tests for the decision lifecycle.
- Regression tests for signal detection models.
- Validation of 5 realistic studio scenarios.
- Production connectors for Jira REST API and Linear GraphQL API.
- Automated Slack notification dispatch.
- Historical bulk import for seeding existing projects.
- "Provenance Gap" proactive surfacing for unexplained Git drift.
- Node.js 18+
- Postgres with
pgvectorextension
pnpm install
# or npm install# Run signal detection benchmark
pnpm run eval:signalsThis project is a portfolio artifact demonstrating expertise in AI Product Quality Management (PQM), Governance, and Operations.