🧪 feat(tests): L6 deterministic-trajectory dogfood + opt-in capture (#108)

[ZaxShen]

Closes #108.

Summary

Manual L5 dogfood was the release bottleneck. L6 automates it:
pre-seed DB → run real claude -p → assert MCP/tool sequence matches FLOWS.md.

The trajectory capture is opt-in (TMB_DEBUG_TRAJECTORY=1) so production users see zero overhead.

What landed

Schema (additive — schema_version stays at 1)

New debug_trajectory table (15th in the DB), populated only when env is set. Indexed on (session_id, step_n) for fast L6 reads.

Capture wiring

Source	Where
MCP tool calls	mcp/trajectory-server/src/index.ts — wraps CallToolRequestSchema handler
Non-MCP tool calls (Bash, Read, Write, Edit, Task, Skill)	New PreToolUse hook scripts/hooks/debug-trajectory.sh, registered in hooks/hooks.json with matcher: "*"

Both fail-soft: any capture error is swallowed so the actual tool call never breaks.

Test infrastructure

tests/dogfood/
├── run-l6.sh                    ← runner (env + tools check, dispatch)
├── lib/flow-helpers.sh          ← l6_setup_scratch_project, l6_seed_db,
│                                  l6_run_claude, l6_assert_trajectory
├── flows/                       ← 16 flow scripts
├── fixtures/                    ← 3 pre-seed SQL files
└── expected/                    ← expected-trajectory files

4 fully wired flows

Flow	Asserts
01-onboarding	identity_get + config_get probes → identity_set + 3x config_set + ledger_log(tmb_onboarding_complete)
02-simple-task	bro detects code-touching ask → triage simple → issue_create + task_create_batch + Task spawn + planning_complete
D-direct-mode	≤3-line typo → Edit + commit + direct_mode_used. Hard invariants: NO task_create_batch, NO Task spawn, exactly one direct_mode_used event
95-anonymous-cold-restart	Cold session with Anonymous identity must skip re-onboarding. Hard invariants: NO identity_set, NO config_set after the cold restart. Locks #95 regression.

12 scaffolded flows (auto-skip until expected-trajectory authored)

03-difficult-task, 04-agent-creator, 05-skill-creation, 06-push-gate, 07-architecture-regen, 08-swe-retry, 09-roundtable, C-consultant, 32-team-config, 92-base-branch, 94-arch-bootstrap, 96-halt-on-error.

Each is ~30 lines of pattern-match copy/paste once the expected-trajectory file is authored.

CI

.github/workflows/l6-dogfood.yml — triggers:

• Tag pushes (every release gets a green/red signal)
• PRs labeled L6 (opt-in for risky doctrine changes)
• Manual workflow_dispatch

Soft-fails when CLAUDE_CODE_OAUTH_TOKEN secret is absent (forks / external PRs don't break red). Uploads trajectory dumps as artifacts on failure for debugging.

Stale doctrine cleanup (audit found 3 issues)

• skills/tmb_first-run-onboarding/SKILL.md: fixed event_type from stale tmb_bootstrap_complete → tmb_onboarding_complete + dropped "file copies" reference (swe + pr-reviewer ship globally now)
• skills/tmb_agent-creator/SKILL.md: dropped tmb_bootstrap reference (skill is gone in v0.3.0+)
• CLAUDE.md: removed the "tmb_bootstrap is being retired" sentence — already retired

Unverified assumption (flagged for follow-up)

claude -p mode behavior with AskUserQuestion is unknown. If the form auto-fails in headless mode, the 01-onboarding flow's trajectory will be shorter than expected — a real signal to address (could be a CC bug, could be a doctrine adjustment for headless mode).

The L6 design pre-seeds DB to skip past forms for every other flow. Only 01-onboarding actually exercises the form path.

Test plan

• L1 lint passes (16 lints)
• L2 unit passes (245 + 2 new schema tests = 247)
• L3 hooks pass
• L4 workflow-sim passes
• Build green; dist/ committed up-to-date
• L0 install-smoke (CI Docker)
• L6 itself — runs in CI on this PR if labeled L6, OR run locally with bash tests/dogfood/run-l6.sh

Risk profile

• Schema change is additive + opt-in — zero risk to production users.
• MCP server wrapper is null-safe and try-wrapped — never breaks the real tool call.
• New hook is matcher: "*" PreToolUse — fires on every tool call but exits 0 immediately when env unset (~1ms overhead).
• All L1-L4 still pass.

Follow-ups (after merge)

1. Wake-up validation: assign label L6 on this PR (or trigger workflow_dispatch) → CI runs the 4 wired flows for the first time. If claude -p behavior surprises us, follow-up issue.
2. Fill in 12 scaffold expected-trajectories — each is a one-line trajectory dump after a single test run with TMB_DEBUG_TRAJECTORY=1.
3. Decide if L6 should run on every PR to dev — currently opt-in via L6 label. Could expand based on token cost data after a few runs.

🤖 Generated with Claude Code