feat: add Steering view — analyze and learn from the impact of human prompts on outcomes

[paulyuk]

Steering View

Problem

When you watch an AI coding session in Replay or Tracks, you see what the agent did: tool calls, file edits, reasoning. But you can't see which human prompts actually mattered or why. Which ones changed the direction? Which ones led to the most code? Which ones caused bugs that led to better tests? Which ones had too many hallucinations or continues retrying achieve a outcome or taste, requiring interventions (as we see in coach)?

Solution

The Steering view extracts every human prompt that redirected the agent, pairs each one with the git commits and code changes it produced, and shows the impact: lines changed, tests passing, and what was learned.

It understands not just the primary coding agent (Copilot, Claude Code, VS Code Copilot), but also sub-agents and squads — capturing the reasoning from dispatched agents as they respond to steering.

An agent-assisted analysis runs automatically in the background using the Copilot SDK to improve the quality of What Happened, Level-Up, and Impact columns beyond what static heuristics can produce. A steering intelligence panel shows density scoring, category breakdown, and actionable insights derived from the session's steering patterns. This is effectively a "double-click" on the insights we see in coach, but scoped to each and every steering message or intervention.

Think of this as a solution hypothesis. More hypotheses and details around valuable insights we can gain from steering and interventions are here.

Screenshot

Quick Start

git fetch origin pull/44/head:feature/steering-view
git checkout feature/steering-view
npm install && npm run dev

Then start a coding session in another terminal. For example with Copilot:

copilot "add a /healthcheck endpoint that returns uptime and version"

Work with the agent for a few turns. Redirect it at least once — say things like "that's too verbose, simplify it" or "don't use console.log, use the existing logger." Those redirections are the steering the view captures.

Then open agentviz, load the session file, and click the Steering tab (last tab, after Coach).

What you see

Each row is either a human steering command (italic quotes, bright text) or a git commit (dimmer text, blue hash). For each steering command:

• What Happened — what the agent did as a result, including squad reasoning
• Level-Up — what was learned or unlocked
• Impact — lines changed, tests passing, eval grade

The detail panel shows the full squad response, files changed, commit info, and a Responding To section showing what the agent said that the user was reacting to.

Steering intelligence

Below the timeline, an expandable panel shows:

• Density score — steering commands per hour (high density = agent not matching taste)
• Category breakdown — quality, tone, bugs, naming, testing, visual, simplification
• Insights — actionable suggestions like "Quality redirected 4 times, consider a skill"

How it works

1. Session extractor — detects steering patterns in user messages (redirections like "instead", "try again", "don't")
2. Git history — commits classified and paired with the steering that caused them
3. Background analysis — Copilot SDK auto-analyzes entries for richer summaries (same SDK as Coach, no extra setup)
4. Intelligence panel — density scoring and category analysis from steering patterns
5. Persistent log — .agentviz/steering-v1.jsonl (gitignored, auto-generated) with redaction for secrets

Guardrails verified (per #43)

Check	Status
npm run typecheck	pass
npm test	473/474 (1 pre-existing)
npm run build	pass
Screenshots (8 required)	all present
Hardcoded hex colors	all theme tokens
README	Steering section added
Style guide	Section 20 added
Test coverage	3 test files, 48 tests

Files (18 total: 11 new, 7 modified)

routes/steering.js                  — git analysis, steering log, AI synthesis endpoint
src/components/SteeringView.jsx     — timeline table, filters, detail panel, intelligence panel
src/lib/steeringExtractor.js        — session heuristic extractor
src/lib/steeringAgent.js            — Copilot SDK analysis agent
src/__tests__/steeringExtractor.test.js — 27 tests
src/__tests__/steeringRoute.test.js     — 15 tests
src/__tests__/steeringView.test.jsx     — 6 tests
docs/evals.md                       — quality rubrics
docs/steering-demo.md               — demo walkthrough
docs/steering-insights.md           — research hypotheses for steering analysis
docs/screenshots/steering-view.png

---

Built with snap-squad — npx snap-squad init