Milestone 4 & 5: Playwright Test Video Integration + Embedded AI via Embabel

[jmjava]

Overview

This PR delivers both planning artifacts (2 milestones, 18 issue specs) and full working implementation of the core Playwright test video integration pipeline and embedded AI infrastructure.

---

Implemented (all tested, 174 passing, 0 regressions)

Issue 11: AI Provider Abstraction Layer — src/docgen/ai_provider.py

Decouples docgen from direct OpenAI API calls. Enables switching providers via config.

Component	Description
AIProvider protocol	chat(), tts(), transcribe()
OpenAIProvider	Drop-in wrapper for current OpenAI SDK calls
OllamaProvider	Local model support via Ollama REST API
HybridProvider	Routes chat→Ollama, tts/transcribe→OpenAI
get_provider(config)	Factory from docgen.yaml or DOCGEN_AI_PROVIDER env var

Three call sites refactored: wizard.py, tts.py, timestamps.py — all use the provider abstraction now. Whisper model is configurable (was hard-coded).

ai:
  provider: openai          # "openai", "ollama", "embabel"
  whisper_model: whisper-1
  ollama_url: http://localhost:11434
  embabel_url: http://localhost:8080/sse

Issue 1: Playwright Trace Event Extractor — src/docgen/playwright_trace.py

Parses Playwright trace.zip archives, extracts browser actions with normalized timestamps.

• Handles before/after, action, and event entry formats
• Outputs events.json for narration sync
• CLI: docgen trace-extract [--trace path]

Issue 2: Playwright Test Runner — src/docgen/playwright_test_runner.py

Runs existing Playwright tests with video+tracing enabled, collects artifacts.

• Supports pytest and npx playwright test frameworks
• Collects video + trace from output directories
• Graceful failure handling (video captured even if assertions fail)
• CLI: docgen playwright-test [--test filter] [--timeout 300]

Issue 3: Event-to-Narration Synchronizer — src/docgen/playwright_sync.py

Aligns narration audio timing to browser events — the core sync engine.

• Matches configured anchors (action+selector → narration keyword)
• Auto-match mode: extracts keywords from selectors/URLs, cross-references with Whisper words
• Computes piece-wise speed adjustment factors between anchor points
• Outputs sync_map.json with anchors and speed segments
• CLI: docgen sync-playwright [--segment 03] [--dry-run]

Issue 4: Video Speed Adjustment + Compose — src/docgen/compose.py

Integrates type: playwright_test into the existing Composer.

• Uniform speed adjustment via ffmpeg setpts filter
• Piece-wise speed: split at anchor points → retime each piece → concatenate
• Handles WebM→MP4 and path resolution fallbacks

Issue 5: Config Extensions — src/docgen/config.py

New config blocks for playwright test integration:

playwright_test:
  framework: pytest
  test_dir: tests/e2e/
  video_dir: test-results/videos/
  trace_dir: test-results/traces/
  min_speed_factor: 0.25
  max_speed_factor: 4.0

visual_map:
  "03":
    type: playwright_test
    test: tests/e2e/test_wizard.py::test_setup
    source: test-results/videos/test_setup.webm
    trace: test-results/traces/trace.zip
    events:
      - narration_anchor: "email"
        action: fill
        selector: "#email"

Issue 6: Pipeline Integration — src/docgen/pipeline.py

Full pipeline flow with Playwright test stages:

TTS → Timestamps → VHS Sync → Manim → VHS → Playwright Tests → Trace Extract → Playwright Sync → Compose → Validate → Concat → Pages

• --skip-playwright-tests flag for docgen generate-all
• Only runs playwright stages when playwright_test segments exist

Issue 14: CLI Chat Interface — src/docgen/chat.py

Interactive terminal chat backed by the AI provider:

$ docgen chat
docgen> generate narration for segment 03 about the wizard setup
[response from AI]

docgen> /status
  Config: docs/demos/docgen.yaml
  Segments: 01, 02, 03...

• /help, /status, /clear, /quit commands
• Project context injection (segments, files, visual_map)
• --non-interactive mode: echo "prompt" | docgen chat --non-interactive
• --provider and --model overrides

---

Test Coverage

Module	Tests	Status
test_ai_provider.py	18	Pass
test_playwright_trace.py	32	Pass
test_playwright_sync.py	11	Pass
test_playwright_test_runner.py	5	Pass
test_chat.py	12	Pass
Existing tests	96	Pass (0 regressions)
Total	174	All pass

---

New CLI Commands

Command	Description
docgen chat	Interactive AI chat for narration, pipeline, diagnosis
docgen playwright-test	Run Playwright tests with video+tracing
docgen trace-extract	Extract events from Playwright traces
docgen sync-playwright	Sync narration timing to browser events
docgen ai-provider	Show active AI provider configuration

---

Issue Implementation Status

Milestone 4 — Playwright Test Video Integration

#	Issue	Status
1	Trace Event Extractor	DONE
2	Test Runner Integration	DONE
3	Event-to-Narration Sync	DONE
4	Video Speed Adjustment	DONE
5	Config & Visual Map	DONE
6	Pipeline Integration	DONE
7	Auto-Discovery	Spec only
8	Anchor Auto-Detection	Spec only
9	Validation Extensions	Spec only
10	Documentation & Dogfood	Spec only

Milestone 5 — Embedded AI via Embabel

#	Issue	Status
11	AI Provider Abstraction	DONE
12	Embabel Agent Definitions	Spec only
13	Python MCP Client	Spec only
14	CLI Chat Interface	DONE
15	Wizard Chatbot Panel	Spec only
16	Script Generation Agent	Spec only
17	Error Diagnosis Agent	Spec only
18	Local Model Support	Partially done (OllamaProvider implemented)

---

Files Changed

NEW:
  src/docgen/ai_provider.py              # AI provider abstraction
  src/docgen/playwright_trace.py         # Trace event extractor
  src/docgen/playwright_test_runner.py   # Test runner integration
  src/docgen/playwright_sync.py          # Event-narration sync engine
  src/docgen/chat.py                     # CLI chat interface
  tests/test_ai_provider.py             # 18 tests
  tests/test_playwright_trace.py        # 32 tests
  tests/test_playwright_sync.py         # 11 tests
  tests/test_playwright_test_runner.py  # 5 tests
  tests/test_chat.py                    # 12 tests
  milestones/milestone-4-*.md           # Architecture doc
  milestones/milestone-5-*.md           # Architecture doc
  issues/playwright-test-integration/   # 10 issue specs
  issues/embedded-ai-embabel/           # 8 issue specs

MODIFIED:
  src/docgen/config.py      # ai_config, playwright_test_config
  src/docgen/cli.py         # 5 new commands
  src/docgen/compose.py     # playwright_test handler + retiming
  src/docgen/pipeline.py    # playwright_test stages
  src/docgen/tts.py         # uses provider
  src/docgen/timestamps.py  # uses provider
  src/docgen/wizard.py      # uses provider
  pyproject.toml            # embabel optional dependency
  tests/test_pipeline.py    # fix for new visual_map attr