Skip to content

Cross-format data model + milestone & quality SOWs#1

Merged
ktsaou merged 1 commit into
masterfrom
sow/data-model-and-milestone-planning
May 26, 2026
Merged

Cross-format data model + milestone & quality SOWs#1
ktsaou merged 1 commit into
masterfrom
sow/data-model-and-milestone-planning

Conversation

@ktsaou
Copy link
Copy Markdown
Member

@ktsaou ktsaou commented May 26, 2026

Summary

This PR lands two deliverables operator requested on 2026-05-26:

  1. Cross-format data model analysis — evidence-based investigation of all five source formats with real fixtures and upstream source code cited. Findings absorbed into the canonical event model and SQLite schema as first-class columns.
  2. Project split into concrete milestones — 11 new pending SOWs covering M2 adapters, M3 APM UI, M4 statistics, M5 release polish, plus the five quality-automation SOWs that enforce every gate in quality-gates.md.

SOW-0001 also moves to current/ with chunk renumber (specs first per the discipline contract).

What's in this PR

Specs updated (cross-format unified model)

  • .agents/sow/specs/canonical-events.md — full rewrite. Adds OpKind values system + compaction; SessionStatus adds abandoned/interrupted; cache tokens; provider_alias; reasoning_kind; chars_in/out; cwd; call_path; RootNativeID/ParentOpKey; cross-format mapping table.
  • .agents/sow/specs/data-model.md — full rewrite. Schema reflects every new field; new catalog_providers + catalog_cwds tables; explicit cross-format compatibility matrix; sub-agent linkage strategy table.
  • .agents/sow/specs/adapter-aiagent-v3.md — evidence-based rewrite (719 lines).
  • .agents/sow/specs/adapter-aiagent-v2.md — evidence-based rewrite (456 lines).
  • .agents/sow/specs/adapter-claude-code.md — evidence-based rewrite (795 lines).
  • .agents/sow/specs/adapter-codex.md — evidence-based rewrite (551 lines; openai/codex@8a94430b).
  • .agents/sow/specs/adapter-opencode.md — evidence-based rewrite (567 lines; anomalyco/opencode@2b3ddf9f).

SOWs

  • Moved: pending/SOW-0001-phase-1-foundation.mdcurrent/. Status: open → in-progress. Implementation plan renumbered: Chunk 1 = spec deltas, Chunk 2 = CI scaffolding (was Chunk 1).
  • Completed (done/): SOW-0002-20260526-cross-format-data-model-analysis.md — records the analysis work.
  • Pending (M2-M5 milestones):
    • SOW-0003-20260526-claude-code-adapter.md
    • SOW-0004-20260526-codex-adapter.md
    • SOW-0005-20260526-opencode-adapter.md
    • SOW-0006-20260526-apm-tracing-ui.md
    • SOW-0007-20260526-statistics-and-analytics.md
    • SOW-0008-20260526-release-polish-v0-1.md
  • Pending (quality automation):
    • SOW-0009-20260526-go-static-analysis-stack.md
    • SOW-0010-20260526-test-infrastructure-coverage.md
    • SOW-0011-20260526-fuzz-property-benchmark.md
    • SOW-0012-20260526-frontend-quality-stack.md
    • SOW-0013-20260526-repo-wide-gates-ci-workflow.md

Investigation methodology

Five parallel deep-research subagents (one per format), each instructed to:

  • Read ≥10 real files
  • Read upstream producer source code (or reverse-engineered equivalents)
  • Cite file:line evidence
  • Produce per-client adapter spec from evidence (not assumptions)
  • Surface gaps in the canonical model

Master assistant synthesized the gaps into the canonical updates.

Stats

  • 7 specs touched (5 adapter + 2 canonical), ~3,900 lines.
  • 13 SOWs total (1 moved + 1 completed + 11 new pending), ~2,400 lines.
  • Zero code changes — this PR is pure specs + SOWs.
  • 4 parallel subagents drafted the milestone + quality SOWs.

Test plan

  • Review canonical-events.md and data-model.md cross-walk tables for completeness.
  • Review at least one per-client adapter spec end-to-end (recommend claude-code or opencode — most novel).
  • Review milestone SOWs M2-M5 for scope correctness.
  • Review quality SOWs Q1-Q5 for gate coverage vs quality-gates.md.
  • Approve to merge — required (branch protection: enforce_admins enabled).

…roject split into milestones

## Cross-format data model analysis (SOW-0002)

Evidence-based investigation of all five source formats: ai-agent v2/v3,
claude-code, codex, opencode. Real fixtures inspected (294,316 v2 +
17,356 v3 + 3,614 claude-code + 2,585 codex rollouts + 3.9 GB opencode
SQLite), upstream source code cited at file:line. Five per-client
adapter specs rewritten from sketches (3,088 lines total).

Findings absorbed into canonical-events.md and data-model.md as
first-class fields rather than extras_json bloat:

- Op kinds: added `system` (v2/v3) and `compaction` (claude-code/codex)
- Session statuses: added `abandoned`, `interrupted`; documented
  indefinite-`running` for claude-code (no terminal signal)
- Session kinds: added `fork` (codex forked_from_id)
- Cache tokens (`tokens_cache_read`, `tokens_cache_write`) on sessions,
  turns, ops — surfaced by v3, claude-code, opencode
- `provider_alias` for opencode's user-defined provider names
- `reasoning_kind` (`summary` | `raw`) for codex's reasoning split
- `chars_in/out` for v2's character-based tool accounting
- `cwd`, `call_path`, `last_activity_ts` promoted from extras_json to
  first-class columns
- `PayloadRef.sha256`
- `RootNativeID` and `ParentOpKey` on SessionStartedEvent
- `Status='truncated'` on ops for codex limited-mode payloads
- `catalog_providers`, `catalog_cwds` rollup tables

Spec adds explicit cross-format compatibility matrix and sub-agent
linkage strategy table.

## SOW-0001 moved to current/ with chunk renumber

Status: open -> in-progress. Implementation plan renumbered to put
"Spec deltas" as Chunk 1 (UI theme, pricing.md, adapter spec
refinements); CI scaffolding becomes Chunk 2. Honors the
spec-first-then-test-then-code discipline.

## Milestone SOWs (pending operator approval, M2-M5)

- SOW-0003 claude-code adapter
- SOW-0004 codex adapter
- SOW-0005 opencode adapter
- SOW-0006 APM-style tracing UI (topology + timeline + span view)
- SOW-0007 Statistics & analytics (rollups + FTS5 search + dashboard)
- SOW-0008 Release polish + v0.1 tag

## Quality automation SOWs (pending operator approval)

- SOW-0009 Go static analysis stack (golangci-lint, gosec, govulncheck)
- SOW-0010 Test infrastructure + coverage gates
- SOW-0011 Fuzz, property-based, benchmark infrastructure
- SOW-0012 Frontend quality stack (ESLint flat, strict TS, Vitest
  coverage, Playwright + axe, bundle budget)
- SOW-0013 Repo-wide gates + canonical CI workflow

Each quality SOW cites the specific gates from quality-gates.md it
lands. Together they enforce every threshold in the spec.
@ktsaou ktsaou merged commit 3480190 into master May 26, 2026
1 check passed
@ktsaou ktsaou deleted the sow/data-model-and-milestone-planning branch May 26, 2026 11:39
ktsaou added a commit that referenced this pull request May 30, 2026
…ctivate

Prepare the opencode adapter SOW (the last of the five). opencode is
SQLite-backed (~/.local/share/opencode/opencode.db, WAL, ~4.36 GB live),
so the adapter replaces the JSONL parser+scanner+stream+tailer with a
read-only SQL delta-query layer + a watermark cursor + a poll loop.

Resolve two spec TBDs before any test/code: ratify task->session "emit
both" (tool Op + session Op as topology parent); firm the per-turn token
rule to delta-from-previous-message-totals with an explicit
implementer-verify-on-live-DB note (only the step-finish cumulative
pattern is independently verified so far).

Fill the gate (problem model, live-DB evidence, SQLite structural map,
watermark cursor, canonical mapping incl. the verified cumulative-token
delta, read-safety against the live concurrent writer, sensitive-data
plan, chunked implementation + validation plans tied to acceptance #1-8,
and six recorded CTO decisions) and move the SOW to current/.

Purely additive: a new internal/adapters/opencode/ package + an additive
auto-discovery probe in cmd/ai-viewer-ingest/sources.go; no canonical,
ingest, or store change (the catalog is already idempotent post-SOW-0004
and every target canonical field exists).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant