Native Claude Code integration (skill / agent harness)

[dawsonpar]

Status

Planned / deferred until the eval set grows.

What

Ship a first-class Claude Code skill that exposes SetOptics rally detection
as a single agent-callable tool, so a user can drop into any project and run:

/setoptics-detect path/to/footage.mp4

without having to know which script to call, which mode is appropriate,
or how to wire the output back into downstream tooling.

Why it's not in v0.1

A first research arc (v1 through v5) built a 1 Hz multivariate detector
trained on a small ground-truth set. It hit F1@2s = 0.98 on the training
videos but only 0.29 on a held-out, unseen video. Without generalization
across recording conditions (camera angle, gym lighting, microphone, court
layout) the skill would mislead users on their own footage. The decision
was to delay shipping the skill until either:

1. The ground-truth dataset grows to >= 10 reference videos spanning
   diverse recording conditions, and
2. An architectural shift (cross-video normalization, scale-invariant
   features, or a domain-adaptation step) removes the systematic
   single-video bias.

Until both are addressed, the public repo ships only the underlying
detectors (signal, videomae, ensemble, llm), which the user can
invoke directly. That's enough to be useful; the skill is the polish on top.

Acceptance criteria

• Skill is installable into Claude Code via the standard skill mechanism.
• One canonical command invokes detection on any local video.
• Output JSON matches the existing {segments: [{start_ms, end_ms, type}]}
  format used by the CLI scripts.
• Works against signal, ensemble, or llm mode via a flag.
• Cross-video F1@2s >= 0.80 on a held-out test set of >= 5 videos that
  were never seen during prompt or threshold tuning.

Out of scope for the skill

• Hosting (this is a local-only skill).
• Auth, billing, accounts. None of that is in the open-source repo.
• A Cursor / Codex / Aider-specific integration. Those harnesses can call
  the scripts directly via AGENTS.md and do not need a custom skill.

Want to help?

PRs welcome on the generalization side, especially:

• Cross-video normalization for the signal features.
• Data augmentation for the VideoMAE head.
• More ground-truth video labeled with the format documented in
  docs/annotation.md.