DetectionThenIdentityVideoPipeline
title: Detection-then-Identity Video Pipeline type: technique created: 2026-06-22 last_updated: 2026-06-22 related: ["radar/languages/Ultralytics", "radar/languages/DeepFace", "radar/tools/FireRedOpenStoryline", "radar/tools/EditMind"] sources: ["https://blog.ippon.fr/2026/05/18/computer-vision-et-rugby-retour-dexperience-sur-la-construction-dun-pipeline-danalyse-video-avec-yolo-et-deepface/"] radar_quadrant: Techniques radar_ring: Assess radar_position: inner
Detection-then-Identity is a two-stage video analysis pattern in which a fast object detector (typically YOLO) processes every frame to locate subjects, and an expensive identity model (typically a face recognition library such as DeepFace) runs only on the detected bounding boxes. Identity results are cached per track ID so that the identity model is called once per subject rather than once per frame. The output is a structured segment list -- timestamps, subject identities, and detected events -- that feeds a downstream editing tool or report generator.
| Stage | Responsibility | Typical Tool |
|---|---|---|
| Detection | Locate subjects per frame; produce bounding boxes and track IDs | YOLO (ultralytics) |
| Identity | Match bounding-box crop to a reference database | DeepFace (FaceNet512) |
| Caching | Skip identity call once a track ID has a confirmed match | Per-track dict |
| Output | Timestamped segment list or structured report | JSON / LLM |
The frame-to-timestamp conversion applies the standard wrapper: timestamp = frame_index / fps.
The identity stage is the bottleneck. DeepFace's find() call on a 600-image database adds meaningful latency per frame. The standard mitigation is a stability counter: once a track ID has returned a consistent identity match across a minimum number of frames (e.g., 5), the result is cached and subsequent frames skip the identity call entirely. The counter prevents false positives from a single noisy frame triggering a permanent wrong assignment.
Nathan Sornet at Ippon Technologies applied this pattern to rugby match footage (May 2026). The pipeline identified individual players using FaceNet512 against a 600-player scraped database (reported accuracy 98.4%). A second fine-tuned YOLO detected jersey numbers as a fallback when faces were obscured (mAP 98.4%). A separate fine-tuned YOLO handled set-piece recognition (mêlée mAP 99.5%, lineout 99.6%). An LLM transformed extracted statistics into a structured coaching report. The author rejected Mask R-CNN/Detectron2 due to Apple Silicon dependency failures and PyTesseract OCR for jersey numbers due to motion blur sensitivity.
The pattern generalises to any video library where footage must be retrieved by subject identity: sports analytics, event highlight extraction, surveillance, or personal video archives. The detection layer is agnostic to identity; swapping YOLO classes (person, car, animal) requires no change to the identity stage. For non-face identity (vehicle plates, logo recognition), the identity stage can be replaced with an alternative matching model while the frame structure remains unchanged.
radar/languages/Ultralytics is the detection layer. radar/languages/DeepFace is the identity layer. radar/tools/FireRedOpenStoryline is the natural downstream editing consumer: it accepts natural language instructions referencing timestamps produced by this pipeline. radar/tools/EditMind implements the same pattern internally but exposes no export API, making it non-composable with downstream tools.
Placed in Techniques / Trial / inner.
The pattern has confirmed production use in a quantified deployment (Ippon Technologies rugby pipeline, May 2026) with documented accuracy metrics and an explicit caching strategy for the identity bottleneck. Inner position reflects direct applicability to the video discovery workflows this radar tracks and the availability of all required components as open-source Python libraries with no infrastructure dependencies.
The ring stays at Trial rather than Adopt because the pattern requires a reference identity database (face images or object templates) that must be assembled per deployment; it is not a zero-configuration technique. Dataset assembly cost and compute constraints on the identity stage (GPU recommended for large databases) are the primary barriers to broader adoption.
Trial gate: confirmed use of radar/languages/Ultralytics and radar/languages/DeepFace together in a production video pipeline with identity caching and structured output consumed by a downstream tool.
