CHAT v1.0.0

First public release of the CHAT reference implementation, accompanying
Computer Vision – ECCV 2026, pp. 597–616.

Six-stage pipeline from one text prompt to a paired dyadic clip, the two IFBR
refinement weights, the evaluation scripts, and 119 unit tests that need neither a
GPU nor a network.

Reproduction numbers

These were produced by this code, at 93 dyadic clips and 186 speaking sides.
The paper evaluates 1000 dialogues on a full-scale training run. The two scales
are not interchangeable, and no number below should be placed in a column beside a
paper number as though it were the same measurement.

Metric	Paper target	This code
CSIM	≥ 0.85	0.9024	meets
LPIPS	≤ 0.38	0.2210	meets
FVD, disjoint reference	≤ 365.03	155.11	meets in protocol; see the floor control below
LSE-D	≤ 8.07	8.471	does not meet
LSE-C	≥ 6.89	4.954	does not meet
FID, disjoint reference	≤ 17.33	31.53	does not meet; the target is unreachable at this scale
FID, identity-matched	—	video_refined 20.10, video 18.13, real-video floor 7.72	gap of 12.38 refined, 10.41 unrefined
Emotion accuracy, single engine	≥ 78.4%	about 58.6%	does not meet without the optional routing
MCD	≤ 4.23	4.433, SEM 0.058 over 300 pairs	slightly above target, consistently

What the numbers do and do not support

FID under a disjoint reference is dominated by identity coverage. Real HDTF
video, reframed and featurised exactly as the generations are, scores 31.24 at
the same identity count, so the generated-to-real distance is 0.29. The metric
cannot see generation quality under that protocol.

The paper's FID target is unreachable at this evaluation scale. Scoring real
video alone at 186, 220, 300 and 582 identities gives 20.604, 18.077, 13.287 and
7.577. Real video crosses 17.33 at about 231 identities; this set holds 186. A
generator producing frames indistinguishable from real HDTF would still score
above 17.33 here.

Under the identity-matched reference, quote the distance above the floor, not
the absolute. The reference holds 7,440 frames, well under the 20,000 the
literature gives for a stable covariance estimate. The refine pass costs 1.97 FID
on this protocol; it is kept for temporal stability, not for FID.

The two sides of an FID comparison are not sampled the same way. Evaluation
identities are screened for animation suitability: up to 30 HDTF clips are sampled
per dyad and the first two scoring above 0.8 on detection confidence, frontal pose
and a closed mouth are kept, because Hallo2 needs such a frame to animate at all.
The FID reference is built without that filter. The generated side therefore comes
from an easier slice of HDTF than the reference does, by an amount this repository
has not quantified. DOCS.md records this under Evaluation.

FVD passes its threshold but does not evidence quality. A real-video floor
control puts real held-out footage at 121.60, inside the generated band of
114.77 to 130.12. At this sample size the metric does not separate the two.

Lip-sync depends on the crop and the speech rule. Aligning the crop to the
implementation the paper cites restated the figures from 8.88 / 4.18 to
8.471 / 4.954 across all 186 sides. That is a protocol alignment, not a model
improvement. Speech gating is material: 94% of the apparent gain from removing the
gate is the gate itself, so it stays on.

Optional extensions, off by default

Not part of CHAT. Their numbers are never merged into the rows above.

Extension	Result
Lip-sync post-processing (LatentSync-1.6, guidance 2.5), 141 speaking sides	LSE-D 6.593, LSE-C 8.825 — meets both lip-sync targets; no runner ships here, so this row is not reproducible from this release
Per-class text-to-speech routing (--emo-route), 744 units	emotion accuracy 82.3% — meets the target end to end
SFBG (--sfbg-ckpt --sfbg-strength 0.3), ten-side pilot	−5.44 FID; no full-set figure is published

The two extensions that reach their targets are the honest answer to "can this
repository hit the paper's lip-sync and emotion numbers": yes, but only with
components the paper does not describe, switched on deliberately.

Weights

File	Size	SHA-256
checkpoints/rfbg_qknorm_ft_030500.pt	13,801,971 B	0d328cc8252abf47150530ae5d96636c3eacc97fb0469eb871803585948e4420
checkpoints/sfbg_pre_020000.pt	5,542,293 B	902da86367e652e4eb8ea0325ddb2ac30f66d0cc24dd2e58f505b03ff39261ac

Both are reproduction artefacts trained at reduced scale on substituted corpora,
not the paper's trained models. Each stores its own config, so the architecture is
read from the file rather than assumed.

Notes

Implementation choices, how they relate to the paper's description, and the
protocol behind every metric are in
DOCS.md. What this version
contains is in
CHANGELOG.md.

HDTF, NoXI and REACT 2024 are licensed corpora obtained from their own sources, so
the numbers above cannot be reproduced without them.