Multi-exemplar voiceprints for speaker matching

[r3dbars]

What

Replaces the single running-average voiceprint per person with a small, bounded set of representative exemplars, and matches a candidate against the best-fitting exemplar instead of one blended-across-conditions average.

Why

A person's clean in-person mic and their compressed remote/Zoom mic produce systematically different embeddings. EMA-blending both into one SpeakerProfile.embedding yields a mediocre centroid that fits neither condition well, depressing match confidence in both. Storing the K most distinct-yet-confirmed session means lets a returning voice match its actual capture condition.

How

• SpeakerExemplarPolicy (new, pure/deterministic): maintains ≤ maxExemplars (3) exemplars via a centroid-distance/diversity heuristic — a same-condition mean (cosine ≥ 0.80 to a representative) EMA-blends into the nearest exemplar (denoise); a distinct condition earns its own slot; at the cap, the most-redundant exemplar is evicted only when the newcomer is strictly more distinct (diversity never decreases).
• SpeakerVectorMath.bestSimilarity(candidate:average:exemplars:): max cosine over {average} ∪ exemplars, skipping dimension-mismatched vectors.
• speaker_exemplars table (SpeakerExemplarStore): batch-loaded into SpeakerProfile.exemplars on read; both matchers (matchAgainstProfiles snapshot path + matchSpeakerImpl) now use bestSimilarity.

Additive & backward-compatible

• SpeakerProfile.exemplars defaults to [] → a legacy single-average profile matches byte-identically (max over the single average). The change only ever adds candidate vectors, so per-profile similarity is monotonically non-decreasing — it cannot lower a true match's score.
• Single-average path (EMA blend, merge, provenance re-derivation) untouched; exemplars are a rebuildable read-side cache on top.
• 192-d/256-d dimension-isolation guard preserved in both matchers.
• Contamination guard preserved: exemplars are written only on a confident/cautious match (write-back alpha > 0) — the same gate that protects the average — so an ambiguous/frozen match can never seed or drift an exemplar.
• Existing match semantics (maturity bonus, separation/ambiguity guard, second-best margin) still operate on the per-profile best similarity.
• Structural edits stay consistent: merge drops both profiles' exemplars; average re-derivation (un-merge / reassign) and profile deletion clear them; they re-accumulate from future confident matches.
• Invisible to the user per the "names just appear" design — model/pipeline layer only, no UI surface.

Validation

• Full Core swift test: 694 tests, 0 failures (12 skipped/timing-sensitive), including:
  • SpeakerNamingSimulationRunnerTests — the in-package deterministic speaker eval harness (false-merge / confusion / identity-stability indicators): no regression.
  • SpeakerDBDimensionIsolationTests, SpeakerMatchingServiceTests, SpeakerEmbeddingMatcherTests, SpeakerProfileMergerTests, SpeakerProfileProvenanceTests: pass.
• New focused tests: SpeakerExemplarPolicyTests (pure policy) + SpeakerMultiExemplarMatchingTests (legacy single-exemplar regression and multi-exemplar benefit through the real DB write→read→match path).
• bash build.sh --no-open (app build + launch smoke) exit 0; source-list guardrail passes; build-deps compiles Core into both the app and SPM archives.

Eval-corpus note

The AMI audio corpus (data/ami/…, gitignored, multi-GB download) is not present locally, so a numeric before/after DER sweep (scripts/run_speaker_eval.sh) was not runnable in this environment. Validation is against the runnable in-package eval (SpeakerNamingSimulationRunner), which exercises the real clusterer + matchAgainstProfiles + naming/write path and reports no regression. A full AMI sweep on a machine with the corpus is the recommended follow-up before broad rollout.

🤖 Generated with Claude Code