You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
hotato contract create/verify/inspect/pack/unpack, the portable failure
contract: turns one real call moment into a self-contained <id>.hotato/
bundle -- contract.json (schema hotato.contract.v1), the (clipped) audio,
frame-level timing evidence, an input-health (trust) report, a shareable SVG
card, a CI policy, and the exact replay/CI commands. create wraps the SAME
round-trip scorability guarantee fixture create gives (via --from-candidate FILE#N, --stereo, --caller+--agent, or the opt-in --mono --diarize path): a not-scorable moment or a mono recording is
refused with the honest reason (exit 2) and no bundle is written; the
diarized-mono path never silently upgrades an indicative-only verdict, and
honestly reports frame-level evidence as unavailable rather than fabricating
it. verify re-scores a contracts directory (or one bundle) against each
contract's own recorded policy and is the CI gate: exit 0 every contract
passes, 1 a regression, 2 a usage error; emits text/JSON/HTML/JUnit. inspect prints one contract. pack/unpack round-trip a bundle through
one deterministic single-file .hotato.pack archive with a sha256 manifest
checked on unpack (a corrupted or tampered archive is refused, exit 2,
nothing partial left behind). hotato card also renders a contract
directly (kind voice-turn-taking-contract). Redacted by default (a
candidate ref / source recording name is hidden unless --include-identifiers); no call ids, paths, or transcript text in any
artifact. New module hotato.contract, new schema schema/contract.v1.json, docs docs/CONTRACTS.md.
hotato trace ingest/attach/export, the voice-trace observability
bridge: ingest --otel FILE --out voice_trace.jsonl parses either a
standard OTel JSON export (resourceSpans, best-effort span/event
flattening) or hotato's own documented OTel bridge JSONL into schema hotato.voice_trace.v1 (caller/agent audio activity, TTS cancel/stop, ASR
partials, tool calls, ...). attach BUNDLE --trace voice_trace.jsonl
writes the trace into <bundle>/traces/voice_trace.jsonl and re-renders evidence/timeline.html with the trace's events drawn as a scale-aligned
row, reading the bundle's OWN evidence/frames.jsonl and contract.json
back in -- it never re-runs the VAD or diarizer, so it works on a
diarized-mono bundle (no frame-level evidence) without the diarization
extra installed, honestly noting the missing base timeline instead of
fabricating one. export BUNDLE --format otel --out FILE writes the
attached trace back out as the same bridge JSONL ingest reads, so ingest -> attach -> export -> ingest round-trips the identical spans.
The evidence page states findings plainly -- "Evidence suggests TTS
cancellation delay: cancel requested at 2.60s, audio stopped at 2.90s
(delta 0.30s)." -- always followed by "Hotato does not prove root cause."
and an explicit "Unknowns: no client-side playout trace was attached."
line. Redacted by default (call id, agent id, ASR transcript text). New
module hotato.trace, new schema schema/voice_trace.v1.json, docs docs/TRACE.md and docs/OTEL.md.
hotato run --mono call.wav --diarize, the opt-in, quality-gated
mono-scorability front-end: a single-channel (mixed) recording -- until now
the hard coverage wall, rejected as not scorable -- becomes scorable by running
an off-the-shelf speaker diarizer over the mono to recover who was active when,
reconstructing two caller/agent tracks, and feeding the EXISTING dual-channel
scorer (zero engine edit). The two-channel path stays the gold reference; this
widens coverage, honestly labeled. A pluggable diarizer-backend seam
(mirroring the neural-VAD seam) ships three backends behind --diarizer: pyannote (local, offline, CPU-viable, default; [diarize]), sortformer
(NVIDIA NeMo, local/GPU, best self-hostable on telephone; [diarize-sortformer]), and pyannoteai (HOSTED, best absolute, requires --egress-opt-in; [diarize-hosted]). A real per-file confidence gate
(extending trust.py's scorability model) reads six signals -- speaker count,
per-speaker activity, mean segmentation posterior, embedding cluster-separation
margin, overlap ratio, segment churn -- into a separation_confidence and a
tier: high scores normally (labeled diarized-mono), low scores but stamps indicative_only and fires NO SLA gate, refuse is not scorable (exit 2).
Caller/agent assignment reuses the floor-dominance heuristic to PROPOSE a
mapping stated as an assumption, overridable with --caller-speaker / --agent-speaker; a balanced mapping downgrades to indicative. On this path signals.echo / crosstalk is definitionally N/A (two slices of one microphone)
and the echo gate never fires. hotato trust --stereo mono.wav --diarize
reports the separability tier (high/low/refuse) WITHOUT scoring. No silent
fallback anywhere: a missing extra / token / model raises a clean BackendUnavailable (exit 2) and NEVER scores raw mono. Honesty invariants:
the default path (no --mono/--diarize) stays byte-identical (a mono file is
still rejected as today), and a diarized-mono verdict is never presented as
equivalent to a true dual-channel measurement. New module hotato.diarize
(DiarizationResult, the backend registry, reconstruct_tracks, separation_confidence, assign_speakers, prepare_diarized_mono, the
pyannote/sortformer/pyannoteai adapters, and a hermetic stub backend); new --mono/--diarize/--diarizer/--caller-speaker/--agent-speaker/ --egress-opt-in flags on hotato run and --diarize/--diarizer on hotato trust; documented in docs/DIARIZE.md. The LIMITS.does_not_do / METHODOLOGY.md "no diarization" framing is superseded: two channels is the
gold reference; mono is scorable via the opt-in, quality-gated front-end,
labeled indicative below the bar. Speaker IDENTIFICATION is still out of scope
(a diarizer assigns anonymous SPEAKER_00/01; it never says who a person is).
New optional extras [diarize], [diarize-sortformer], [diarize-hosted]
(the [diarize] path raises the effective Python floor to >=3.10; the stdlib
core stays >=3.9). Dependency licenses are logged in docs/DIARIZE.md and
carried in the envelope's diarization.licenses block (pyannote-audio MIT;
community-1 weights CC-BY-4.0; Sortformer streaming v2 CC-BY-4.0 -- the offline
v1 is CC-BY-NC and is never shipped).
hotato card INPUT[#REF] --out card.svg, a shareable card from any hotato
result: renders a DETERMINISTIC, stdlib-only, 1200x630 SVG with NO external
resource (no font, image, stylesheet, script, or link; all color inline), so
it drops straight into a PR, an issue, or a slide and looks the same forever.
Four kinds are auto-detected: a talk-over candidate and a false-stop
candidate (from a sweep/analyze candidate ref FILE#N), the threshold-funnel fix plan (decision: do_not_tune_single_threshold -- the
hero card: "NO SINGLE THRESHOLD CAN FIX THIS", fix class: engagement-control),
and a supported verify rollup ("FIX VERIFIED WITHOUT BREAKING
BACKCHANNELS"). Every card names the MEASURED timing moment and never a verdict
about intent, and carries no accuracy number. Redacted by default: a call
id, a filesystem path (only a basename is ever shown), and a vendor recording
name are hidden unless --include-identifiers. Exit 0 written / 2 bad input.
New module hotato.card; three commit-ready assets under docs/assets/cards/ regenerated by scripts/render_card_assets.py; documented
in docs/CARDS.md.
hotato start --demo, the guided, credential-less first run: one command,
no account, no network. It sweeps the two bundled real demo calls, writes the
sweep result (hotato-sweep.json), a self-contained HTML dashboard
(hotato-sweep.html), and the threshold-funnel card
(hotato-no-single-threshold.svg), then turns one real missed-interruption
candidate into a demo failure contract (contracts/demo-missed-interruption.hotato, --expect yield) and verifies it immediately -- it genuinely fails, so the
loop is visible end to end in one command: a real failure becomes a
candidate, becomes a portable contract, and contract verify catches it.
It then prints the exact next commands: promote a candidate into a
permanent fixture, run those fixtures in CI, re-verify the demo contract,
and render a card. The --stack/--folder/--stereo modes are
placeholders in this build and route to hotato sweep/analyze/run.
Exit 0 done / 2 usage. New module hotato.start; documented in docs/START.md.
Fixed
hotato.diarize pyannote 4.x compatibility: Pipeline.from_pretrained
now tries the token= kwarg first and falls back to the removed use_auth_token= name on TypeError, and a new _unpack_pipeline_output
branches on the DiarizeOutput object pyannote.audio >=4.0 returns from a
pipeline call (previously unpacked as a 3.x (Annotation, embeddings)
tuple, raising TypeError/AttributeError against a 4.x install) so both
pyannote 3.x and 4.x load and score cleanly. _embedding_margin no longer
divides by a zero norm and fabricates cos = 0 (read by the confidence
gate as adequate separation) for a degenerate (zero-norm / non-finite)
speaker centroid; it now returns None -- "no margin available" -- the
same honest no-signal result a missing embeddings array already gave.
New yield-boundary confidence gate signal (signals.yield_boundary):
benchmarked against a real pyannote community-1 backend over the AMI
corpus (in-repo harness: tools/bench_diarize/, dev-only, never shipped),
the existing six diarization-quality signals measured clean, well-separated,
stable speakers but were anti-correlated with verdict correctness -- the high confidence tier reproduced the dual-channel did_yield verdict
LESS often than low, concentrated in short-yield, backchannel, and
sub-second talk-over cases, and present even at DER 0.000, because the
verdict turns on a sub-250 ms agent-quiet gap that DER's collar and the
quality signals forgive. The new 7th signal replays the engine's yield
logic directly over the diarization timelines (no model calls, no
reconstruction), perturbs the speaker boundaries by +/- 0.25s, and checks
the verdict survives; a yield resting on a briefer-than-0.5s caller run
(backchannel-grade) or that flips under the boundary nudge is barred from
the high tier (drops to low, indicative-only) even when the other six
signals look clean. Honest high coverage on real material shrinks as a
result -- that is the point: high now requires a boundary-robust
verdict, not just clean diarization. The embedding-margin signal itself
measured uninformative on the AMI benchmark (clustered ~0.43-0.52
regardless of verdict correctness); left as measured, redesign deferred to
a future gate-recalibration stage rather than tuned blind here.