Skip to content

v0.3.1

Choose a tag to compare

@quantumCF quantumCF released this 07 Jul 20:30

Fixed

  • Self-consistent source distribution: the sdist now ships the small,
    non-audio test dependencies (scenario labels, manifests, the corpus
    validator, and the deterministic builders under corpus/ and examples/),
    so an extracted sdist collects and runs the full test suite instead of
    hitting collection errors. The heavy real and rendered audio stays pruned;
    suite and class audio is regenerated deterministically by
    tests/conftest.py (seed = sha256(id)) when absent, and the tests that
    depend on genuinely-absent heavy real audio skip cleanly rather than error.
  • Honest fix-plan wording in README.md and docs/WHY.md: reworded the
    claim that every failing event returns a fix that "names the exact setting to
    change in your stack." A plan() may correctly refuse to tune a single
    threshold, report insufficient coverage, or emit a checklist, so the copy now
    says a fix class is always returned and, when the failure maps cleanly to
    stack config, the setting family and direction to investigate.

Added

  • hotato scan self-truncation candidate: a new candidate kind,
    agent_stop_no_caller, surfaces the agent going from active to quiet with
    zero caller energy anywhere nearby, a drop nothing on the caller channel
    explains (not a barge-in, not a caller-driven handoff). Additive; existing
    candidate kinds and scoring are unchanged.
  • Response-gap percentiles and a latency SLA gate: hotato team and
    hotato export now also report p95 (alongside the existing mean/median/p90)
    for talk-over and time-to-yield, and pool response_gap_sec (dead air
    before the agent speaks) into the same distribution shape. --max-response-gap SECONDS on both commands gates the pooled p95 and fails (exit 1) exactly
    when it is exceeded, the same pass/fail contract as --max-talk-over /
    --max-time-to-yield. A plain export's envelope.json stays byte-identical
    to before; the new numbers live only in the printed summary and the
    returned manifest.
  • Echo detector: a deterministic cross-channel coherence signal
    (signals.echo: coherence, lag_sec, echo_suspected) on every scored
    event, flagging when the caller channel looks like a lag-shifted copy of
    the agent's own audio (leaked TTS), so a stop the agent makes because it
    heard itself is no longer indistinguishable from a clean yield. New scan
    candidate kind echo_correlated_activity; a loud WARNING in hotato diagnose and the single-run text output for every echo-suspected yield;
    opt-in --echo-gate on hotato run holds a bleed-induced yield out of the
    verdict (scorable: false) instead of counting it as a pass. Default run
    behavior, and every existing golden verdict, is unchanged; --echo-gate is
    off by default.
  • Resume/restart detector: on events where the agent yielded, an additive
    signals.resume block (resumed, resume_gap_sec, restart_suspected)
    measures from the agent's own VAD track whether it came back after the
    yield, how fast, and whether the post-resume run is long enough to look
    like a restart-from-the-top rather than a short continuation. Whether the
    resumed words repeat the earlier ones is out of scope (a transcript
    question, not a timing one).
  • Four corpus scenario classes under corpus/classes/ (deterministic,
    additive, same seeded-render/--check contract as corpus/suites/):
    mid-utterance-pause, backchannel-multilingual, noise-hold, and
    telephony-degraded. Detail: corpus/classes/README.md.
  • "Is this even a turn-taking bug?" triage in docs/WHY.md and README.md:
    names the failure modes commonly conflated with turn-taking bugs (STT
    hallucination, client-side audio buffering, LLM verbosity/tool-selection,
    safety false-refusal, wrong-language STT) and which tool class to reach for
    instead, alongside a plain statement of the flagship case Hotato covers
    (agent-talks-over-caller and false-stop-on-backchannel).
  • Report accessibility: the generated HTML report carries aria-labels on
    its inline SVGs and audio elements, a main landmark, and measured titles,
    so the artifact is navigable by assistive tech. Visual output is unchanged.
  • Startup lazy-imports: importlib.resources, the report renderer, and the
    numpy accelerator used only inside scan are deferred to first use, so plain
    CLI paths import less at startup. Behavior and output are unchanged.

All of the above are additive optional fields or opt-in flags: did_yield
and passed for every existing fixture are unchanged, and the vendored
src/hotato/_engine is untouched (verify with python3 sync_engine.py --check).