* Add scheduling package: RRULE subset, wall-clock expansion, DST-safe resolution
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* Store scheduled tasks: migration, claims with claim_id fencing, stale-run reclaim, user timezone
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* Agent config scheduling block and session timezone
Agent configs gain a scheduling block (enabled, max_tasks_per_user,
default_timezone) through create, patch and agent.yaml sync, validated
against the IANA zones the router knows. CreateSessionRequest gains a
timezone: it becomes the session's zone, is remembered as the user's
zone for later sessions and tasks unless the session is incognito, and
is refused with 400 invalid_timezone when unknown. When scheduling
tools are offered the prompt says what day it is for the user, so
at/in/repeat resolve against the day the user means.
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* Scheduling tools for the agent: schedule, list, update, cancel
A scheduleRunner in the session's tool chain runs the four tools against
the store when the agent config enables scheduling, the caller is a
verified end user or a backend acting for one, the session is not
incognito and there is a store. A caller that declared a tool under one
of the names keeps it. Refusals (unknown zone, too many tasks, a
schedule that will not parse) come back as tool results the model can
act on, and a session opened without a zone runs in its user's stored
one. Describing the tools has to mind the agent's empty-argument guard,
which reads a tool's description for what it needs.
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* Scheduler worker: claim, deliver, advance, under a lease
A Scheduler on the event forwarder's shape wakes at start, on a kick,
at the next task due and at least once a lease, claims due tasks and
delivers each occurrence under a renewed lease. It computes the next
run from the scheduled occurrence, counts failures and fails the task
after five in a row, delivers a one-shot up to a day late, skips a
repeating task's occurrences older than an hour as missed runs, and
settles an occurrence whose advance was lost without delivering again.
The router wiring lands with the deliverer it is handed, in the api
package.
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* FollowUp takes a caller-chosen command id so a scheduled run is duplicate-safe
conversation.FollowUp now begins with a caller-chosen id: a retried hand-off
answers with the same receipt instead of speaking twice, and the receipt
survives a restart because the conversation rebuilds command records for
assistant messages no user message anchors. The scheduler supplies one id
per run; login, connector consent and thread answers mint their own.
A scheduled-turn marker in routing lets the LLM router charge the customer:
scheduled runs never consume the end user's daily caps, while usage is
still recorded.
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* Deliver scheduled tasks into the user's own conversation
The deliverer runs one claimed occurrence's follow-up in the conversation
its task belongs to: live on this node, followed up in place; live on
another, handed to that node through the peer forwarder, which answers
when the follow-up is accepted, never when the turn ends; ended, reopened
from its row and held until it has answered. A first occurrence with no
session yet opens one on the task's config and records it on the task. A
session on a call is waited out, never interrupted; a busy one is waited
out inside the delivery window.
Every router runs the worker (cmd/router), a deleted session takes its
tasks with it, and the turn is marked scheduled so it is customer spend.
End-to-end: delivery in the scheduling conversation, a first occurrence
opening its session, the mid-call wait, the peer hand-off, session
deletion, and the llmrouter charging rule.
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* Scheduled tasks over REST: /v1/agents/scheduled-tasks
Create (server-side, for the user X-Stream-User-Id names; a reused id
answers 409), list, get, PATCH (title, schedule and next from a device;
instructions server-side), pause, resume counted from now, run by hand
(202, the schedule unmoved), delete (the runs go with it) and the runs
list. A task belongs to its user as user id plus caller kind and is
reached the way a session is (Owner.Reaches): the owner's device sees
the tasks its backend made, anybody else's answer 404. Scheduling
refusals carry their own codes; the spec is regenerated.
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* Go SDK: regenerated client, SessionOptions.Timezone, folder scheduling
The generated client gains the nine scheduled-task operations and the
scheduling blocks. SessionOptions (agents and client) and stream.Call
take a Timezone onto session create. An agent.yaml scheduling: block
syncs as SchedulingSettings, refused as an unknown key without it.
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* Python SDK: regenerated client and folder scheduling
The generated client gains the scheduled-task operations and the
AgentScheduling model. An agent.yaml scheduling: block parses as
SchedulingSettings and syncs onto the request, refused as an unknown
key without it.
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* JS SDK: regenerated types and folder scheduling
The generated types gain the scheduled-task operations. An agent.yaml
scheduling: block parses onto the Declaration and is sent on a sync;
an unknown key, a non-boolean enabled or a fractional cap is refused.
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* Docs: scheduled tasks feature doc, README tables, changelog, SDK note
.factory/features/scheduled-tasks.md records what was asked for and the
decisions; the acceleration README tables gain the endpoints and the
tables; the changelog announces it under Unreleased. The SDK skill note
says which SDKs moved (Go, Python, JavaScript) and which still need
their generated clients (Python done; .NET, Ruby, Rust, PHP pending;
Swift, Kotlin, Dart by filter), and the three moved SDKs' update logs
are stamped.
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* Scheduled tasks: store support for delivery channels
quiet_hours on the task and postponed_until on the run, so a held
occurrence is delivered when the hold ends rather than missed; who
wrote last on a channel delivery, so WhatsApp's 24-hour window can be
counted; an identity found by its user and the chat a reply goes to,
so a channel can be written to first; and a postpone that moves a
claim's lease to the hold's end under its claim id.
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* Scheduled tasks: quiet hours and a held delivery in the worker
QuietEnd reads a start and end the user's clock reads, across midnight
and across a daylight-saving jump, as the minutes after midnight they
are. A delivery that returns Postponed is held rather than failed: the
worker moves the claim's lease to the hold's end and counts the
occurrence's age from there, so a task held overnight is delivered in
the morning and never counted a failure.
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* Channels: Notify, writing to a user first on WhatsApp, SMS or iMessage
The outbound half of answering, on its own: the user's identity on the
channel names where the message goes, the agent's line carries it, the
dlc gate refuses who opted out and counts a sandboxed app's message,
and WhatsApp is only written inside the day after the person last
wrote, counted from when their messages arrived.
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* Scheduled tasks: deliver along the task's channels, by phone too
DeliverScheduledTask walks the task's channels in order and stops at
the first that lands. The conversation answers on acceptance as
before; a text channel runs the same follow-up, reads what the agent
answers once it settles, and carries those words out with
channels.Notify; phone places a call the way a campaign rings, on a
session opened over the task's conversation, and counts the agent
speaking as the delivery, no answer within the ring as none. Quiet
hours hold the occurrence through the scheduler's Postponed, on every
channel, never for a run made by hand. The peer hand-off now walks
the chain on the node holding the conversation and answers where it
landed.
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* Scheduled tasks: deliver_to and quiet_hours on the REST API
Create and update take where an occurrence is delivered, in order,
and the part of the user's day one is held through; a channel nothing
carries and a hold that does not read are refused on the write, not at
three in the morning. The task reads both back, and a run's
delivered_to names the channel it landed on.
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* Scheduled tasks: Phase 2 delivery tests
A task texted on WhatsApp arrives there with the words the agent
answered, one outside WhatsApp's window lands in the conversation
instead, a call this router cannot place falls back to the text, a
phone-only task fails saying so, quiet hours hold a task with its run
still running and nothing delivered, and deliver_to with quiet_hours
read back, change, and are refused when they do not read.
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* Scheduled tasks: an SMS delivery test
A task whose only channel is sms is texted there with the words the
agent answered, over a line seeded the way buying the number and
connecting it leaves it.
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* SDKs: regenerated clients for deliver_to and quiet_hours
The scheduled-task create, update and read models gain deliver_to and
quiet_hours, and the run's delivered_to names the text channels and
phone. Go, Python and JavaScript clients regenerated; the skill note
says which SDKs still need it.
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* Docs: scheduled tasks delivered on channels, held overnight
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* gofmt: realign the fields Phase 2 added
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* gofmt: one spacing nit in the scheduling tools
Co-authored-by: Neevash Ramdial (Nash) <mail@neevash.com>
* AI-1517: Scheduled tasks: base merge, one migration, API fixes (#1054)
* [AI-1431] Refuse live STT pinned to a batch-only model (#1025)
* AI-1431: refuse a live STT session pinned to a batch-only model
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1431: refuse a batch-only STT target once at agent join, and on recorded shortcuts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1433] Open no STT session for a participant who left (#1028)
* AI-1433: Open no transcription session for a participant who left
Also drop muse's dead SetWriteDeadline check (AI-1434) and test that a
session drops audio before its fatal error reaches the caller (AI-1435).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1433: Let audio after a leave reopen a session once a second has passed
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1433: Pin the leave grace to a second from the first dropped frame
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1433: Soften the departed-grace comment, rename refuses to dropsDeparted
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Tools can show the model pictures (#1011)
* [AI-1437] Keep speaker labels across a voice switch (#1026)
* AI-1437: keep speaker labels across a voice switch, number same names
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1437: Cover the swap at session level, upgrade id labels to names
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1437: Test an unnamed person speaking twice, drop the redundant label guard
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1437: Test that a given display name stays when the person renames
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1439] Name speakers to Python LLMs, bound STT start (#1029)
* AI-1439: Name the speaker to Python LLM plugins, bound stream.STT start (AI-1440)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1439: Name the speaker in stream.LLM, type recorded bodies, cite STT bounds
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1439: Put the openai_api fixture back in each test file, a second conftest broke CI imports
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1462] Record speech with Cartesia (#1031)
* AI-1462: Record speech with Cartesia on /tts/bytes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1462: Record on a Cartesia batch model, lowercase the language, test the refusals
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1462: Say a Cartesia refusal fails the job, not a 400
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1462: Note Cartesia's batch entry in the router-tts skill
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1462: Test that a recording's format reaches Cartesia's batch model
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1460] Answer a turn whose decision call timed out (#1032)
* AI-1460: Rule on a turn whose decision call hit its deadline
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1460: Check the timed-out turn's words are stored
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1455] Refuse batch-only STT at config save; close 1008 (#1034)
* AI-1455: Refuse a batch-only STT when a voice config is saved; close 1008 on request errors
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1455: Note the save-time STT check and the 1008 close in the changelog and router-stt skill
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1455: Close 1008 on an unknown target, test the no-STT-router guard
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1459] Test named history at six more plugin sites (#1035)
* AI-1459: Test the named history at six more plugin sites
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1459: Close the plugin client in the fixture teardown
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
* [AI-1458] Label a person with no name Speaker N, not their id (#1033)
* AI-1458: Label a person with no display name Speaker N, never an id
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1458: Test flow names of named people and a person named like a fallback label
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1461] Tell Python LLMs when several people are talking (#1036)
* AI-1461: Tell Python LLMs when several people are talking, label the nameless Speaker N
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1461: Tell the model about several people on an eager second-person turn
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1461: Build each Gemini request config on a copy of the base config
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1461: Test the several-people line in Claude requests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1461: Test the several-people line on a Bedrock tool-call follow-up
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1461: Test the several-people line in a HuggingFace input request
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1461: Test the several-people line in a new xAI chat
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1461: Test a person whose name matches their own fallback label
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1461: Format the Bedrock and xAI tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(android): The Kotlin SDK is one module, as Swift is one package (#1005)
* refactor(sdk-kotlin): one module, as Swift is one package
core, chat, rtc and ui become one Android library, vision-agents-core,
in io.getstream.visionagents.android.core. VoiceSession and
session.chat() move into it; the SDK's own Compose views go, since apps
compose Stream Chat's AI components. ChatFailed and CallFailed join
AgentsException, and the shared Stream client plumbing is internal now
that one module uses it.
Refs AND-1669
* docs(sdk-kotlin): the README, skills and changelog for one module
Refs AND-1669
* docs(agents): Kotlin is one module in the repo map
Refs AND-1669
* feat(android): an Android demo, the Android half of swift_demo: Chat (#1018)
* feat(android-demo): the Android half of swift_demo, its Chat screen
Refs AND-1674
* feat(android): the Android demo's Voice tab (#1038)
* fix(sdk-kotlin): a voice call asks only for the microphone
Refs AND-1674
* feat(android-demo): the Voice tab
Refs AND-1674
* feat(android-demo): ask for the local network on Android 17
Refs AND-1674
* fix(android-demo): the voice transcript keeps its newest line clear of the controls
Refs AND-1674
* [AI-1472] First-class role field on agent configs (#1041)
* AI-1472: Add a first-class role field to agent configs
The dashboard marked Nora with a free-form tag, tags.role=ai_moderator,
which nothing validated and nothing kept unique. agent_configs now has a
role column, with ai_moderator as its one value, so the chat worker and
the dashboard can find the app's moderator reliably.
- Migration adds the column, backfills the newest live config each app
tagged role=ai_moderator, and adds a partial unique index: one live,
non-test-copy config per app holds a role.
- The API takes role on create, update, patch and sync, refuses a second
holder with a 409 role_taken, and keeps the stored role when a save
leaves it out. listAgentConfigs takes ?role= to find the moderator.
- The Go SDK reads role: from agent.yaml (Settings.Role) and sends it
on sync. Go, JavaScript and Python clients are regenerated; the other
SDKs are noted in the sdk skill.
* AI-1472: Test that agent.yaml's role reaches the sync
* AI-1472: Name the role agent_moderator instead of ai_moderator
The backfill still matches the dashboard's existing tag, role=ai_moderator,
and stores it as agent_moderator.
* AI-1472: Backfill from the role=agent_moderator tag
* AI-1471: Run an agent config to completion over HTTP (#1039)
POST /v1/agents/runs runs an agent config as an incognito headless text
session on one input, waits for it to settle and answers with the agent's
last reply, the session id, the model route and what its model calls spent.
Server-side only.
The per-run instructions are added to the config's; the input is handed to
the agent as JSON data. Failures have their own codes: invalid_config (400),
budget_exceeded (429), model_failed (503) and a new timeout type (504).
settle moves from mcpevents to session.Session.Settle so both callers share
it, and llm.CallTiming now carries each model call's tokens and cost.
* [AI-1477] Tag and meter headless runs per app and product (#1042)
* AI-1471: Run an agent config to completion over HTTP
POST /v1/agents/runs runs an agent config as an incognito headless text
session on one input, waits for it to settle and answers with the agent's
last reply, the session id, the model route and what its model calls spent.
Server-side only.
The per-run instructions are added to the config's; the input is handed to
the agent as JSON data. Failures have their own codes: invalid_config (400),
budget_exceeded (429), model_failed (503) and a new timeout type (504).
settle moves from mcpevents to session.Session.Settle so both callers share
it, and llm.CallTiming now carries each model call's tokens and cost.
* AI-1477: Prove a run's tags reach its spend breakdowns
A headless run carries the caller's tags onto every request its model
calls make, so chat can tag each Nora review with product=moderation,
feature=agent_review, app and review_item and read the cost back per app
and per item from the tag breakdowns. The run's usage was already in the
response from AI-1471.
The new test runs a config with those four tags and checks that
/v1/stats/spend grouped by each of them reports the run's cost under the
tag's value.
* AI-1478: Cap each app's headless runs in flight (#1043)
POST /v1/agents/runs now holds a slot per app while a run is in flight,
counted in Redis so every router shares the count. Each slot is a lease
that ends at the run's timeout plus a grace, so a router that dies mid-run
cannot hold a slot forever. The cap is runs.concurrent_per_app
(ROUTER_RUNS_CONCURRENT_PER_APP), 20 by default; 0 or no Redis caps nothing.
A run over the cap is a 429 concurrency_limited with Retry-After, so the
chat worker backs off and retries. A spent budget moves from 429 to 403
budget_exceeded: waiting does not refill it, so it must not be retried.
* [AI-1479] Nora agent template and run-time output contract (#1044)
* AI-1479: Nora agent template and the run-time output contract
Adds acceleration/templates/agent_moderator, the starting point for an
app's Nora, ported from the GetStream/nora PoC: a text agent on
llm-thinking with role agent_moderator, reviewer instructions (content is
untrusted data, flags are hypotheses, judge the current content first,
quote evidence exactly) and a skill each for profanity, harassment,
suggestive language and actions.
The JSON answer format the chat review worker parses (per-case, per-label
judgments with evidence, a flag assessment and summaries) lives apart from
the editable template, in internal/moderator/contract.md. createRun adds
it after the config's and the caller's instructions on every run of an
agent_moderator config, so a customer edit cannot break parsing. Sessions
such as Ask Nora don't get it.
Also restores a closing brace in runs_test.go that #1043 dropped, which
kept the integration-tagged api package from compiling.
* AI-1479: Say where cases and labels come from in the answer format
The contract now names the input's cases array and that a label is the
name the policy gives it, so the chat worker's bundle and validator have
one shape to match. The moderator run test also checks that the caller's
policy is present before comparing positions.
* Clean voice transcripts, hedged flow decisions (#1040)
* fix(agent): record lines said as written in the transcript
A line the agent says without the model (Say, and so the greeting) went
straight to the voice with no reply announced. The voice transcript
stores a turn's text when its voice finishes, and only if it was told
the text first, so the Chat channel and responses.items never had it,
and the caller's answer to a greeting's question reached the model and
the flow controller with no question.
Say now announces the line as a reply before speaking it and keeps it
for the model: in the history part way through a conversation, and in
the instructions when it opens the call, since chat templates that need
a user turn first refuse a conversation opening on the assistant's. A
line the voice could not say is taken back. Conversations are not named
after their greeting.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(chatlog): mark the agent's spoken replies ai_generated
A conversation held in writing sets ai_generated on the assistant's
replies, which is how Stream's AI components tell them from the rest.
The voice transcript never did, so a call's Chat channel rendered the
agent's spoken replies as ordinary messages. Every line the agent
writes there now carries it; what participants said does not.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(agent): hear only the new words of a run kept open
A transcriber that keeps one run of speech open across an answer
(Deepgram Flux resuming a turn it already ended) sends the whole run
each time. Cadence passed it on as said, so every later turn repeated
the earlier ones to the model, the flow controller, Heard and the
transcript, more each turn.
A candidate now records the words of its numbered run the agent already
answered, and Said is the rest. Turn-taking still compares the whole
run; what the caller is taken to have said (Heard, the reply, the
history, the flow question, the monologue check) is Said. Words added
to something being read out stay whole: a number, a number said as
words, or letters spelled out.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(harness): cap a stalled flow decision on a settled turn at 1 s
A finished turn the flow controller does not rule on is answered all
the same, so waiting out the 3 s flow deadline for it only adds
silence. On hosted calls the controller stalled on its first token on
most turns, and every one of them cost the caller the full 3 s before
an answer it would have got anyway.
A turn the transcriber marked as ended (EndMarked, which unlike Ended
is not set for every turn of a transcriber that never marks ends), or
that was already waited on, from the caller to a quiet agent, now gets
1 s of the controller, counted from when it is asked rather than from
when it joined the queue. Turns not yet ended, turns in another voice
and turns spoken over the agent keep their deadlines.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(changelog): reference #1040 in its entries
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* revert(harness): drop the 1 s cap on a settled turn's flow decision
Run against a local router with Deepgram Flux and a stalled flow model,
the cap never applied: Flux sends its final transcript 1-8 s after the
caller stops, and the controller is asked 350 ms after the words hold
still, so a turn is asked about before anything has marked it ended.
Every turn still waited out the 3 s deadline. The flow controller is
hedged instead, in the next commit.
* feat(llmrouter): hedge a flow decision that has said nothing
The flow controller's decision is a few tokens the caller waits on in
silence, before the reply is even asked for. When the flow model
stalled on its first token, every turn waited out the controller's 3 s
deadline and was then answered with no ruling.
A flow decision that has said nothing for ROUTER_FLOW_HEDGE (400ms by
default, 0 turns it off, cmd/agent -flow-hedge) is now asked of another
candidate of llm-flow as well, the way a late reply already is under
ROUTER_REPLY_HEDGE, and whichever says something first rules on the
turn. It is a setting of its own because a decision that is late is
stalled far sooner than a reply that is.
Against a local router with Deepgram Flux and a flow model that never
sends a token, the median turn decision went from 3.0 s to 1.5 s and
voice-in to voice-out from 4.9 s to 3.7 s, each turn ruled on by the
hedge.
* fix(agent): take a lone I or a as a word, not a letter being spelled
The rule that keeps something read out a piece at a time whole took any
one-letter last word as a letter being spelled, so after a turn
answered on "It came on gradually I" the next one carried the whole run
again. A letter is now only taken as spelled when a letter or a number
comes before it, as in "A B C".
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1463] Record speech with Inworld (#1037)
* AI-1463: Record speech with Inworld on /tts/v1/voice
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1463: Say temperature is sent in the router-tts skill
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1464] Settle the caller's turn before an STS reply opens (#1047)
* AI-1464: settle the caller's turn before a reply opens on its words
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1464: Drop a cut-off reply's remainder before it ends the caller's turn
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-891] fix(router): a phone session ends after the caller hangs up or nobody answers (#1045)
* Merge the scheduled-task migrations into one at 20261103150000
Both branch migrations (20261103120000, which collides with accelerate's
agent_config_role, and 20261104120000) become one file newer than
accelerate's newest. quiet_hours and postponed_until move into the CREATE
TABLEs; channel_deliveries (account_id, from_address, received_at DESC) and
scheduled_tasks (customer_id, origin_session_id) get indexes (N14). B4.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Docs: scheduled-task SDK note, feature status, CORS PATCH changelog
Q1: Python is regenerated, so it leaves the to-do list in the sdk skill.
N16: the feature table names the delivery channels and quiet hours.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Fix the scheduled-task API surface found in review
- B1: a device naming no user lists no tasks, as session lists do.
- B3: GET renders no scheduling block for a config without one, and no
max_tasks_per_user when unset, so a PUT of the GET body is not 400.
- N5: PUT merges a scheduling block field by field, as its doc says;
list state is an enum tag; empty or blank instructions are refused on
create and PATCH.
- S12: a blank title is a 400, not a 500.
- Q6: a create naming a session that keeps no conversation (incognito,
or a row without one) is refused.
- N2/N4: unused codes dropped, invalid_schedule documented,
errTooManyScheduledTasks is a value.
- Q10: CORS allows PATCH, checked against every method the spec serves.
- Go SDK: created_by names its enum constants, so AgentLogSource keeps
Agent; Swift: drop timestamp examples that YAML reads as datetimes.
- N7: Python client is accelerate's plus only the scheduling diff.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1466] Report a cancelled model call as cancelled (#1049)
* AI-1466: Report a cancelled model call as cancelled, not as success
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1466: Log a cancelled request as info in the single-database trigger
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1517: Regenerate Dart client, type the scheduling block apart
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* [AI-1465] Fail a recording over to the next model (#1048)
* AI-1465: Fail a recording over to the next model when the vendor fails it
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1465: Walk only the candidates Select would start, from one resolution
Router.Walk runs Select's filters, ranking and overwrite rule once and hands each started candidate to the caller; Select is Walk with a use that keeps the provider. Record walks it, so a voice, a policy or an overwrite that rules a candidate out keeps it out, and health read during routing cannot skip or rebuild one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1465: Describe the recording walk as Router.Walk in the router-tts skill
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1465: Leave an unbuilt candidate out of the walk's met errors
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1517: Put the phone-session bullet back under Bug Fixes
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Martin Mitrevski <martinmitrevski.oh@gmail.com>
Co-authored-by: Aleksandar Apostolov <apostolov.alexandar@gmail.com>
Co-authored-by: Darko <darkoatanasovski@gmail.com>
Co-authored-by: Oliver Lazoroski <oliver.lazoroski@gmail.com>
Co-authored-by: Ali Aliyev <ali.aliyev@getstream.io>
* AI-1523: Schedule time resolution fixes (#1058)
* AI-1523: Keep named instants, calendar-day in, future REST repeats
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1523: test the start-instant guard, bound in, align the schedule docs
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1523: test the in-bound at exactly 366 days
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1519: Scheduled turns are not counted on the user's day (#1057)
* Keep a scheduled turn's mark through the agent's turn
DeliverScheduledTask marked its context with routing.WithScheduledTurn,
but Agent.RespondTo kept only the sender and generate() asks the model on
the agent's own context, so llmrouter's charged() never saw the mark and
a scheduled follow-up was counted on the end user's day. The agent now
keeps the mark with the turn, as it keeps the sender, and the replies
that finish the turn carry it too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1519: clear the scheduled mark when a speculation is adopted
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1521: Scheduled worker stops on lost claim, bounded, drains (#1059)
* Stop a scheduled delivery when its claim is lost, bound it, drain the worker
A renewal that finds the claim gone cancels the delivery and its retries;
renewals now work at all (now() + bigint failed). One occurrence holds a
slot for at most 20 minutes. Start takes a ctx: claims stop at the stop
signal and the drain waits for deliveries in flight. Replaying a missed
run keeps the failure streak.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1521: test that the drain counts a busy scheduler and a lost claim leaves its run alone
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1521: build the drain test's cleanup
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1521: wait for the recurring-skip test's own delivery and runs
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1518: One reply per scheduled run, task text escaped (#1062)
* Capture a scheduled run's reply once and frame its task safely
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1518: escape the title, read a retried follow-up's reply back, cover read-back and abandon
R1.1 title through taskText; R1.2 commandsIn keeps a follow-up's latest reply;
R1.3 read-back tests (failed reply, reply still being written); R1.4 abandoned
follow-up test on a real session.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1525: Scheduling tools and prompt: move floor, list order (#1061)
* AI-1525: Scheduling tools: move floor, backend instructions, list order, clock last
- S2: a snooze and PATCH next land 15 minutes from now at the soonest
(scheduling.MinMove = MinInterval), refused with interval_too_short.
- S8: update_scheduled_task refuses to rewrite the instructions of a task a
backend made, as the API refuses a device.
- S23: list_scheduled_tasks lists active, paused, failed, then completed
tasks, at most as many as the user may run, and says when there are more.
- S25: the clock line goes after the instructions, so the sessions of one
config share the cached prompt prefix.
- N1: prompt()'s doc comment sits on prompt() again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1525: bound reschedules, allow snoozes past the next run, test the list limit
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1525: regenerate the spec and the clients for the move rule
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1525: Test a next moved with the schedule in one update
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1539] Router: hide internal connectors from apps not marked internal (#1069)
* [AI-1539] feat(router): hide internal connectors from apps not marked internal
A manifest may say internal: true, and an app's settings may say
internal_connectors. One store helper, visibleTo, now decides which
definitions an app sees for every read that repeated the built-ins-or-own
rule, and hides internal ones from apps without the flag. No shipped
built-in is internal, so nothing changes for existing apps.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1539] Derive the control catalog from the manifests, scope the flag values, document internal: true
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1516: Merge accelerate into the scheduled-tasks branch (#1071)
* [AI-1431] Refuse live STT pinned to a batch-only model (#1025)
* AI-1431: refuse a live STT session pinned to a batch-only model
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1431: refuse a batch-only STT target once at agent join, and on recorded shortcuts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1433] Open no STT session for a participant who left (#1028)
* AI-1433: Open no transcription session for a participant who left
Also drop muse's dead SetWriteDeadline check (AI-1434) and test that a
session drops audio before its fatal error reaches the caller (AI-1435).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1433: Let audio after a leave reopen a session once a second has passed
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1433: Pin the leave grace to a second from the first dropped frame
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1433: Soften the departed-grace comment, rename refuses to dropsDeparted
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Tools can show the model pictures (#1011)
* [AI-1437] Keep speaker labels across a voice switch (#1026)
* AI-1437: keep speaker labels across a voice switch, number same names
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1437: Cover the swap at session level, upgrade id labels to names
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1437: Test an unnamed person speaking twice, drop the redundant label guard
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1437: Test that a given display name stays when the person renames
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1439] Name speakers to Python LLMs, bound STT start (#1029)
* AI-1439: Name the speaker to Python LLM plugins, bound stream.STT start (AI-1440)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1439: Name the speaker in stream.LLM, type recorded bodies, cite STT bounds
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1439: Put the openai_api fixture back in each test file, a second conftest broke CI imports
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1462] Record speech with Cartesia (#1031)
* AI-1462: Record speech with Cartesia on /tts/bytes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1462: Record on a Cartesia batch model, lowercase the language, test the refusals
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1462: Say a Cartesia refusal fails the job, not a 400
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1462: Note Cartesia's batch entry in the router-tts skill
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1462: Test that a recording's format reaches Cartesia's batch model
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1460] Answer a turn whose decision call timed out (#1032)
* AI-1460: Rule on a turn whose decision call hit its deadline
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1460: Check the timed-out turn's words are stored
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1455] Refuse batch-only STT at config save; close 1008 (#1034)
* AI-1455: Refuse a batch-only STT when a voice config is saved; close 1008 on request errors
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1455: Note the save-time STT check and the 1008 close in the changelog and router-stt skill
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1455: Close 1008 on an unknown target, test the no-STT-router guard
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1459] Test named history at six more plugin sites (#1035)
* AI-1459: Test the named history at six more plugin sites
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1459: Close the plugin client in the fixture teardown
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
* [AI-1458] Label a person with no name Speaker N, not their id (#1033)
* AI-1458: Label a person with no display name Speaker N, never an id
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1458: Test flow names of named people and a person named like a fallback label
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1461] Tell Python LLMs when several people are talking (#1036)
* AI-1461: Tell Python LLMs when several people are talking, label the nameless Speaker N
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1461: Tell the model about several people on an eager second-person turn
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1461: Build each Gemini request config on a copy of the base config
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1461: Test the several-people line in Claude requests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1461: Test the several-people line on a Bedrock tool-call follow-up
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1461: Test the several-people line in a HuggingFace input request
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1461: Test the several-people line in a new xAI chat
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1461: Test a person whose name matches their own fallback label
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1461: Format the Bedrock and xAI tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(android): The Kotlin SDK is one module, as Swift is one package (#1005)
* refactor(sdk-kotlin): one module, as Swift is one package
core, chat, rtc and ui become one Android library, vision-agents-core,
in io.getstream.visionagents.android.core. VoiceSession and
session.chat() move into it; the SDK's own Compose views go, since apps
compose Stream Chat's AI components. ChatFailed and CallFailed join
AgentsException, and the shared Stream client plumbing is internal now
that one module uses it.
Refs AND-1669
* docs(sdk-kotlin): the README, skills and changelog for one module
Refs AND-1669
* docs(agents): Kotlin is one module in the repo map
Refs AND-1669
* feat(android): an Android demo, the Android half of swift_demo: Chat (#1018)
* feat(android-demo): the Android half of swift_demo, its Chat screen
Refs AND-1674
* feat(android): the Android demo's Voice tab (#1038)
* fix(sdk-kotlin): a voice call asks only for the microphone
Refs AND-1674
* feat(android-demo): the Voice tab
Refs AND-1674
* feat(android-demo): ask for the local network on Android 17
Refs AND-1674
* fix(android-demo): the voice transcript keeps its newest line clear of the controls
Refs AND-1674
* [AI-1472] First-class role field on agent configs (#1041)
* AI-1472: Add a first-class role field to agent configs
The dashboard marked Nora with a free-form tag, tags.role=ai_moderator,
which nothing validated and nothing kept unique. agent_configs now has a
role column, with ai_moderator as its one value, so the chat worker and
the dashboard can find the app's moderator reliably.
- Migration adds the column, backfills the newest live config each app
tagged role=ai_moderator, and adds a partial unique index: one live,
non-test-copy config per app holds a role.
- The API takes role on create, update, patch and sync, refuses a second
holder with a 409 role_taken, and keeps the stored role when a save
leaves it out. listAgentConfigs takes ?role= to find the moderator.
- The Go SDK reads role: from agent.yaml (Settings.Role) and sends it
on sync. Go, JavaScript and Python clients are regenerated; the other
SDKs are noted in the sdk skill.
* AI-1472: Test that agent.yaml's role reaches the sync
* AI-1472: Name the role agent_moderator instead of ai_moderator
The backfill still matches the dashboard's existing tag, role=ai_moderator,
and stores it as agent_moderator.
* AI-1472: Backfill from the role=agent_moderator tag
* AI-1471: Run an agent config to completion over HTTP (#1039)
POST /v1/agents/runs runs an agent config as an incognito headless text
session on one input, waits for it to settle and answers with the agent's
last reply, the session id, the model route and what its model calls spent.
Server-side only.
The per-run instructions are added to the config's; the input is handed to
the agent as JSON data. Failures have their own codes: invalid_config (400),
budget_exceeded (429), model_failed (503) and a new timeout type (504).
settle moves from mcpevents to session.Session.Settle so both callers share
it, and llm.CallTiming now carries each model call's tokens and cost.
* [AI-1477] Tag and meter headless runs per app and product (#1042)
* AI-1471: Run an agent config to completion over HTTP
POST /v1/agents/runs runs an agent config as an incognito headless text
session on one input, waits for it to settle and answers with the agent's
last reply, the session id, the model route and what its model calls spent.
Server-side only.
The per-run instructions are added to the config's; the input is handed to
the agent as JSON data. Failures have their own codes: invalid_config (400),
budget_exceeded (429), model_failed (503) and a new timeout type (504).
settle moves from mcpevents to session.Session.Settle so both callers share
it, and llm.CallTiming now carries each model call's tokens and cost.
* AI-1477: Prove a run's tags reach its spend breakdowns
A headless run carries the caller's tags onto every request its model
calls make, so chat can tag each Nora review with product=moderation,
feature=agent_review, app and review_item and read the cost back per app
and per item from the tag breakdowns. The run's usage was already in the
response from AI-1471.
The new test runs a config with those four tags and checks that
/v1/stats/spend grouped by each of them reports the run's cost under the
tag's value.
* AI-1478: Cap each app's headless runs in flight (#1043)
POST /v1/agents/runs now holds a slot per app while a run is in flight,
counted in Redis so every router shares the count. Each slot is a lease
that ends at the run's timeout plus a grace, so a router that dies mid-run
cannot hold a slot forever. The cap is runs.concurrent_per_app
(ROUTER_RUNS_CONCURRENT_PER_APP), 20 by default; 0 or no Redis caps nothing.
A run over the cap is a 429 concurrency_limited with Retry-After, so the
chat worker backs off and retries. A spent budget moves from 429 to 403
budget_exceeded: waiting does not refill it, so it must not be retried.
* [AI-1479] Nora agent template and run-time output contract (#1044)
* AI-1479: Nora agent template and the run-time output contract
Adds acceleration/templates/agent_moderator, the starting point for an
app's Nora, ported from the GetStream/nora PoC: a text agent on
llm-thinking with role agent_moderator, reviewer instructions (content is
untrusted data, flags are hypotheses, judge the current content first,
quote evidence exactly) and a skill each for profanity, harassment,
suggestive language and actions.
The JSON answer format the chat review worker parses (per-case, per-label
judgments with evidence, a flag assessment and summaries) lives apart from
the editable template, in internal/moderator/contract.md. createRun adds
it after the config's and the caller's instructions on every run of an
agent_moderator config, so a customer edit cannot break parsing. Sessions
such as Ask Nora don't get it.
Also restores a closing brace in runs_test.go that #1043 dropped, which
kept the integration-tagged api package from compiling.
* AI-1479: Say where cases and labels come from in the answer format
The contract now names the input's cases array and that a label is the
name the policy gives it, so the chat worker's bundle and validator have
one shape to match. The moderator run test also checks that the caller's
policy is present before comparing positions.
* Clean voice transcripts, hedged flow decisions (#1040)
* fix(agent): record lines said as written in the transcript
A line the agent says without the model (Say, and so the greeting) went
straight to the voice with no reply announced. The voice transcript
stores a turn's text when its voice finishes, and only if it was told
the text first, so the Chat channel and responses.items never had it,
and the caller's answer to a greeting's question reached the model and
the flow controller with no question.
Say now announces the line as a reply before speaking it and keeps it
for the model: in the history part way through a conversation, and in
the instructions when it opens the call, since chat templates that need
a user turn first refuse a conversation opening on the assistant's. A
line the voice could not say is taken back. Conversations are not named
after their greeting.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(chatlog): mark the agent's spoken replies ai_generated
A conversation held in writing sets ai_generated on the assistant's
replies, which is how Stream's AI components tell them from the rest.
The voice transcript never did, so a call's Chat channel rendered the
agent's spoken replies as ordinary messages. Every line the agent
writes there now carries it; what participants said does not.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(agent): hear only the new words of a run kept open
A transcriber that keeps one run of speech open across an answer
(Deepgram Flux resuming a turn it already ended) sends the whole run
each time. Cadence passed it on as said, so every later turn repeated
the earlier ones to the model, the flow controller, Heard and the
transcript, more each turn.
A candidate now records the words of its numbered run the agent already
answered, and Said is the rest. Turn-taking still compares the whole
run; what the caller is taken to have said (Heard, the reply, the
history, the flow question, the monologue check) is Said. Words added
to something being read out stay whole: a number, a number said as
words, or letters spelled out.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(harness): cap a stalled flow decision on a settled turn at 1 s
A finished turn the flow controller does not rule on is answered all
the same, so waiting out the 3 s flow deadline for it only adds
silence. On hosted calls the controller stalled on its first token on
most turns, and every one of them cost the caller the full 3 s before
an answer it would have got anyway.
A turn the transcriber marked as ended (EndMarked, which unlike Ended
is not set for every turn of a transcriber that never marks ends), or
that was already waited on, from the caller to a quiet agent, now gets
1 s of the controller, counted from when it is asked rather than from
when it joined the queue. Turns not yet ended, turns in another voice
and turns spoken over the agent keep their deadlines.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(changelog): reference #1040 in its entries
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* revert(harness): drop the 1 s cap on a settled turn's flow decision
Run against a local router with Deepgram Flux and a stalled flow model,
the cap never applied: Flux sends its final transcript 1-8 s after the
caller stops, and the controller is asked 350 ms after the words hold
still, so a turn is asked about before anything has marked it ended.
Every turn still waited out the 3 s deadline. The flow controller is
hedged instead, in the next commit.
* feat(llmrouter): hedge a flow decision that has said nothing
The flow controller's decision is a few tokens the caller waits on in
silence, before the reply is even asked for. When the flow model
stalled on its first token, every turn waited out the controller's 3 s
deadline and was then answered with no ruling.
A flow decision that has said nothing for ROUTER_FLOW_HEDGE (400ms by
default, 0 turns it off, cmd/agent -flow-hedge) is now asked of another
candidate of llm-flow as well, the way a late reply already is under
ROUTER_REPLY_HEDGE, and whichever says something first rules on the
turn. It is a setting of its own because a decision that is late is
stalled far sooner than a reply that is.
Against a local router with Deepgram Flux and a flow model that never
sends a token, the median turn decision went from 3.0 s to 1.5 s and
voice-in to voice-out from 4.9 s to 3.7 s, each turn ruled on by the
hedge.
* fix(agent): take a lone I or a as a word, not a letter being spelled
The rule that keeps something read out a piece at a time whole took any
one-letter last word as a letter being spelled, so after a turn
answered on "It came on gradually I" the next one carried the whole run
again. A letter is now only taken as spelled when a letter or a number
comes before it, as in "A B C".
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1463] Record speech with Inworld (#1037)
* AI-1463: Record speech with Inworld on /tts/v1/voice
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1463: Say temperature is sent in the router-tts skill
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1464] Settle the caller's turn before an STS reply opens (#1047)
* AI-1464: settle the caller's turn before a reply opens on its words
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1464: Drop a cut-off reply's remainder before it ends the caller's turn
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-891] fix(router): a phone session ends after the caller hangs up or nobody answers (#1045)
* [AI-1466] Report a cancelled model call as cancelled (#1049)
* AI-1466: Report a cancelled model call as cancelled, not as success
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1466: Log a cancelled request as info in the single-database trigger
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1465] Fail a recording over to the next model (#1048)
* AI-1465: Fail a recording over to the next model when the vendor fails it
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1465: Walk only the candidates Select would start, from one resolution
Router.Walk runs Select's filters, ranking and overwrite rule once and hands each started candidate to the caller; Select is Walk with a use that keeps the provider. Record walks it, so a voice, a policy or an overwrite that rules a candidate out keeps it out, and health read during routing cannot skip or rebuild one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1465: Describe the recording walk as Router.Walk in the router-tts skill
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1465: Leave an unbuilt candidate out of the walk's met errors
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* AI-816: report plugin use to Sentry (#1055)
* feat(plugins): report every deprecated plugin use to Sentry
Each of the 10 sites that logged plugins.DeprecatedUse now calls
plugins.ReportUse, which logs the same warn line and captures a Sentry
warning fingerprinted by path (and via), with the line's attrs as tags.
Pod logs live about an hour; Sentry keeps the history AI-816 phase 2
needs to prove the plugin system unused before deleting it.
The event carries no request: the login callback's query holds the
OAuth code and state, which Sentry's default scrubbing keeps.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-816: Cut a Sentry tag value to the 200 characters it keeps
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-816: Pin the Sentry tag limit to 200 in its test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1534][AI-1535] Time STS stops from provider end of speech (#1060)
* AI-1534: Time STS stops from the provider's own end of speech
Gemini places the stop at ACTIVITY_END's audioOffset; openairealtime and
the native agent give a stop heard during an open reply to that reply (AI-1535).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1534: Give an open Gemini reply the stop and test the end-of-speech guards
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1533] Open a GPT-Live reply on audible output only (#1056)
* AI-1533: Open and close a GPT-Live reply on audible output only
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1533: keep silence before a reply's first sound off the wire
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1533: give the one-reply test a gap a scheduler stall cannot cross
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* AI-1533: give the other gap-sensitive openailive tests a gap a stall cannot cross
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1505] Configuration-suggestion skill for Ask Nora (#1065)
* AI-1505: Add Nora's configuration-suggestion skill to the moderator template
Move the proof of concept's suggest_policy flow into the agent_moderator
template as a skill. From the live policy, the last 7 days of reviews and
Nora's disagreements with the first layer, it proposes label, app context,
severity, blocklist, rule and flooding changes as the JSON the dashboard's
apply-configuration.ts saves after the moderator confirms. It only proposes.
* AI-1505: Let the caller's format win and keep severity tiers
The agents-shell button asks for a written draft, Ask Nora parses JSON.
Restore the proof of concept's guidance on per-severity actions.
* [AI-1504] Read-only Stream moderation tools for Ask Nora (#1066)
* AI-1504: Read-only Stream moderation tools for the app's moderator
A session or run of an agent_moderator config is offered six tools that
read the app's moderation data with the Stream identity the router already
holds for it: the review queue, one item with its flags and actions, the
moderation configs (with their LLM labels), rules, blocklists and logs.
The tools only exist in an app of the customer's own, never the
deployment's shared app, which holds every other customer's data too. The
review queue is read without locking, and a tool the caller declares under
the same name stays the caller's.
* AI-1504: Offer the moderation tools to server-side callers only
createSession is open to end users' devices, and any of them could open a
session on the app's moderator and read every user's reports. The tools
are now offered only when the customer's backend opened the session. A
query's limit is also cut to 25 whatever the model asks for, so one page
cannot fill the context it answers from.
* AI-1506: Port Nora's eval cases to simulations on the moderator template (#1068)
Adds acceleration/templates/agent_moderator/simulations/moderation.yaml:
eight simulations ported from GetStream/nora's eval/cases.json and
agent/simulations/moderation.yaml. They cover casual against targeted
profanity under two app policies, an educational quotation, repeat
confirmed harassment, an overturned incident that must not escalate, an
innocent message from a previously banned user, and a prompt injection.
A simulation talks to Nora in chat, not through POST /v1/agents/runs, so
it gets neither the app's policy as instructions nor the router's answer
contract. Each caller therefore opens with a compact policy and a case
bundle shaped like the chat review worker's, kept under the caller's
300-token turn, and each assertion judges the verdicts and evidence rather
than the JSON shape. Actions are the server's to choose, so the eval's
expected actions became checks that Nora claims none and reads history
the way its instructions say.
* AI-1059: none connections move revisions (#1064)
* fix(connectors): none connections move revisions on a credentials save
A none connection refuses a consent, so it could never leave an old or
broken connector revision. Saving its empty credentials again now moves
it to the latest revision, as saving a bearer or api_key token again
does. none stays out of core.Static: refusedStatic would read any 4xx on
a validate as a rejected credential it does not have.
Docs: the add-connector skill shows schemes ["none"] in place of the
dummy api_key workaround.
AI-1059
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1059: one helper for connections that are saved again, not consented
none refuses a consent, so a none connection on a broken revision or
rejected needs the same save-again remedy as a bearer one. none.SavedAgain
covers the credentials save, both validate branches and the session's two
unavailable reasons. Control: an oauth2_cc connection keeps its revision.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* [AI-1550] Metrics listener and overlapping-speech warning (#1063)
* feat(router): metrics listener and an overlapping-speech warning
The router had no metrics, and nothing said when a voice published two
utterances of one session interleaved, which the caller hears as the
agent talking over itself (#video-ai report 2026-10-20, staging call
agent:01a15a81-e9f7-7699-9d48-53611de52472).
- metrics_addr (ROUTER_METRICS_ADDR) serves Prometheus /metrics on its
own listener, OpenMetrics enabled; empty by default, stopped with the
router.
- consumeTTS logs one WARN per interleaved pair (both synthesis and
turn ids, provider) and counts router_agent_speech_overlaps_total and
router_agent_speech_overlap_seconds_total by provider. An utterance
counts from its first published chunk, so a sentence a non-streaming
voice starts early is not an overlap.
- Agent logs carry session and call.
What is published to the edge is unchanged.
AI-1550
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* AI-1550: test the metrics address gate, the warning's call id and overlap cleanup
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Martin Mitrevski <martinmitrevski.oh@gmail.com>
Co-authored-by: Aleksandar Apostolov <apostolov.alexandar@gmail.com>
Co-authored-by: Darko <darkoatanasovski@gmail.com>
Co-authored-by: Oliver Lazoroski <oliver.lazoroski@gmail.com>
Co-authored-by: Ali Aliyev <ali.aliyev@getstream.io>
* AI-1524: Refuse repeat rules the description cannot say (#1075)
* Refuse repeat rules the schedule description cannot say
A new repeat whose Descri…
