Skip to content

livekit-agents@1.8.9

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 24 Oct 02:35
livekit-agents@1.8.9
d4de578
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Decisions through LiveKit Inference

You can evaluate named decisions with LiveKit credentials instead of a provider key. inference.DecisionModel batches evaluations and reports refusals per decision without discarding the other answers. Access requires a project feature flag.

model = inference.DecisionModel("openai/gpt-6-luna-decisions")
response = await model.evaluate(
    input=chat_ctx,
    decisions={"handoff": llm.Probability("The caller requests a human.")},
)

Custom LLM request attributes

You can attach prompt metadata directly to the provider’s llm_request span from your custom llm_node, without a custom OpenTelemetry processor. Attributes stay scoped to that node’s calls.

async def llm_node(self, chat_ctx, tools, model_settings):
    gen_ai.set_llm_request_attributes(
        {"langfuse.observation.prompt.name": "support-agent"}
    )
    async for chunk in Agent.default.llm_node(
        self, chat_ctx, tools, model_settings
    ):
        yield chunk

Silent tool progress

You can record background-tool progress without prompting a spoken response. Final background-tool results can now trigger follow-up tool calls.

await ctx.update("checking the fare rules", silent=True)

New providers and models

  • ThunderPhone: new realtime speech-to-speech plugin for saved-agent and inline sessions, including bolt.
  • OpenAI: Decisions API support directly and through Inference with openai/gpt-6-luna-decisions.
  • TypeSafe Jev and LiveKit Clef Omni: decision evaluation through LiveKit Inference.
  • Microsoft-Decision-1: batched decisions through Microsoft Foundry.
  • AssemblyAI assemblyai-tts-1: streaming TTS with word timestamps and six languages.
  • Keyframe persona-2.0-live: speech-timed avatar expressions.
  • Fish Audio drama-3: Inference TTS with free-form expressive steering.
  • Speechify simba-3.5: streaming synthesis with timestamps.
  • Google gemini-3.8-flash: added to supported chat models and selected by default.

Security

  • Tool errors: raw arguments and validation details no longer bypass PII filtering through log message bodies.
  • Simli: startup error logs no longer expose the session token.
  • RoomIO: configured participant identity now restricts chat before linking and during reconnection, preventing another participant from starting a turn.

Breaking changes

  • Rime: WebSocket v1 requires explicit model; add it if you relied on URL-path detection.
  • Gradium: passing both voice and voice_id now raises ValueError; choose one.
  • Inworld: max_connections and idle_connection_timeout are deprecated and ignored; TTS now uses one WebSocket per instance.

Behavior changes

  • Adaptive interruptions: errors now fall back to VAD without reconnecting by default.
  • STT flushing: supported engines now flush at manual and detected turn boundaries; buffering custom nodes must disable manual_flush.
  • Gradium: the default voice is now Everly; explicit voice selections are honored.
  • Anthropic: forced tool choices become auto for Opus 5.5, Sonnet 5.5, Fable 5.1 and Mythos 5.1, so tool use is not guaranteed.
  • Inworld: inworld-tts-2 uses server-side sentence splitting by default; client-side splitting defaults to 250 characters.
  • Dependencies: OpenAI requires SDK 3.28+, Phonic requires phonic>=0.32.36, and the framework requires livekit-protocol>=1.1.28.

This release also improves reliability across realtime history, tool handoffs, task shutdown, TTS retries, avatar interruptions, worker prewarming, and session reporting.

What's Changed

  • fix(workflows): support turn_handling in warm transfers by @JonasDaWi in #7847
  • feat(room_io): warn when agent and client both apply noise cancellation by @lukasIO in #7848
  • docs: stop flagging LiveKit identifiers as PII in Devin review by @chenghao-mou in #7849
  • fix(inference): send keyterms to speechmatics/linden-1 by @shawnfeldman in #7853
  • feat(inference): add DecisionModel for the gateway's /v1/decisions by @shawnfeldman in #7855
  • docs(inference): Deepgram mip_opt_out is always true on the gateway by @shawnfeldman in #7860
  • fix(elevenlabs): offset text-to-dialogue alignment by chunk start by @satheeshk50 in #7784
  • prewarm Resource.create to avoid blocking runloop during start by @davidzhao in #7872
  • fix(voice): skip audio turn handling warnings in text simulations by @u9g in #7878
  • fix(gradium): identify the STT plugin to the API with x-api-source: livekit-plugin by @hhugo-gradium in #7873
  • docs: add Google realtime models to inference SUPPORTED REALTIME MODELS list by @detail-app[bot] in #7861
  • docs: add the shared LiveKit sections to AGENTS.md by @JustinScitech in #7885
  • fix(anthropic): fall back to auto tool_choice for Opus 5.5, Sonnet 5.5, Fable 5.1 and Mythos 5.1 by @omChauhanDev in #7511
  • feat(deepgram): support speed parameter in Flux TTS by @73bruno in #7483
  • fix(llm): drop nonexistent name from all by @BlueX888 in #7894
  • Clarify what "item" is in logged warning by @kath0la in #7934
  • Send lk.agent.session messages with one-shot send_bytes instead of stream_bytes by @1egoman in #7751
  • fix(cli): apply connect credentials to the agent server by @BlueX888 in #7929
  • feat(telemetry): let llm_node set attributes on the llm_request span by @rrfunde in #7827
  • fix(simli): retry session start and stop the error handler from crashing by @sidxh in #7498
  • feat(soniox): report output tokens in RECOGNITION_USAGE for token-based billing by @Panmax in #6435
  • fix(cartesia): expand TTS language support and add WebSocket timestamp params by @Namit1867 in #7754
  • fix(voice): keep ToolError detail out of log message bodies by @Li-john1021 in #7804
  • fix(llm): keep a nullable enum's null out of the shared $defs entry by @ANDROIDHASSAN in #7760
  • fix(llm): render missing {} and formatted template placeholders as empty by @ANDROIDHASSAN in #7783
  • fix(tts): don't repeat the reply when a FallbackAdapter stream retries by @RachelWanggg in #7840
  • fix(ipc): drop exited idle processes from the warmed queue by @RGB-loop in #7864
  • fix(bey): use the current session endpoint and send 16 kHz audio by @niqodea in #7877
  • fix(bey): don't retry 4xx when starting a session, and keep the provider's error by @sidxh in #7496
  • fix: do not reconnect the adaptive interruption stream by @chenghao-mou in #7923
  • chore: remove unused BoundedDict.set_or_update method by @detail-app[bot] in #7568
  • feat(openai): add DecisionModel for OpenAI's Decisions API by @tinalenguyen in #7882
  • fix(proto): stringify map values in encode_by_name by @u9g in #7962
  • fix(rime): require explicit v1 models and anonymize endpoints by @naszzz in #7964
  • fix(meta): don't delay Muse STT audio by the websocket connect time by @DEVJHAWAR11 in #7890
  • fix(voice): stop wait_for_idle from spinning through a speech delay by @longcw in #7965
  • fix(voice): preserve tool outputs across concurrent realtime context updates by @Epsilon006 in #7905
  • fix: honor participant identity for chat before linking by @chenghao-mou in #7435
  • fix(tokenize): don't hold a CJK reply until the end in word streams by @dev-S-t in #7669
  • fix(hume, simli): make every all entry resolve by @omChauhanDev in #7981
  • fix(inference): emit every parallel tool call sent in one chunk by @pcbeingused333 in #7887
  • fix(realtime): treat credit_balance_exhausted as a fatal OpenAI error by @jipeng6036-del in #7819
  • fix(gradium): identify the TTS plugin to the API with x-api-source (1/4) by @hhugo-gradium in #7815
  • fix(voice): add say() text to the realtime chat context when it is spoken by @xiaoyuCR7 in #7947
  • fix(gradium): stop voice_id default from overriding an explicit voice (1/2) by @hhugo-gradium in #7813
  • fix(room_io): size trailing silence by num_channels by @Hugo-DDT in #7968
  • fix: prevent stale avatar clear timeouts from finishing new replies by @chenghao-mou in #7980
  • feat(microsoft-ai): add DecisionModel for Microsoft-Decision-1 by @tinalenguyen in #7883
  • fix: keep sarvam stt streams open across flushes by @chenghao-mou in #7990
  • fix: flush supported stt at manual and detected turn boundaries by @chenghao-mou in #7285
  • feat: enable manual flushing in elevenlabs stt by @chenghao-mou in #7287
  • feat(keyframe): sync avatar emotions to speech for persona-2.0-live by @kradkfl in #7846
  • fix(elevenlabs): pad short commits so Scribe v2 keeps the session by @u9g in #7992
  • ci: run the data-capture simulation with lk agent simulate text by @u9g in #7982
  • feat: add ThunderPhone realtime (speech-to-speech) plugin by @kolchinski in #7202
  • fix(voice): don't cut an AgentTask's speech on shutdown() by @RachelWanggg in #7839
  • fix(cartesia): accept an integer speed for sonic-3 models by @ShivamSharma43 in #7945
  • fix(voice): reset realtime agent state without audio output by @Luke-salinero in #7869
  • fix(gradium): report the configured model name from TTS.model and STT.model by @hhugo-gradium in #7814
  • Add audio_sample_rate and model args for LemonSlice avatars by @jp-lemon in #8005
  • fix(utils): do not let a blank or non-numeric LK_* debug flag break package import by @Lesereingrape in #7470
  • feat(gradium): default the TTS voice to Everly by @hhugo-gradium in #7874
  • fix(inference): Drop the sampling params gpt-6 rejects while it reasons by @adrian-cowham in #8001
  • feat(inworld): cancel the TTS context when a stream is abandoned by @ianbbqzy in #8006
  • feat(inworld): let the server split streamed text on inworld-tts-2 by @ianbbqzy in #8000
  • fix(phonic, baseten): register plugin loggers so debug/info logs show up by @omChauhanDev in #8007
  • fix(soniox): emit END_OF_SPEECH when an endpoint follows a fully retracted segment by @iamdgarcia in #7897
  • fix(speechify): list_voices() iterates the SDK's async pager by @vitalragaz in #8034
  • feat(speechify): stream simba-3.5 and any later model without a plugin update by @vitalragaz in #8029
  • docs(inworld): correct buffer_char_threshold default in README example by @detail-app[bot] in #8036
  • fix(minimax): validate options in update_options like the constructor by @ShivamSharma43 in #8009
  • feat(hotel_receptionist): publish suggested replies by @ShayneP in #7345
  • chore: remove unused GladiaLanguages type alias by @detail-app[bot] in #7863
  • fix(google): leave TTS ssml_gender unspecified when no gender is given by @sarankumar1325 in #7954
  • fix(neuphonic): create segments channel per-run to fix retry after connection error by @muntazir868 in #7622
  • fix: don't bind names inside assert statements by @bercedev in #8046
  • fix: update certifi, uv and Go (eMudhra intermediate CA compromise) by @kyle-livekit in #7998
  • fix(gradium): surface TTS server errors, close codes and transport errors (1/2) by @hhugo-gradium in #7672
  • fix(voice): stop the preemptive generation when an interim moves past the preflight by @Sakuta014 in #7949
  • feat(fishaudio): free-form expressive steering for drama-3 by @cshape in #8049
  • fix(google): use role=system for Vertex update_instructions by @Li-john1021 in #7802
  • fix(voice): end the call after the turn in EndCallTool when no reply is asked for by @TANGBUDU in #8011
  • docs(examples): fix broken drive-thru README links by @kun654 in #8052
  • fix(openai): report reasoning tokens for Responses API usage by @pcbeingused333 in #8033
  • feat(anam): use lk.playback_started to track turn state by @sebvanleuven in #8042
  • chore(deps): update dependency aiohttp to v3.14.4 [security] by @renovate[bot] in #7961
  • feat(phonic): add external_id option to RealtimeModel by @arunwpm-work in #8070
  • fix(agents): set SessionReport.duration when audio recording is off by @JustinScitech in #8083
  • docs(soniox): fix moved STT/TTS documentation links by @kun654 in #8084
  • fix(voice): wait for a paused activity before delivering a tool reply by @longcw in #7685
  • fix(voice): arm the user-away timer while an AgentTask runs by @longcw in #8056
  • feat(voice): add silent to ctx.update() and allow tools in the final reply by @longcw in #7966
  • fix(anthropic): send parallel_tool_calls when no tool_choice is set by @sarankumar1325 in #8082
  • fix(tts): keep StreamPacer buffered tail after end_input by @Hugo-DDT in #7816
  • chore: ignore google coroutine to avoid tripping up fail_on_leaked_tasks by @davidzhao in #8087
  • feat(assemblyai): add Streaming TTS (assemblyai-tts-1) by @dlange-aai in #8040
  • feat(plugins-tavus): add AvatarSession.set_emotion() by @ggarber in #8025
  • include code tracking & agent name in logs by @andrewnitu in #8096
  • refactor(inworld): use a single TTS websocket instead of a connection pool by @ianbbqzy in #8038
  • feat(beta): add TranslationSession for speech-to-speech translation m… by @tinalenguyen in #7776
  • telemetry: send a session summary in the recording header by @jmcclanahan in #8069
  • fix(google): default the LLM to gemini-3.8-flash by @pcbeingused333 in #8081
  • feat(inference): claim aligned transcripts for AssemblyAI TTS by @adrian-cowham in #8101
  • feat(keyframe): [tag] emotion markers for ElevenLabs v3/v4 voices by @kradkfl in #8104
  • feat(azure): pronunciation assessment on the streaming STT by @Bilnaa in #8102
  • livekit-agents@1.8.9 by @github-actions[bot] in #8100

New Contributors

Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.8.8...livekit-agents@1.8.9

Contributors

davidzhao, jmcclanahan, and 59 other contributors
Assets 2
Loading