Skip to content

livekit-agents@1.8.11

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 05 Nov 19:48
· 3 commits to main since this release
livekit-agents@1.8.11
296f28b
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Responses API through LiveKit Inference

You can now use the Responses API with your LiveKit Inference credentials, including structured output and web search. REST is the default; opt into WebSocket transport to reuse connections across turns.

from livekit.agents import inference

llm = inference.responses.LLM("openai/gpt-4.1", use_websocket=True)

New providers and models

  • Zoom AI: Zoom Scribe STT with live streaming and per-utterance recognition.
  • Kalpa: kalpa-tts-1 streaming TTS in English and Hindi.
  • Rime: STT and Prism speech-to-speech through the standard realtime interface; both require Python 3.11+, and STT requires the stt extra.
  • Fish Audio: transcribe-stream-1 realtime STT with word timings and recognition context.
  • OpenAI: gpt-realtime-translate through the beta translation interface, with gpt-realtime-whisper input transcription.
  • Google: gemini-3.5-live-translate-preview translation through Gemini API and Vertex AI.
  • Respeecher: public Swedish, Brazilian Portuguese, German, and Arabic TTS models.

Security

  • Respeecher: moves WebSocket API keys from request URLs to handshake headers, keeping credentials out of URLs.

Behavior changes

  • Inference Responses: now defaults to each known model’s lowest reasoning effort; set reasoning=None to retain the provider default.
  • STT turn detection: late finals after speech resumes join the resumed turn, which waits for STT’s next end-of-speech event.
  • Uninterruptible speech: user transcripts are now retained without interrupting playback or generating a reply.

This release also improves realtime history and reconnection, task handoffs, STT timing, transcript synchronization, VAD updates on Python 3.14, worker cleanup, and multichannel TTS fallback.

What's Changed

  • fix(realtime): keep item ids within OpenAI's 32-char limit by @russellmartin-livekit in #7958
  • docs(mistralai): add missing store option to LLM options example by @detail-app[bot] in #8219
  • feat(soniox): pass client_reference_id to the TTS websocket config by @grace0950 in #8236
  • fix(thunderphone): use _shorten_id after the openai_utils rename by @chenghao-mou in #8229
  • test(drive_thru): replace the eval tests with simulation scenarios graded on the final order by @u9g in #8176
  • fix(voice): suspend an AgentTask's awaiter in the run instead of unwatching it by @u9g in #7794
  • fix(voice): don't fail the run with a task error its awaiting tool handled by @osekkat in #8269
  • Sunset Sarvam TTS model bulbul:v2 and remove bulbul:v3-beta by @dhruvladia-sarvam in #7056
  • feat(rime): add Coda custom_lexicon option by @saad-py in #8245
  • fix(voice): end a turn on a late STT final after a later END_OF_SPEECH by @acharan-tech-200037 in #7739
  • update livekit sdk/api to 1.2/1.2.2 by @davidzhao in #8299
  • fix(voice): keep the user turn in the chat context when the current speech cannot be interrupted by @valemicolgarcia in #7922
  • chore: remove dead AgentsConsole members from cli.py by @detail-app[bot] in #8277
  • fix(worker): close resources when startup fails before run() completes by @BlueX888 in #7942
  • fix(deepgram): guard empty languages list in multilingual STT decoding by @Speedyzyzz in #7976
  • fix(vad): resize the speech buffer by copying so it works on Python 3.14 by @RitikaxG in #8162
  • fix(tools): omit cancel_task from duplicate reject/confirm when not cancellable by @ninglinLiu in #8270
  • reduce status update interval to 1.5s by @davidzhao in #8150
  • fix(openai): end a failed realtime response without an unretrieved exception by @zek in #8218
  • fix(voice): ignore a repeated flush of the same audio segment by @sehaj23 in #8316
  • fix(telemetry): keep the stall sampler out of linecache by @zek in #8226
  • fix: resolve a failed local turn-detector prediction to the positive default by @chenghao-mou in #8171
  • feat(rime): STT Support by @augustohalvarez in #8013
  • chore(krisp): bump livekit-plugins-krisp-internal to 0.2.3 by @lukasIO in #8250
  • feat(mistralai): expose reasoning effort on the LLM by @muzman123 in #8285
  • fix(respeecher): use WebSocket auth headers and add public models by @mitrushchienkova in #8286
  • feat(speechmatics): support 8kHz audio and emit_sentences config by @LArmstrongDev in #7952
  • chore(deps): update dependency langgraph to v1.2.14 [security] by @renovate[bot] in #8254
  • feat: add Zoom AI STT plugin by @cdliao in #7708
  • feat(cambai): add speaking_rate option to TTS by @nihaalnz in #8324
  • update dependencies by @davidzhao in #8325
  • feat(openai): gpt-realtime-translate as a beta TranslationModel by @tinalenguyen in #8105
  • fix(stt): anchor transcript timestamps on the session input start by @chenghao-mou in #7795
  • fix: report speech end timestamps across stt plugins by @chenghao-mou in #7796
  • refactor(openai): Move the Responses engine into livekit-agents by @adrian-cowham in #8188
  • test(openai): Add a live probe for multi-message Responses replies by @adrian-cowham in #8209
  • feat(inference): Add inference.responses.LLM by @adrian-cowham in #8210
  • feat(inference): WebSocket transport for inference.responses.LLM by @adrian-cowham in #8211
  • feat(inference): update_options and response_format for inference.responses.LLM by @adrian-cowham in #8212
  • docs(examples): Show inference.responses.LLM in basic_agent by @adrian-cowham in #8213
  • fix(inference): Don't retry terminal quota denials on inference.responses by @adrian-cowham in #8268
  • fix: drop self from the static type of bound function tools by @u9g in #8336
  • fix(openai): stop resending the previous output when chaining Responses requests by @longcw in #8085
  • fix(google): start a fresh Gemini session on a config change by @longcw in #7944
  • fix(voice): don't hide IGNORE_ON_ENTER tools after on_enter returns by @BhaskarKapri07 in #8280
  • fix(inference): don't crash on tool_call arguments that arrive before a name by @AKomplished-bug in #8249
  • feat(gradium): optional VAD-based start of speech in STT by @davencyw in #8247
  • fix(room_io): date a run's end at submission, not after a stalled capture by @u9g in #8168
  • fix(aws): hold on to the user-text and deferred-recycle tasks by @Rehansanjay in #7052
  • fix(voice): let only the STT end a turn the STT reopened by @chenghao-mou in #8318
  • fix(transcription): stop speaking-rate bursts from blocking the event loop by @longcw in #7938
  • fix(duplex): keep the words still waiting when the output stream ends by @basnijholt in #8301
  • fix(inference): Send agent_context only to STT models that read it by @adrian-cowham in #8365
  • feat(google): Gemini 3.5 Live Translate as a beta TranslationModel by @tinalenguyen in #8106
  • feat(openai): Add GPT-Live Decisions support by @shawnfeldman in #8338
  • feat(fishaudio): add realtime STT by @cshape in #8357
  • feat(inference): reasoning= for inference.LLM and inference.responses.LLM by @adrian-cowham in #8358
  • feat(inference): default inference.responses.LLM to the model's lowest reasoning effort by @adrian-cowham in #8359
  • fix(fireworksai): honor stream language override by @JY-M666 in #8390
  • feat(examples): support Tavus alongside LemonSlice in the avatar example by @ggarber in #8131
  • feat(voice): send session_closed to the remote session when the session closes by @u9g in #8099
  • fix(aws): honor language passed to STT.stream() by @pcbeingused333 in #8374
  • fix(tts): preserve channels when resampling fallback audio by @feiiiiii5 in #8393
  • feat(rime): add Prism through the standard realtime interface by @naszzz in #7856
  • fix(realtime): end a forced ask on its tool call and restore tool_choice once by @longcw in #8395
  • feat: report the agent join steps to the server by @boks1971 in #8173
  • feat(kalpa): add Kalpa TTS plugin by @hvg639 in #8017
  • fix(gpt-live): cap caller audio queued before session.started by @zunkelty in #8387
  • fix(ipc): mark supervised processes dead for prometheus by @Ricky-7-Yan in #7017
  • feat(inference): Add rime/prism to the realtime models by @tinalenguyen in #8396
  • livekit-agents@1.8.11 by @github-actions[bot] in #8397

New Contributors

Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.8.10...livekit-agents@1.8.11

Contributors

davidzhao, ggarber, and 42 other contributors
Assets 2
Loading