@livekit/agents@1.7.0

Introducing PII Redaction Support

This release adds PII redaction support for Agent Observability. It semantically redacts detected entities from chat history and audio recordings. It also filters sensitive data from logs and traces while retaining useful diagnostic context.

To support this feature, we renamed sensitive trace attributes and log fields emitted by the framework. If your third-party observability queries depend on these names, update them when you upgrade.

Introducing Expressive Mode

Expressive mode allows voice agents to speak with natural prosody and emotion. Speech delivery is determined by emotion tags generated from the context of the conversation. Enable expressive mode in your AgentSession:

from livekit.agents import AgentSession, inference

session = AgentSession(
stt=inference.STT(model="deepgram/nova-3", language="multi"),
llm=inference.LLM(model="google/gemma-4-31b-it"),
tts=inference.TTS(model="fishaudio/s2.1-pro", voice="b347db033a6549378b48d00acb0d06cd"),
expressive=True,
# ... vad, turn_detection
)

See the documentation and the blog post for more information.

What's Changed

• ci: bump gh-action-pypi-publish for metadata 2.5 support by @u9g in #6836
• ci: pin the publish action to a commit, not a tag object by @longcw in #6837
• fix(elevenlabs): send keyterms on the realtime STT WebSocket by @pawlowskipawel in #6746
• fix: emit completed tool events after interruption by @chenghao-mou in #6823
• (expressive example): update readme by @tinalenguyen in #6829
• fix: preserve Cartesia inference timestamp spacing by @chenghao-mou in #6848
• fix(openai): fail generate_reply fast on conversation_already_has_act… by @ByteMaster-1 in #6818
• fix(voice): answer every tool call by @longcw in #6785
• fix: preserve adaptive interruption across tool calls by @chenghao-mou in #6564
• chore(deps): update github actions (major) by @renovate[bot] in #6395
• feat(spatius): forward avatar session extra params, update spatius to 1.0.5 by @3DRX in #6878
• fix(voice): keep the turn transcript when the tts sends no word timings by @biztex in #6704
• fix(voice): keep transcript pacing speed nonzero for syllable-free segments by @kevtae in #6751
• fix(voice): capture the full on_enter output when starting with capture_run by @biztex in #6872
• fix(audio): check AudioArrayBuffer frame size after resampling by @longcw in #6879
• fix(telemetry): log a warning when the audio recording can't be read at upload time by @dubbl-a in #6833
• fix(tokenize): include the last character in the final sentence span by @uuzzrm in #6866
• fix(telemetry): report cached input tokens on LLM spans by @Panmax in #6852
• feat(expressive): allow dynamic updates via Agent and update_options by @tinalenguyen in #6886
• plugins/phonic: support built-in tools via phonic_tools + configs_for_tools by @arunwpm-work in #6853
• feat: correlate inference requests with session ids by @chenghao-mou in #6863
• fix(llm): preserve provider events on restart by @Bortlesboat in #6568
• fix(agents): omit separator for empty exception details by @u9g in #6894
• fix: wait for final user turn during close by @chenghao-mou in #6897
• chore: bump local inference to 0.2.7 by @chenghao-mou in #6892
• docs: fix typos in healthcare example by @Excelius-Wang in #6912
• feat(ultravox): support server-side external TTS voices via externalVoice by @ssgutierrez42 in #6850
• fix(cli): restore terminal settings on console exit by @kamalmostafa in #6046
• fix: prevent wait_for_idle cancellation from poisoning user turns by @Excelius-Wang in #6913
• fix(tts): include supported encodings in RuntimeError message by @YusufKeremKezer in #6910
• fix: place Gemini realtime user turns before replies by @chenghao-mou in #6914
• fix(rime): remove Arcana model support by @naszzz in #6773
• feat(bland): BTTS ws integration by @rv-blandai in #6752
• feat(telemetry): tag content-bearing trace and log keys with lk.pii. for redaction AGT-3074 by @chenghao-mou in #6356
• fix(proto): one ChatItem serializer that carries every declared field by @u9g in #6756
• livekit-agents@1.7.0 by @github-actions[bot] in #6928

New Contributors

• @pawlowskipawel made their first contribution in #6746
• @kevtae made their first contribution in #6751
• @dubbl-a made their first contribution in #6833
• @Bortlesboat made their first contribution in #6568
• @Excelius-Wang made their first contribution in #6912
• @ssgutierrez42 made their first contribution in #6850
• @kamalmostafa made their first contribution in #6046
• @YusufKeremKezer made their first contribution in #6910
• @naszzz made their first contribution in #6773

Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.6.10...livekit-agents@1.7.0

@livekit/agents@1.6.10

What's Changed

• fix(cartesia): redact API keys from websocket handshake errors by @LHMQ878 in #6740
• fix: respect non-retryable STT errors by @chenghao-mou in #6759
• fix: fallback adapters report the active instance's model/provider by @Fyko in #6690
• fix(inference): don't crash the session on null usage token counts by @biztex in #6763
• fix(voice): stop the interrupt walk at a queued speech that disallows interruptions by @biztex in #6644
• fix(voice): don't raise when interrupting a cancelled or finished speech by @longcw in #6767
• fix: isolate bundled VAD inference from the default executor by @chenghao-mou in #6770
• fix(xai): align realtime Voice Agent defaults and API coverage by @damien-xai in #6755
• feat(spatius): default audio format to ogg_opus by @3DRX in #6749
• fix: don't count discarded room audio as played by @chenghao-mou in #6728
• feat(azure): replace unknown tts model string with host/region + voice by @alvxrog in #6775
• feat: expose cache_creation_tokens in LLM metrics by @chasleslr in #6655
• fix: treat a silent agent turn as silence, not a run_input error by @theomonnom in #6779
• fix: stop dropping session host responses by @theomonnom in #6781
• fix(llm): never give a const field the null default sentinel by @u9g in #6688
• fix: hold pending replies at playout launch by @chenghao-mou in #6745
• fix(inference): keep a stream retryable until it emits generation by @u9g in #6697
• test: stabilize text email collection eval by @chenghao-mou in #6791
• feat(evals): run judges at low inference priority by @u9g in #6794
• feat(simulation): pin text simulations to the low inference class by @theomonnom in #6721
• fix(google): allow explicit project for Speech-to-Text by @uuzzrm in #6810
• fix(silero): update bundled VAD checkpoint to v6.2 by @longcw in #6816
• fix(voice): export uncommitted user_turn spans on close by @chenghao-mou in #6592
• feat(telemetry): include recording options in session reports by @chenghao-mou in #6547
• fix: respect agent endpointing delay during reply hold-off by @chenghao-mou in #6820
• docs: require confirmation for non-patch version bumps by @chenghao-mou in #6821
• examples: switch homepage agent to Fish Audio with expressive mode by @theomonnom in #6822
• fix: stabilize LiveKit pronunciation with Fish Audio by @chenghao-mou in #6825
• fix(deps): pin openai below 3 by @longcw in #6831
• livekit-agents@1.6.10 by @github-actions[bot] in #6834

New Contributors

• @Fyko made their first contribution in #6690
• @damien-xai made their first contribution in #6755
• @alvxrog made their first contribution in #6775
• @chasleslr made their first contribution in #6655

Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.6.9...livekit-agents@1.6.10

@livekit/agents@1.6.9

What's Changed

• Add support for Deepgram's Flux TTS API (v2/speak) integration by @dg-edcharbeneau in #6511
• fix(cerebras): expose max_completion_tokens by @ryanl-cerebras in #6685
• Make every example a uv workspace member by @u9g in #6649
• feat(bland): add Bland TTS plugin by @rv-blandai in #6554
• voice: forward agent_false_interruption over the RemoteSession by @theomonnom in #6691
• plugins/phonic: expose more realtime config options by @arunwpm-work in #6687
• fix(tts): give each fallback recovery path its own task slot by @LHMQ878 in #6683
• tests: assert which session events the host forwards, not how many by @theomonnom in #6693
• fix(livekit-agents): forward the STT hooks through MultiSpeakerAdapter by @LHMQ878 in #6679
• fix(google): release the genai client when the LLM or Gemini TTS closes by @LHMQ878 in #6686
• fix(exceptions): name the underlying cause when an LLM/TTS/STT connection fails by @u9g in #6696
• feat(openai): add gpt-5.5 to model registry by @xtreme-sameer-vohra in #6263
• feat(google): accept a credentials object on STT and TTS by @biztex in #6618
• Feat/anam support ai disclosure by @sr-anam in #6695
• fix(soniox): report RECOGNITION_USAGE on every frame by @vadimatmurphy in #6585
• fix(voice): keep the run open while a tool holds the floor by @longcw in #6694
• feat(metrics): broadcast llm tps + ttfs by @Bobronium in #6373
• fix(amd): start SIP detection timeout after answer by @chenghao-mou in #6580
• fix(deepgram): use punctuated_word for word-level transcripts by @mdylan2 in #6699
• feat(eot): let a cloud TurnDetector opt out of the local fallback by @theomonnom in #6719
• (expressive mode): remove extra space after stripping tag by @tinalenguyen in #6717
• fix(audio): don't report a stopped playback as a decode error by @longcw in #6723
• fix(aws): gate audio frames during text block sends to prevent Nova Sonic ValidationException by @lanazhang in #6720
• feat(openai): support context hints in STT by @longcw in #6705
• fix(deepgram): end_time should be the last word, not the first by @mdylan2 in #6712
• fix(tts): probe a skipped provider for recovery on the streamed path by @biztex in #6684
• fix(ipc): report actual consumed memory instead of RSS by @Bobronium in #6082
• feat(voice): add session_host to AgentSession.start() by @theomonnom in #6725
• fix(ipc): don't raise from is_alive after the inference process is closed by @RGB-loop in #6732
• feat(baseten): add Qwen3-ASR STT and Qwen3-TTS support, and language_options for Whisper STT by @A-K-Erol in #6700
• feat(fishaudio): enable quality-guard on all TTS requests by @cshape in #6734
• (core): support duplicate tool policy specification by @tinalenguyen in #6610
• fix(xai): emit one final user transcript per turn by @longcw in #6729
• feat(inference): log gateway quota headers when LLM requests hit a 429 by @adrian-cowham in #6736
• feat(expressive): expose expressive mode and add expressive_agent example by @theomonnom in #6698
• livekit-agents@1.6.9 by @github-actions[bot] in #6753

New Contributors

• @dg-edcharbeneau made their first contribution in #6511
• @rv-blandai made their first contribution in #6554
• @LHMQ878 made their first contribution in #6683
• @mdylan2 made their first contribution in #6699
• @RGB-loop made their first contribution in #6732
• @A-K-Erol made their first contribution in #6700

Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.6.8...livekit-agents@1.6.9

@livekit/agents@1.6.8

What's Changed

Warning

console and dev modes are deprecated in favor of lk agent (#6656). Existing workflows that shell out to python agent.py console or python agent.py dev should migrate to the lk agent CLI.

Important

Inference Ink-2 speech onset fix (#6630) — if you rely on the STT model for interruption detection without a local VAD, upgrade for this fix. Previously the provider's speech-onset signal was delayed until first non-empty interim transcript, so STT interruptions could be delayed on Ink-2 via Inference. Setups with a local VAD in the pipeline are unaffected.

Features

• feat(core): add user transcription timeout by @chenghao-mou in #6182
• feat(voice): auto-disable realtime server-side turn detection by @longcw in #6495
• feat(cli): deprecate console and dev modes in favor of lk agent by @u9g in #6656
• feat(elevenlabs): support previous_text on realtime STT by @tulanisahil1-beep in #6534
• feat(smallestai): add endpointing, keyword boosting, format, and sentence timestamps to Smallest STT by @harshitajain165 in #6565
• (xai stt): expose vad_threshold, smart_turn, smart_turn_timeout, and keyterm params by @ajayarora1235 in #6587
• (cartesia stt): expose ink-2 turn-detection thresholds and keyterm params by @ajayarora1235 in #6594
• (inference): type Cartesia turn-detection params in CartesiaOptions by @ajayarora1235 in #6597
• (inference openai llm): set default reasoning to none by @tinalenguyen in #6609
• Update Cerebras models and default by @ryanl-cerebras in #6593
• plugins/phonic: add configs_for_tools for per-tool behavior overrides by @arunwpm-work in #6634
• (expressive mode): refactor options, remove prompt presets by @tinalenguyen in #6406
• (expressive mode): register inference replaces presets by @cshape in #6632
• (expressive mode): add fish audio by @cshape in #6631
• (expressive mode): remove unconditional bracket stripping by @tinalenguyen in #6590
• feat(inference): let the gateway mint the avatar worker token by @shawnfeldman in #6575
• add agent id and deployment resource attrs by @jmcclanahan in #6570
• feat(deps): relax opentelemetry constraint to >=1.39.0,<2 by @biztex in #6552
• chore(krisp): adjust default noise_suppression_level by @lukasIO in #6640

Bug Fixes

• fix: defer interim interruptions to local VAD by @chenghao-mou in #6599
• fix(voice): retain concrete language for turn detection by @Oxygen56 in #6531
• fix(openai realtime): treat create_response=False as client-side turn taking by @longcw in #6642
• fix(inference): report Ink-2 speech onset when the provider detects it by @russellmartin-livekit in #6630
• fix(inference): don't claim word alignment for models that send none by @russellmartin-livekit in #6629
• fix(stt): send the turn commit when a flush yields no buffered audio by @chenghao-mou in #6598
• fix: account for STT fallback transcript alignment by @chenghao-mou in #6633
• fix(voice): tolerate formatting changes in preemptive transcripts by @swayamg20 in #6667
• fix(elevenlabs): populate SpeechData.confidence from Scribe logprobs by @captainbanan in #6461
• fix(sarvam): saaras:v3 does not support the config/prompt message by @biztex in #6617
• fix(aws): emit streaming STT usage metrics by @swayamg20 in #6653
• fix(voice): avoid dropping turns and resuming early while a speech is paused by @longcw in #6662
• fix(voice): place a realtime user turn where the turn began by @u9g in #6616
• fix(voice): reset tool call created_at at execution start in realtime replies by @u9g in #6615
• fix(room_io): don't re-fire playback_started on resume by @longcw in #6636
• fix(google): release the genai client when the realtime session closes by @biztex in #6643
• voice: strip markdown emphasis in CJK text by @longcw in #6560
• fix(llm): apply tool defaults across all schema shapes; keep response formats sentinel-free by @u9g in #6444
• "ValueError: EOF while parsing a string" during tool calls by @AlvaroBalbin in #6382
• fix(llm): tolerate list-shaped streaming delta content in strip_thinking_tokens by @biztex in #6625
• fix(workflows): return corrective instructions as tool output instead of generate_reply by @u9g in #6372
• fix(anthropic): disable vendor SDK retries by default by @axelray-dev in #6604
• fix(aws): count cached tokens in Bedrock's prompt_tokens by @arthi-arumugam-git in #6663
• fix: disable AEC warmup for outbound SIP calls by @chenghao-mou in #6608
• fix(livekit-agents): propagate prewarm through the STT fallback and stream adapters by @fxhxdxd in #6583
• fix(livekit-agents): forward prewarm to the primary LLM in the fallback adapter by @fxhxdxd in #6582
• bithuman: skip the SDK dependency on Python 3.14 by @longcw in #6561
• bithuman: fail closed on Python versions the SDK has no build for by @u9g in #6614
• fix(examples): pass LemonSlice avatar options via extra_kwargs by @shawnfeldman in #6574
• fix(evals): avoid overriding judge reasoning by @inickt in #6566

Other

• Add homepage voice agent example by @ShayneP in #6422
• docs: correct krisp noise_suppression_level default to 75 by @detail-app[bot] in #6651
• fix(scripts): type-check for linux on every host OS, matching CI by @biztex in #6623
• bump livekit sdk to 1.1.14 by @theomonnom in #6654
• livekit-agents@1.6.8 by @github-actions[bot] in #6659

New Contributors

• @tulanisahil1-beep made their first contribution in #6534
• @AlvaroBalbin made their first contribution in #6382
• @biztex made their first contribution in #6552
• @Oxygen56 made their first contribution in #6531
• @ajayarora1235 made their first contribution in #6587
• @fxhxdxd made their first contribution in #6583
• @ryanl-cerebras made their first contribution in #6593
• @axelray-dev made their first contribution in #6604
• @arthi-arumugam-git made their first contribution in #6663
• @swayamg20 made their first contribution in #6653
• @captainbanan made their first contribution in #6461

Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.6.7...livekit-agents@1.6.8

@livekit/agents@1.6.7

What's Changed

• fix(core): reset user away timer on final STT transcript AGT-3149 by @chenghao-mou in #6478
• Fix stripping of Gemma reasoning markers in streamed output by @C0deRatoR in #6398
• feat: Spatius avatar plugin by @3DRX in #5821
• fix(avatar): skip removal if avatar not found in the room by @longcw in #6486
• fix(endpointing): keep max_delay constant in dynamic endpointing by @chenghao-mou in #6265
• fix(stt): don't clear the primary speaker on an unattributed segment by @chuenchen309 in #6460
• fix(drive-thru): assume quantity of one when unstated to avoid count confirmation by @u9g in #6463
• fix: make AMD settle on endpointing backstop by default by @chenghao-mou in #6496
• fix(voice): include function name in FunctionCallOutput protos by @u9g in #6491
• feat: tag simulation and redaction telemetry by @chenghao-mou in #6497
• Update READMEs to use gemma by @keepingitneil in #6502
• fix(voice): don't drop the realtime turn when the chat-ctx push times out by @ByteMaster-1 in #6445
• feat(voice): expose realtime provider request IDs by @yaodong-shen in #6501
• Add 'zh-HK' to DeepgramLanguages by @devb-enp in #6507
• (gnani): Update authors in pyproject.toml by @tinalenguyen in #6489
• Allow local VAD inference to use a dedicated executor by @basnijholt in #6498
• feat(fishaudio): add normalize_loudness and generation-tuning TTS options by @adrian-cowham in #6520
• fix(voice): don't hold session.run() open on update_agent handoffs with long-lived on_enter by @k-zaher in #6500
• plugins(krisp): add voice isolation telephony mode by @lukasIO in #6510
• Support adaptive interruption for realtime models by @longcw in #6488
• feat(inference): add inference.AvatarSession (avatar provisioning via the gateway) by @shawnfeldman in #6492
• (inference): add fish audio tts by @tinalenguyen in #6529
• slng: rewrite plugin around the Unmute Bridge (v2) by @metehan-slng in #6442
• feat(speechify): streaming TTS with word-level timestamps via official SDK by @luke-speechify in #6327
• fix(openai): recover from stale pooled Responses WebSocket connections by @longcw in #6523
• prewarm LLM to skip connection establishment by @davidzhao in #6484
• fix(otel): skip otel upload error logs if it's disabled by the project setting AGT-3015 by @chenghao-mou in #6167
• chore(spatius): bump spatius to 1.0.4 by @3DRX in #6536
• fix(livekit-agents): require livekit-protocol>=1.1.21 for MetricsRecordingHeader.simulated by @u9g in #6541
• (xai tts): add new voices by @tinalenguyen in #6543
• fix(speechify): restore caller header, close owned httpx client by @luke-speechify in #6542
• fix(plugins-aws): Polly TTS silent on aiobotocore >=3.8.0 (AioStreamingBody has no .content) by @Gooh456 in #6540
• feat(telemetry): include agent_name as an OTLP resource attribute by @jmcclanahan in #6539
• livekit-agents@1.6.7 by @github-actions[bot] in #6532

New Contributors

• @C0deRatoR made their first contribution in #6398
• @3DRX made their first contribution in #5821
• @chuenchen309 made their first contribution in #6460
• @yaodong-shen made their first contribution in #6501
• @basnijholt made their first contribution in #6498
• @k-zaher made their first contribution in #6500
• @shawnfeldman made their first contribution in #6492
• @luke-speechify made their first contribution in #6327
• @Gooh456 made their first contribution in #6540
• @jmcclanahan made their first contribution in #6539

Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.6.6...livekit-agents@1.6.7

livekit-agents@1.6.6

What's Changed

• livekit-plugins-fishaudio: pooled websocket reuse + prebuffer to fix cold-start crackle by @cshape in #6368
• rename reload-addr flag to cli-addr by @davidzhao in #6376
• fix(drive-thru): instruct agent to add stated quantities without confirming by @u9g in #6383
• elevenlabs: deprecate STT model_id in favor of model by @u9g in #6378
• feat(phonic): add additional params support by @qionghuang6 in #6384
• fix(openai): omit unset fields when serializing Responses WS requests by @markine in #6385
• fix(voice): preserve completed tool results in chat context on interruption by @hello-rory in #6349
• elevenlabs: log trace id for failed requests by @longcw in #6396
• frontdesk: end-to-end simulation example with dynamic tool mocking by @theomonnom in #6089
• (xai tts): adjust websocket deadline by @tinalenguyen in #6393
• (examples revamp): remove redundant files by @tinalenguyen in #5523
• Add LiveKit cloud auth to krisp viva plugin by @lukasIO in #5914
• feat(soniox): add endpoint latency adjustment parameter by @FLoppix in #6400
• fix(assemblyai): stop forcing continuous_partials=true for U3-Pro models by @u9g in #6401
• (inference xai): add speed param by @tinalenguyen in #6394
• fix: Instructions.resolve_template double-renders under render(modality) by @longcw in #6414
• fix(openai): stop realtime reconnect loop on fatal server errors (insufficient_quota, invalid_api_key) by @ByteMaster-1 in #6352
• fix(openai): include model in WebSocket URLs by @toubatbrian in #6403
• livekit-plugins-fishaudio: drop the startup prebuffer (fish now streams the opening chunks smoothly) by @cshape in #6423
• (google llm): support mixed tool use by @tinalenguyen in #6416
• fix(google): gate mixed tool use to the Gemini 3 Developer API + build tool config in chat() by @tinalenguyen in #6426
• (google llm): strip thought summaries by @tinalenguyen in #6428
• feat(fishaudio): expose temperature, top_p, bitrate, and normalize TTS options by @cshape in #6456
• feat(assemblyai): add language_codes steering param, enable agent_context_carryover by default by @dlange-aai in #6443
• chore(deps): update dependency mcp to v1.28.1 [security] by @renovate[bot] in #6457
• chore(deps): update dependency nltk to v3.10.0 [security] by @renovate[bot] in #6458
• fix(deps): update dependency json-repair to v0.60.1 [security] by @renovate[bot] in #6413
• fix: avoid mid-word splits in text_transforms.replace by @longcw in #6439
• fix(google): close realtime output streams on generation_complete by @longcw in #6431
• feat(mcp): per-tool options and progress update for MCPToolset by @longcw in #5694
• feat(agent): add update_models() to swap STT/VAD/LLM/TTS at runtime by @longcw in #6440
• fix(voice): recover STT stream after unrecoverable errors by @longcw in #6418
• chore(pii): add job_id in MetricsRecordingHeader by @chenghao-mou in #6326
• voice: add RemoteSession.get_framework_info() client wrapper by @u9g in #6464
• (deepgram stt): fix reconnect on send-side socket drops instead of crashing the session by @tinalenguyen in #6429
• (expressive mode): add to update_options by @tinalenguyen in #6367
• fix(plugins-aws): depend on aiobotocore directly instead of aioboto3 by @xavieralmendros-aily in #6438
• (xai tts inference): add optimize_streaming_latency param, fix typecheck by @tinalenguyen in #6469
• fix(stt): raise retryable APIConnectionError on mid-send socket drops by @tinalenguyen in #6467
• feat(smallestai): use Waves continuous WebSocket streaming protocol for TTS by @harshitajain165 in #6363
• fix(soniox): sentence-buffer tts input, rotate streams on idle and age by @chenghao-mou in #6434
• (assemblyai inference): support agent_context_carryover by @tinalenguyen in #6466
• feat(gnani): update TTS voices/models and STT sample rates + streaming toggle by @tinalenguyen in #6468
• assemblyai: make agent_context_carryover opt-in (off by default) by @tinalenguyen in #6471
• feat(stt): add stt_context_options for keyterms and chat context by @longcw in #6475
• livekit-agents@1.6.6 by @github-actions[bot] in #6476

New Contributors

• @markine made their first contribution in #6385
• @hello-rory made their first contribution in #6349
• @xavieralmendros-aily made their first contribution in #6438

Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.6.5...livekit-agents@1.6.6

@livekit/agents@1.6.5

What's Changed

• fix(google): broaden _is_gemini_3_flash_model to cover all Gemini 3 Flash variants by @balazssandor in #6210
• fix(assemblyai): respect mode preset for turn-silence; remap deprecated u3-pro by @dlange-aai in #6220
• Move frame processor url/token/stream info to client sdk by @1egoman in #5867
• fix(worker): exit when connection fails after max retries by @longcw in #6103
• fix(voice): don't re-issue in-flight tool calls on a new turn by @longcw in #6227
• (phonic): Use stream_ahead_of_real_time mode for Phonic WebSocket and sample rate 24000 by @qionghuang6 in #6236
• fix(anthropic): use non-whitespace trailing dummy to satisfy provider validation by @JasonGordonD in #6216
• fix(deepgram): sync missing STT/TTS API params, fix duplicate languages, add TTS model list by @Namit1867 in #5362
• chore(deps): update dependency msgpack to v1.2.1 [security] by @renovate[bot] in #6243
• docs: fix hotel_receptionist architecture section after refactor by @detail-app[bot] in #6230
• fix(deepgram): remove duplicate redact param by @longcw in #6245
• fix(openai): delete trimmed empty audio items by @he-yufeng in #5887
• fix(openai realtime): discard orphaned response after interrupt or timeout by @longcw in #6244
• fix CI due to realtime_model test by @davidzhao in #6261
• fix: preserve transcription item id by @nightcityblade in #6120
• fix(openai): skip realtime truncate when no audio was played fixes : #6157 by @ByteMaster-1 in #6158
• feat(assemblyai): add language_code streaming param for language steering by @gsharp-aai in #6219
• fix(xai): emit interim transcripts for in-progress status in realtime by @ByteMaster-1 in #6272
• conversation-aware STT recognition (keyterms + chat context) by @longcw in #6039
• Allow LemonSlice avatars to join third party meeting platforms by @aweitz in #6233
• (xai realtime): pass voice parameter by @tinalenguyen in #6276
• feat(certifi): user certifi as a fallback when system store is not present by @chenghao-mou in #6231
• chore(eot): update log message about slow stt transcript by @chenghao-mou in #6179
• fix(fallback_adapter): update stream start time offset in stt fallbackadapter by @chenghao-mou in #6246
• feat(voice): stream tool call status events by @longcw in #6100
• set default shutdown reason by @davidzhao in #6284
• bump worker drain timeout to 1hr by @chenghao-mou in #6286
• feat(rime): support speed_alpha for arcana and coda models by @balazssandor in #6287
• feat(deepgram): apply Flux option updates in-band by @longcw in #6277
• feat: realtime model fallback adapter by @longcw in #6282
• update keyterm detection default model by @longcw in #6289
• fix(eot): tighten eot cancellation by speech acitivity by @chenghao-mou in #6274
• silero: use onnx when explicitely using silero.VAD.load() by @theomonnom in #6290
• fix(openai): emit start/end of speech events for realtime STT AGT-3051 by @chenghao-mou in #6297
• feat(smallestai): word_timestamps for TTS, v4 STT endpoints, eou_timeout fix by @harshitajain165 in #5953
• fix(aws): nest endpointingSensitivity under turnDetectionConfiguration for Nova Sonic 2 by @St-Luciferr in #6201
• switch to blingfire tokenizer by default by @davidzhao in #6301
• feat(plugins-tavus): rename replica/persona to face/pal by @carolin-tavus in #6239
• fix(openai): avoid mutating raw tool schemas for Responses by @VectorPeak in #6305
• feat(openai): add temperature parameter to STT by @longcw in #6307
• fix(liveavatar): preserve provider status errors by @VectorPeak in #6306
• fix incorrect attribution of latency in tts.ttfb by @davidzhao in #6300
• update livekit to 1.1.13 by @davidzhao in #6311
• fix(voice): race between session.run and an in-flight AgentTask handoff by @longcw in #6315
• fix(voice): resolve video task leak in AgentSession by @Mayank-MSJ-Singh in #6318
• Add remove() method to ChatContext for flexible item removal by @3eid in #5131
• fix: expose speech generation errors via SpeechHandle.exception() by @longcw in #6304
• chore(deps): update actions/checkout action to v7 by @renovate[bot] in #6325
• fix(test): stop false-positive "Audio data isn't PCM" failures in TTS tests by @u9g in #6331
• feat(anam) add director notes config by @sr-anam in #6267
• fix(aws): preserve image format for Bedrock images by @VectorPeak in #6328
• (feat) 11Labs STT : Add enable_logging by @Ludobaka in #6273
• feat: expressive mode by @tinalenguyen in #6116
• Add connection pooling for xAI TTS by @godququ5-code in #6317
• voice: output retries for run(output_type=...) by @theomonnom in #6080
• disable openai sdk max_retries by @chenghao-mou in #6341
• Allow direct image uploads for LemonSlice avatars by @jp-lemon in #6343
• remove structured LLM output from voice pipeline by @tinalenguyen in #6345
• remove livekit-blingfire (moved) by @theomonnom in #6320
• (expressive mode): expression marker dialect by @tinalenguyen in #6347
• (expressive mode): fix markup tags impacting pacing by @tinalenguyen in #6350
• feat(core): add plain_text_content property for ChatMessage by @chenghao-mou in #6348
• fix(tts): restart ChunkedStream retries under a fresh request_id by @u9g in #6346
• send ServerInfo to CLI host on start by @davidzhao in #6260
• docs: fix livekit-durable README title copied from removed blingfire plugin by @detail-app[bot] in #6355
• Feat/updating documentation for Gnani by @Gnani-AI-Mintlify in #6004
• feat(soniox): support speed parameter in TTS by @yui0303 in #6339
• fix(azure): recover from STT error cancellations instead of hanging by @apoorva-01 in #6362
• fix: honor IGNORE_ON_ENTER for tools inside a Toolset by @longcw in #6361
• fix: send DTMF to the active session room by @longcw in #6360
• docs(soniox): add speed parameter to TTS example in README by @detail-app[bot] in #6366
• (expressive mode): hide and remove from examples by @tinalenguyen in #6370
• fix(core): flip text content fields AGT-3103 by @chenghao-mou in #6353
• livekit-agents@1.6.5 by @github-actions[bot] in #6371

New Contributors

• @balazssandor made their first contribution in #6210
• @1egoman made their first contribution in #5867
• @JasonGordonD made their first contribution in #6216
• @ByteMaster-1 made their first contribution in #6158
• @aweitz made their first contribution in #6233
• @St-Luciferr made their first contribution in #6201
• @carolin-tavus made their first contribution in #6239
• @VectorPeak made their first contribution in #6305
• @Mayank-MSJ-Singh made their first contribution in #6318
• @3eid made their first contribution in #5131
• @godququ5-code made their first contribution in #6317
• @yui0303 made their first contribution in https://github.com/liveki...

livekit-agents@1.6.4

What's Changed

Bug fixes

Turn Detector

• disable retry for eot errors by @chenghao-mou in #6195
• disable eot connection error retry too by @chenghao-mou in #6196

Turn handling/committing

• fix(agents): keep STT input anchor on the pipeline across handoff by @chenghao-mou in #6207

Warning

users are advised to upgrade if you use agent handoff with STT and are on 1.5.14 ~ 1.6.3

Other updates

Core

• chore(deps): update livekit dependency by @chenghao-mou in #6194
• chore(example): update otel trace example by @chenghao-mou in #6178
• chore(dep): update local-inference dep by @chenghao-mou in #6214
• update livekit version by @chenghao-mou in #6217
• Hotel Receptionist Scenario Expansion by @ShayneP in #6186

Plugins

• chore(elevenlabs): update language_code doc to remove error case by @inickt in #6197
• (xai realtime): set text modality by @tinalenguyen in #6198
• fix(google): clean up STT input frame task by @gyx09212214-prog in #6193
• feat: add Protoface avatar plugin by @protoface-team in #6164

Version update

• livekit-agents@1.6.4 by @github-actions[bot] in #6218

New Contributors

• @gyx09212214-prog made their first contribution in #6193
• @protoface-team made their first contribution in #6164

Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.6.3...livekit-agents@1.6.4

livekit-agents@1.6.3

What's Changed

• fix(slng): expose speed in update_options by @adityajha2005 in #6175
• fix(google): reject tool calls when tool_choice="none" in realtime by @longcw in #6166
• fix(eot): restore timeout behavior for eot inference by @chenghao-mou in #6188
• (assembly ai): add inference params by @tinalenguyen in #6185
• livekit-agents@1.6.3 by @github-actions[bot] in #6189

Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.6.2...livekit-agents@1.6.3

livekit-agents@1.6.2

What's Changed

• feat(assemblyai): add universal-3-5-pro (now default) and Voice Focus streaming params by @dlange-aai in #6119
• feat: add Gemini 3.1 flash TTS support, implement streaming response … by @thorwebdev in #6134
• (google): fix formatting by @tinalenguyen in #6142
• chore(examples): update livekit-agents version to >=1.6 in requirements.txt by @Bobronium in #6150
• fix(openai): default reasoning_effort to 'none' for mini/nano models by @sarvesh4396 in #6148
• feat(assemblyai): add streaming mode (latency/accuracy preset) param by @dlange-aai in #6156
• feat(fishaudio): add speed/volume prosody params and default to s2.1-pro by @cshape in #6159
• docs: add missing google_gemini_tts.py example to voice agents README by @detail-app[bot] in #6155
• feat(soniox): support stt-rt-v5 with endpoint_sensitivity option by @mihafabcic-soniox in #6126
• fix(xai): expose TTS websocket options by @nightcityblade in #6095
• (phonic) hotfix: add a config that lists tools with forbid-speech-after-tool-call by @arunwpm-work in #6143
• fix(workflows): normalize two-digit years in GetDOBTask by @he-yufeng in #6124
• fix(azure): emit STT recognition usage metrics by @nightcityblade in #6161
• (phonic): reuse ws connection across handoffs by @tinalenguyen in #6163
• (google): surface context exhaustion error by @tinalenguyen in #6144
• (inference): add assembly ai model by @tinalenguyen in #6162
• docs(phonic): add forbid_speech_after_tool_call to README config table by @detail-app[bot] in #6168
• feat(phonic): handle user text input by @tinalenguyen in #6169
• livekit-agents@1.6.2 by @github-actions[bot] in #6170

New Contributors

• @thorwebdev made their first contribution in #6134
• @sarvesh4396 made their first contribution in #6148

Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.6.1...livekit-agents@1.6.2