Blaze Livekit plugin integration by HoangPN711 · Pull Request #5050 · livekit/agents

HoangPN711 · 2026-03-09T15:01:57Z

Summary

Add Blaze plugin support for LiveKit Agents.

Changes

add Blaze STT plugin
add Blaze TTS plugin
add Blaze LLM plugin
add Blaze plugin packaging/config helpers
add Blaze plugin README

Motivation

Enable Blaze voice AI services to be used through the existing LiveKit Agents plugin architecture.

Notes

The implementation is isolated under livekit-plugins/livekit-plugins-blaze and follows the existing provider plugin pattern.

CLAassistant · 2026-03-09T15:02:08Z

All committers have signed the CLA.

HoangPN711 · 2026-05-05T23:09:30Z

Hi @tinalenguyen,

This PR adds the Blaze provider integration for LiveKit Agents and keeps the implementation scoped under livekit-plugins/livekit-plugins-blaze.

It follows the existing STT/TTS/LLM plugin structure used by other provider integrations. I’ve also addressed the outstanding review comments on the PR.

Would appreciate a review when you have time. Thanks!

tinalenguyen · 2026-05-13T00:43:59Z

hi @HoangPN711! thank you for the contribution, could you bump the version and add the plugin to this pyproject file

HoangPN711 · 2026-05-13T23:17:54Z

Hi @tinalenguyen , thanks for reviewing this PR!

Updated as requested:

Bumped livekit-plugins-blaze to 1.5.9 (aligned with the current livekit-agents release)
Added blaze = ["livekit-plugins-blaze>=1.5.9"] to livekit-agents/pyproject.toml

Please let me know if any further changes are needed.

CI ruff format --check failed after the streaming STT changes.

ChunkedStream early-return on blank text left the emitter unstarted, so base end_input() raised RuntimeError. Always initialize (and flush) so empty synthesis completes cleanly.

The transcribe endpoint only accepts ModelVersionEnum v1.0|v2.0. stt-async-1.5 belongs to /v1/stt/execute (async jobs). Keep stt-stream-1.5 as the realtime WebSocket default.

AgentSession stt turn-detection opens the user turn on START_OF_SPEECH. Emit it on the first non-empty partial/final and re-arm after END_OF_SPEECH.

WebSocket audio chunks are not sample-aligned. Carry trailing odd bytes and only fade/push even-length PCM so cast('h') never raises.

result.get("confidence", 1.0) still yields None when the key is present with a null value, which crashes %.3f logging. Match the streaming path.

POST /v1/tts/realtime returns 404 on the public gateway — realtime TTS is WebSocket-only. ChunkedStream now follows the same handshake as streaming TTS (auth → speech-start → query → speech-end) so synthesize() works for the plugin tester and any non-streaming callers.

- Point plugin docs at https://blaze.vn - Set authors to LiveKit <hello@livekit.io> - Empty py.typed PEP 561 marker to match sibling plugins - Convert PCM→WAV via rtc.AudioFrame.to_wav_bytes() - Drop custom TTS sentence batching; use framework SentenceTokenizer - Merge all blaze unit tests into tests/test_plugin_blaze.py

BlazeConfig accepts api_token, not auth_token (which is the per-plugin constructor param). Align the package docstring example.

Match llm.py: POST /v1/voicebot-call/{bot_id}/chat-conversion-stream.

Do not early-return when chat context only has system/developer messages. Blaze loads the voicebot prompt server-side; match agents-js by still POSTing messages=[] so generate_reply can speak first.

trunghlt · 2026-07-25T05:11:35Z

Rebased onto latest main (48c179329) to resolve merge conflicts.

Conflict resolutions:

livekit-agents/pyproject.toml: kept main's plugin version pins (>=1.6.7) and retained blaze = ["livekit-plugins-blaze>=1.6.7"]
livekit-plugins-blaze/pyproject.toml: aligned livekit-agents dependency to >=1.6.7
Plugin version set to 1.6.7 to match monorepo

Pushed to Actable-AI/agents (feat/blaze-voicebot-plugin @ 65a0085fa). Light ruff check on blaze paths passed.

trunghlt · 2026-07-25T06:31:21Z

Friendly reminder: this Blaze LiveKit plugin PR is ready for re-review when you have a moment, @tinalenguyen.

All points from the 2026-07-24 pass addressed (README/URLs, authors, py.typed, PCM→WAV, tokenizer, tests)
Rebased onto latest main (plugin aligned to 1.6.7)
CI green (unit/ruff/type-check/blockguard/Devin)
CLA still pending for one committer (trunghlt) — separate from review

Happy to address any further feedback. Thanks!

tinalenguyen · 2026-07-25T09:03:12Z

Hi @HoangPN711, when trying the LLM of the plugin, I kept hitting this error:

livekit.agents._exceptions.APIStatusError: message='Chatbot service error 422: {"detail":[{"type":"list_type","loc":["body"],"msg":"Input should be a valid list","input":{"messages":[]}}]}', status_code=422, retryable=False, request_id=267a6876-363d-4f36-b687-e69d2ce11714, body={"detail":[{"type":"list_type","loc":["body"],"msg":"Input should be a valid list","input":{"messages":[]}}]}

are you able to reproduce this?

Blaze /chat-conversion-stream expects a list body, matching agents-js. Sending {"messages": [...]} caused 422 list_type (Tina's repro with []).

trunghlt · 2026-08-02T23:24:23Z

Hi @tinalenguyen — yes, that 422 is reproducible. Thanks for catching it.

Root cause: POST /v1/voicebot-call/{bot_id}/chat-conversion-stream expects the body to be a JSON array of messages ([{role, content}, ...]), matching agents-js. The Python plugin was sending {"messages": [...]}, so FastAPI rejected it with list_type (including the empty agent-first case you hit: {"messages": []}).

Fix in 3a4d14ca7:

Request body is now the messages list directly (json=messages)
Unit tests updated for empty + non-empty turns

Please try the LLM again on the latest commit. Happy to dig further if anything still fails.

- STT: raise APIConnectionError on unexpected peer close so the framework pump reconnects (normal close ended async-for silently) - TTS: set input_done only after successful input drain so reconnect does not drop remaining tokenizer/input text - Upgrade remote http:// API bases to wss:// so auth tokens are not sent on cleartext WebSockets (localhost keeps ws://)

trunghlt · 2026-08-03T09:14:22Z

Addressed the latest Devin review findings in 5c4708cf594e301dff2785de9eba5301205f052a:

STT silent death after server WS close — unexpected peer close now raises APIConnectionError so recognition reconnects
TTS truncated speech on mid-sentence reconnect — input_done only set after successful input drain
Auth over cleartext WS — remote http:// bases upgrade to wss:// (localhost stays ws://)

All 56 blaze unit tests pass. Thanks!

Keep blaze plugin alongside new bland optional dep; bump blaze to 1.6.8 and regenerate uv.lock. Optional-deps pins follow main (>=1.6.8).

trunghlt · 2026-08-04T08:37:59Z

Resolved merge conflicts with latest main and pushed.

Kept blaze optional extra + workspace source alongside new bland plugin
Aligned optional-deps pins to >=1.6.8 (main)
Bumped livekit-plugins-blaze to 1.6.8
Regenerated uv.lock

Head includes prior LLM body + STT/TTS WS fixes. Happy to address any further feedback.

devin-ai-integration

Devin Review found 2 new potential issues.

View 11 additional findings in Devin Review.

devin-ai-integration · 2026-08-04T08:44:27Z

+        if not text.strip():
+            # Empty result — decide whether to buffer or discard
+            self._pending_empty_count += 1
+            total_pending_duration = pending_duration + segment_duration
+
+            if (
+                self._pending_empty_count <= self._max_pending_segments
+                and total_pending_duration <= self._max_pending_duration
+            ):
+                # Buffer this segment's PCM for the next call
+                self._pending_pcm = pcm_data  # includes already-prepended pending
+                self._pending_sample_rate = sample_rate
+                self._pending_num_channels = num_channels
+                logger.info(
+                    "[%s] STT empty → buffered (count=%d, duration=%.1fs, latency=%.3fs)",
+                    request_id,
+                    self._pending_empty_count,
+                    total_pending_duration,
+                    latency,
+                )
+            else:
+                # Safety limit reached — discard buffer
+                logger.info(
+                    "[%s] STT empty → discarded pending buffer "
+                    "(count=%d, duration=%.1fs, latency=%.3fs)",
+                    request_id,
+                    self._pending_empty_count,
+                    total_pending_duration,
+                    latency,
+                )
+                self._pending_pcm = b""
+                self._pending_empty_count = 0
+


🟡 Leftover speech audio can be mixed into a different conversation's transcription

Unrecognized audio is stashed on the shared speech-to-text object (self._pending_pcm at livekit-plugins/livekit-plugins-blaze/livekit/plugins/blaze/stt.py:366) instead of per-conversation state, so audio left over from one caller can be glued onto the front of another caller's next utterance.
Impact: When one speech-to-text instance serves more than one conversation at a time, a caller can get a transcript containing someone else's speech, or their own words duplicated.

Instance-level pending PCM buffer is mutated by every concurrent recognize call

STT._recognize_impl keeps _pending_pcm, _pending_empty_count, _pending_sample_rate, and _last_recognize_time on the STT object itself (livekit-plugins/livekit-plugins-blaze/livekit/plugins/blaze/stt.py:126-135). A single STT instance is commonly reused across sessions/streams (e.g. via StreamAdapter, FallbackAdapter, or a plugin created once and passed to multiple AgentSessions). Each _recognize_impl call reads self._pending_pcm at line 264-271 and writes it at line 366, with no per-stream scoping and no locking, so two interleaved segments from different streams will prepend each other's PCM. The idle-timeout auto-clear at lines 251-261 also uses a single shared _last_recognize_time, so activity in one stream keeps another stream's stale buffer alive.

A fix would be to move the accumulation state into a per-stream object (or key it by the calling stream) rather than storing it on the shared STT instance.

Was this helpful? React with 👍 or 👎 to provide feedback.

Addressed in b344111: empty-segment PCM accumulation now uses a task-local ContextVar (_RecognizePending) instead of fields on the shared STT instance. Concurrent StreamAdapter/AgentSession tasks each get their own buffer so leftover audio cannot be prepended across conversations. Unit test test_stt_pending_pcm_is_task_local covers isolation.

- Keep empty-segment PCM accumulation in a task-local ContextVar so concurrent streams/sessions cannot mix leftover audio - After TTS starts draining input, do not private-reconnect on WS drop; raise APIConnectionError so framework can replay buffered text - Wrap lines over 100 chars (ruff / project style)

trunghlt · 2026-08-05T08:35:44Z

Addressed the latest Devin review findings in b34411182654c1534246e5e905a81757e3376adb:

STT pending PCM shared across sessions — accumulation is task-local via ContextVar
TTS mid-turn text loss on private reconnect — no private reconnect after input starts; raise for framework replay
Line length > 100 — reformatted

57 blaze unit tests pass.

devin-ai-integration

Devin Review found 1 new potential issue.

View 11 additional findings in Devin Review.

devin-ai-integration · 2026-08-05T08:43:57Z

+        except httpx.TimeoutException as e:
+            raise APITimeoutError(f"LLM request timed out: {e}") from e
+        except httpx.NetworkError as e:
+            raise APIConnectionError(f"LLM network error: {e}") from e
+        except APIStatusError:
+            raise
+        except Exception as e:
+            raise APIConnectionError(f"LLM connection error: {e}") from e


🔴 Chatbot answers can be spoken twice when the connection drops mid-answer

A failure that happens after part of the answer was already delivered is reported as retryable (raise APIConnectionError(...) at livekit-plugins/livekit-plugins-blaze/livekit/plugins/blaze/llm.py:412-419) instead of final, so the whole answer is requested again and the already-delivered part is repeated.

Impact: The user can hear the beginning of the assistant's reply repeated (or a duplicated/garbled reply) whenever the chat connection hiccups mid-response.

Mechanism: framework retry loop replays `_run()` after chunks were already pushed

LLMStream._main_task (livekit-agents/livekit/agents/llm/llm.py:254-300) retries _run() for any APIError whose retryable flag is True. APIError/APIConnectionError/APITimeoutError default to retryable=True (livekit-agents/livekit/agents/_exceptions.py:40,112,119). Chunks already sent via self._event_ch.send_nowait(chunk) (llm.py:396, llm.py:410) are not retracted, so the retry appends a second copy of the response to the same stream.

Other providers guard against this by flipping a retryable flag to False as soon as the first chunk is emitted — see livekit-agents/livekit/agents/inference/llm.py:463-498 and livekit-plugins/livekit-plugins-anthropic/livekit/plugins/anthropic/llm.py:340-349.

Fix sketch: track a local retryable = True that is set to False right after the first send_nowait, and pass retryable=retryable to the raised APITimeoutError/APIConnectionError/APIStatusError.

Prompt for agents

In livekit-plugins/livekit-plugins-blaze/livekit/plugins/blaze/llm.py, LLMStream._run() pushes ChatChunks into self._event_ch as SSE data arrives, but every error it raises (APITimeoutError, APIConnectionError, APIStatusError) uses the default retryable=True. The base LLMStream._main_task in livekit-agents/livekit/agents/llm/llm.py re-invokes _run() for retryable APIErrors, which replays the request and re-emits content the consumer already received, producing duplicated assistant text/speech. Follow the convention used by livekit-agents/livekit/agents/inference/llm.py and the anthropic plugin: keep a local `retryable` flag initialized to True, set it to False the moment the first chunk is sent to self._event_ch (both the tool-call chunk and the content chunk paths), and pass retryable=retryable when constructing the raised errors.

Was this helpful? React with 👍 or 👎 to provide feedback.

This comment was marked as resolved.

Sign in to view

HoangPN711 changed the title ~~feat: add blaze voicebot plugin~~ Blaze Livekit plugin integration Apr 3, 2026

This comment was marked as resolved.

Sign in to view

HoangPN711 force-pushed the feat/blaze-voicebot-plugin branch from 4653f88 to 77090e4 Compare April 18, 2026 15:06

This comment was marked as resolved.

Sign in to view

HoangPN711 force-pushed the feat/blaze-voicebot-plugin branch 3 times, most recently from 2b94037 to e9052a6 Compare May 19, 2026 22:52

This comment was marked as resolved.

Sign in to view

HoangPN711 force-pushed the feat/blaze-voicebot-plugin branch 3 times, most recently from 1cea05c to 2c82c75 Compare June 11, 2026 22:13

HoangPN711 and others added 14 commits July 25, 2026 12:10

style(blaze): ruff format stt.py

5f390a6

CI ruff format --check failed after the streaming STT changes.

fix(blaze): initialize TTS AudioEmitter before empty-text return

8d48dcb

ChunkedStream early-return on blank text left the emitter unstarted, so base end_input() raised RuntimeError. Always initialize (and flush) so empty synthesis completes cleanly.

fix(blaze): use v2.0 for batch STT /v1/stt/transcribe

21a4a7b

The transcribe endpoint only accepts ModelVersionEnum v1.0|v2.0. stt-async-1.5 belongs to /v1/stt/execute (async jobs). Keep stt-stream-1.5 as the realtime WebSocket default.

fix(blaze): emit START_OF_SPEECH in native STT stream

6d81c84

AgentSession stt turn-detection opens the user turn on START_OF_SPEECH. Emit it on the first non-empty partial/final and re-arm after END_OF_SPEECH.

fix(blaze): tolerate odd-length PCM WS frames in TTS fade

65c157c

WebSocket audio chunks are not sample-aligned. Carry trailing odd bytes and only fade/push even-length PCM so cast('h') never raises.

fix(blaze): coerce null STT confidence to 1.0

5d2ee19

result.get("confidence", 1.0) still yields None when the key is present with a null value, which crashes %.3f logging. Match the streaming path.

fix(blaze): preserve 0.0 STT confidence; only default null

e8d3ff3

fix(blaze): honor explicit connect timeout equal to framework default

1425374

fix(blaze): report STT ready-wait timeout as APITimeoutError

34cbe97

fix(blaze): use api_token in BlazeConfig docstring example

0b58fe8

BlazeConfig accepts api_token, not auth_token (which is the per-plugin constructor param). Align the package docstring example.

docs(blaze): fix README LLM endpoint path

fc628e4

Match llm.py: POST /v1/voicebot-call/{bot_id}/chat-conversion-stream.

fix(blaze): POST empty messages for agent-first LLM turns

65a0085

Do not early-return when chat context only has system/developer messages. Blaze loads the voicebot prompt server-side; match agents-js by still POSTing messages=[] so generate_reply can speak first.

trunghlt force-pushed the feat/blaze-voicebot-plugin branch from e042630 to 65a0085 Compare July 25, 2026 05:11

fix(blaze): send LLM chat body as JSON array of messages

3a4d14c

Blaze /chat-conversion-stream expects a list body, matching agents-js. Sending {"messages": [...]} caused 422 list_type (Tina's repro with []).

This comment was marked as resolved.

Sign in to view

merge: resolve conflicts with origin/main (1.6.8)

c664c4a

Keep blaze plugin alongside new bland optional dep; bump blaze to 1.6.8 and regenerate uv.lock. Optional-deps pins follow main (>=1.6.8).

devin-ai-integration Bot reviewed Aug 4, 2026

View reviewed changes

style(blaze): drop unused contextvars import in test

7d06165

devin-ai-integration Bot reviewed Aug 5, 2026

View reviewed changes

Uh oh!

Conversation

HoangPN711 commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Motivation

Notes

Uh oh!

CLAassistant commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

HoangPN711 commented May 5, 2026

Uh oh!

tinalenguyen commented May 13, 2026

Uh oh!

HoangPN711 commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

trunghlt commented Jul 25, 2026

Uh oh!

trunghlt commented Jul 25, 2026

Uh oh!

tinalenguyen commented Jul 25, 2026

Uh oh!

trunghlt commented Aug 2, 2026

Uh oh!

This comment was marked as resolved.

Uh oh!

trunghlt commented Aug 3, 2026

Uh oh!

trunghlt commented Aug 4, 2026

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

trunghlt Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

HoangPN711 commented Mar 9, 2026 •

edited

Loading

CLAassistant commented Mar 9, 2026 •

edited

Loading

HoangPN711 commented May 13, 2026 •

edited

Loading

devin-ai-integration Bot Aug 4, 2026 •

edited

Loading