feat: Add Camb.ai TTS plugin #4442
Conversation
|
|
- Implement TTS class with streaming audio synthesis - Support MARS models (mars-8, mars-8-flash, mars-8-instruct) - Add voice listing functionality via list_voices() - Support configurable voice parameters (speed, language, voice_id) - Include PCM, WAV, FLAC, and ADTS output formats - Add examples and documentation
eRuaro
force-pushed
the
feature/camb-ai-tts-plugin
branch
from
56da073 to
b42239a
Compare
|
demo with |
Replace raw aiohttp HTTP calls with camb-sdk client: - Use AsyncCambAI client for TTS streaming - Use SDK's voice_cloning.list_voices() for list_voices() - Replace aiohttp dependency with camb-sdk>=1.5.0
eRuaro
force-pushed
the
feature/camb-ai-tts-plugin
branch
from
aac2584 to
f4128ab
Compare
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
📝 WalkthroughWalkthroughAdds a new Camb.ai plugin package: packaging and workspace entries, plugin registration and logging, async voice listing, typed TTS models/options, streaming TTS implementation with chunked audio emitter and LiveKit error translation, README, and version metadata. Changes
Sequence Diagram(s)sequenceDiagram
participant Client as LiveKit Client
participant TTS as Camb TTS
participant Stream as ChunkedStream
participant CambAI as Camb.ai API
participant Emitter as LiveKit Audio Emitter
Client->>TTS: synthesize(text)
TTS->>TTS: validate options / ensure session
TTS->>Stream: create stream task
Stream->>CambAI: POST /tts-stream (headers, payload)
CambAI-->>Stream: stream audio chunks
Stream->>Emitter: push(chunk) / flush
Stream-->>Client: stream completion
rect rgba(200,100,100,0.5)
CambAI-->>Stream: non-200 / timeout / connection error
Stream->>Stream: translate -> APIStatusError / APITimeoutError / APIConnectionError
Stream->>Emitter: emit error / close
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~30 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. 📜 Recent review detailsConfiguration used: Organization UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
✏️ Tip: You can disable this entire section by setting Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Fix all issues with AI agents
In `@livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/__init__.py`:
- Around line 54-84: The AsyncCambAI client instance created as client is never
closed, potentially leaking connections; wrap its usage in an async context
manager (with AsyncCambAI(...) as client:) or ensure you call await
client.aclose() in a finally block around the voice_cloning.list_voices() logic
so the client is always closed even on ApiError before raising the
APIStatusError; update the block that creates AsyncCambAI and the try/except to
guarantee cleanup.
In `@livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/tts.py`:
- Around line 153-169: The update_options method currently updates
self._opts.speech_model but doesn't update the instance _sample_rate set in
__init__, causing mismatched sample rate when switching models; modify
update_options (in the TTS class) so that when model is not None you also set
self._sample_rate to the correct rate for that model (use the same mapping/logic
used in __init__—or extract it into a helper like _get_sample_rate_for_model or
a MODEL_SAMPLE_RATES dict—and call that to compute and assign the new sample
rate whenever self._opts.speech_model is changed).
- Around line 179-183: The aclose method is a no-op while
_close_client_on_cleanup may be true and _client may be an httpx AsyncClient
created internally; implement explicit async cleanup by checking if
self._close_client_on_cleanup and self._client is not None, then await the
appropriate close method (prefer AsyncCambAI.aclose() if the client exposes it,
otherwise await self._client.aclose() for an httpx.AsyncClient); after awaiting
close set self._client to None to avoid reuse and swallow or re-raise any
expected exceptions as appropriate.
In `@livekit-plugins/livekit-plugins-camb/README.md`:
- Around line 32-45: The examples use top-level await which causes a
SyntaxError; wrap the snippets in an async entrypoint (e.g., define async def
main(): ... call asyncio.run(main())) so calls to await stream.collect() and
await list_voices() execute inside an async function, and update the examples
that reference TTS(), TTS.synthesize(), stream.collect(), and list_voices()
accordingly to show asyncio.run(...) invoking the async main.
In `@livekit-plugins/livekit-plugins-camb/TESTING.md`:
- Around line 11-23: Update TESTING.md to remove or correct the broken example
script references: either delete the "Simple demo" and "Comprehensive test
suite" sections that point to examples/simple_demo.py and
examples/test_camb_tts.py, or replace those commands with the correct existing
example scripts or test commands in the repo (use the actual script filenames or
test runner commands present in the repository). Ensure the updated instructions
run successfully (adjust the cd/uv run invocation if necessary) and that the two
referenced paths (examples/simple_demo.py and examples/test_camb_tts.py) are no
longer mentioned unless those files are added.
🧹 Nitpick comments (1)
livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/tts.py (1)
223-231: Consider explicit handling for unknown output formats.The fallback to
audio/aacfor unrecognized formats (line 231elsebranch) may silently produce incorrect MIME types. If a new format is added toOutputFormatbut not handled here, it would default to AAC.♻️ Suggested explicit mapping
# Determine MIME type based on output format - if self._opts.output_format in ("pcm_s16le", "pcm_s32le"): - mime_type = "audio/pcm" - elif self._opts.output_format == "wav": - mime_type = "audio/wav" - elif self._opts.output_format == "flac": - mime_type = "audio/flac" - else: # adts or other - mime_type = "audio/aac" + mime_type_map = { + "pcm_s16le": "audio/pcm", + "pcm_s32le": "audio/pcm", + "wav": "audio/wav", + "flac": "audio/flac", + "adts": "audio/aac", + } + mime_type = mime_type_map.get(self._opts.output_format, "application/octet-stream")
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (9)
livekit-plugins/livekit-plugins-camb/README.mdlivekit-plugins/livekit-plugins-camb/TESTING.mdlivekit-plugins/livekit-plugins-camb/livekit/plugins/camb/__init__.pylivekit-plugins/livekit-plugins-camb/livekit/plugins/camb/log.pylivekit-plugins/livekit-plugins-camb/livekit/plugins/camb/models.pylivekit-plugins/livekit-plugins-camb/livekit/plugins/camb/py.typedlivekit-plugins/livekit-plugins-camb/livekit/plugins/camb/tts.pylivekit-plugins/livekit-plugins-camb/livekit/plugins/camb/version.pylivekit-plugins/livekit-plugins-camb/pyproject.toml
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Format code with ruff
Run ruff linter and auto-fix issues
Run mypy type checker in strict mode
Maintain line length of 100 characters maximum
Ensure Python 3.9+ compatibility
Use Google-style docstrings
Files:
livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/version.pylivekit-plugins/livekit-plugins-camb/livekit/plugins/camb/log.pylivekit-plugins/livekit-plugins-camb/livekit/plugins/camb/models.pylivekit-plugins/livekit-plugins-camb/livekit/plugins/camb/__init__.pylivekit-plugins/livekit-plugins-camb/livekit/plugins/camb/tts.py
🧬 Code graph analysis (3)
livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/models.py (2)
livekit-plugins/livekit-plugins-elevenlabs/livekit/plugins/elevenlabs/tts.py (1)
voice_id(527-528)livekit-plugins/livekit-plugins-fishaudio/livekit/plugins/fishaudio/tts.py (1)
output_format(126-127)
livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/__init__.py (3)
livekit-agents/livekit/agents/_exceptions.py (1)
APIStatusError(45-81)livekit-agents/livekit/agents/plugin.py (2)
Plugin(13-56)register_plugin(31-36)livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/tts.py (1)
TTS(50-183)
livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/tts.py (4)
livekit-agents/livekit/agents/types.py (1)
APIConnectOptions(54-88)livekit-agents/livekit/agents/utils/misc.py (2)
is_given(25-26)shortuuid(21-22)livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/models.py (1)
_TTSOptions(45-53)livekit-agents/livekit/agents/tts/tts.py (1)
TTSCapabilities(47-51)
🪛 LanguageTool
livekit-plugins/livekit-plugins-camb/TESTING.md
[uncategorized] ~195-~195: The official name of this software platform is spelled with a capital “H”.
Context: ...if CAMB_API_KEY is set in secrets) See .github/workflows/tests.yml and `.github/workf...
(GITHUB)
[uncategorized] ~195-~195: The official name of this software platform is spelled with a capital “H”.
Context: ... See .github/workflows/tests.yml and .github/workflows/ci.yml for details.
(GITHUB)
🔇 Additional comments (10)
livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/log.py (1)
15-17: Logger setup looks good.Clear, minimal module-level logger initialization.
livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/models.py (1)
20-61: Types and defaults are clear and consistent.Nice, concise model/output definitions with explicit defaults.
livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/version.py (1)
1-1: Version constant looks good.livekit-plugins/livekit-plugins-camb/pyproject.toml (1)
1-48: Packaging config looks solid.livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/__init__.py (2)
1-28: LGTM on imports and constants.The license header, imports, and gender mapping are well-structured. The
GENDER_MAPcovers standard ISO/IEC 5218 gender codes (0, 1, 2, 9).
87-94: LGTM on plugin registration.The
CambPluginclass correctly extendsPluginand passes the required arguments. Registration at module import time follows the expected pattern.livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/tts.py (4)
15-48: LGTM on imports.The imports are well-organized with clear grouping: standard library, third-party, livekit-agents, and local modules. The
replacefrom dataclasses is correctly used for copying_TTSOptionsinChunkedStream.
50-131: LGTM on TTS constructor.The constructor is well-documented with Google-style docstrings. Key observations:
- Sample rate auto-detection from model with sensible fallback (22050 Hz)
- Proper API key validation with support for future Vertex AI credentials
- Correct ownership tracking for HTTP session cleanup
186-196: LGTM on ChunkedStream initialization.Using
dataclasses.replace()to create an immutable snapshot of options is good practice — it prevents race conditions ifupdate_options()is called during synthesis.
260-275: Error handling is comprehensive.The error mapping correctly translates:
httpx.ConnectError→APIConnectionErrorhttpx.TimeoutException→APITimeoutErrorApiError→APIStatusErrorwith status code and request IDThe defensive re-raise check (lines 271-274) ensures LiveKit exceptions aren't wrapped in another
APIConnectionError.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/__init__.py
Outdated
Show resolved
Hide resolved
livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/tts.py
Outdated
Show resolved
Hide resolved
|
thanks! taking a look now. |
|
Note that the Camb SDK's AsyncCambAI doesn't support:
Which were suggested by CodeRabbit. This fix properly cleans up resources by accessing the internal httpx client directly via |
davidzhao
left a comment
davidzhao
left a comment
There was a problem hiding this comment.
is the API just a HTTP post? if so we would prefer not to pull in additional SDKs and just use the APIs with the provided http client.
it introduces issues like lifecycle cleanup and version maintenance overhead.
you'd also want to add it to the following toml files:
livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/__init__.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/tts.py`:
- Around line 96-101: The constructor currently allows skipping CAMB API key
when credentials_info or credentials_file are present, but those paths aren't
implemented; change the validation to require an API key always by raising
ValueError whenever self._api_key is falsy (ignore is_given(credentials_info)
and is_given(credentials_file) in the check). Also update or remove the warning
that mentions credentials_info/credentials_file (the log near lines after the
validation) so it no longer suggests those options are supported; reference
self._api_key, is_given, credentials_info, credentials_file and the warning log
call in your edits.
🧹 Nitpick comments (4)
livekit-plugins/livekit-plugins-camb/pyproject.toml (1)
29-32: Unclear purpose ofvertexoptional dependency.The
vertexoptional dependency withgoogle-authappears unrelated to a Camb.ai TTS plugin. Whiletts.pyhas placeholder parameters forcredentials_infoandcredentials_filewith comments about "Future Vertex AI", there's no actual google-auth usage in this plugin. If Vertex AI integration is planned for the future, consider either:
- Removing this optional dependency until it's actually used, or
- Adding a comment explaining the roadmap for this feature
Unused optional dependencies can confuse users and bloat installations.
livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/tts.py (3)
182-190: Default MIME type assumption is fragile.The
elseclause assumes any unmatched format isadtsand assignsaudio/aac. IfOutputFormatis extended inmodels.pywith new formats, they would silently receive an incorrect MIME type.Consider being explicit:
🔧 Suggested fix
if self._opts.output_format in ("pcm_s16le", "pcm_s32le"): mime_type = "audio/pcm" elif self._opts.output_format == "wav": mime_type = "audio/wav" elif self._opts.output_format == "flac": mime_type = "audio/flac" - else: # adts + elif self._opts.output_format == "adts": mime_type = "audio/aac" + else: + raise ValueError(f"Unsupported output format: {self._opts.output_format}")
215-218: Hardcoded total timeout ignoresconn_options.timeout.The
sock_connecttimeout correctly usesconn_options.timeout, but thetotaltimeout is hardcoded to 60 seconds. For long text inputs or slow networks, this may not be appropriate. Consider making the total timeout configurable or deriving it fromconn_options.🔧 Suggested approach
timeout=aiohttp.ClientTimeout( - total=60, + total=max(60, self._conn_options.timeout * 6), # Scale with connect timeout sock_connect=self._conn_options.timeout, ),Alternatively, consider adding a dedicated read timeout parameter to the TTS constructor.
229-234: Consider using server-provided request ID for tracing.Line 225 captures the server's
x-request-idheader for error responses, but successful responses use a locally generated UUID (line 230). Using the server's request ID (when available) would improve traceability for debugging and support requests with Camb.ai.🔧 Suggested fix
output_emitter.initialize( - request_id=utils.shortuuid(), + request_id=resp.headers.get("x-request-id") or utils.shortuuid(), sample_rate=self._tts._sample_rate, num_channels=NUM_CHANNELS, mime_type=mime_type, )
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/__init__.pylivekit-plugins/livekit-plugins-camb/livekit/plugins/camb/tts.pylivekit-plugins/livekit-plugins-camb/pyproject.toml
🚧 Files skipped from review as they are similar to previous changes (1)
- livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/init.py
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Format code with ruff
Run ruff linter and auto-fix issues
Run mypy type checker in strict mode
Maintain line length of 100 characters maximum
Ensure Python 3.9+ compatibility
Use Google-style docstrings
Files:
livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/tts.py
🧠 Learnings (1)
📓 Common learnings
Learnt from: CR
Repo: livekit/agents PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-01-16T07:44:56.353Z
Learning: Follow the Plugin System pattern where plugins in livekit-plugins/ are separate packages registered via the Plugin base class
🧬 Code graph analysis (1)
livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/tts.py (5)
livekit-agents/livekit/agents/_exceptions.py (2)
APIConnectionError(84-88)APITimeoutError(91-95)livekit-agents/livekit/agents/types.py (1)
APIConnectOptions(54-88)livekit-agents/livekit/agents/utils/misc.py (2)
is_given(25-26)shortuuid(21-22)livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/models.py (1)
_TTSOptions(45-53)livekit-agents/livekit/agents/utils/http_context.py (1)
http_session(40-51)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: type-check (3.13)
- GitHub Check: type-check (3.9)
- GitHub Check: unit-tests
🔇 Additional comments (3)
livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/tts.py (3)
133-150: LGTM!The
update_optionsmethod correctly updates the sample rate when the model changes (line 148), which addresses the previously flagged issue. The implementation is clean and straightforward.
160-161: LGTM!The no-op
aclose()is appropriate now that the implementation usesutils.http_context.http_session()for session management. The session lifecycle is handled by the LiveKit agents framework rather than this class, so there's nothing to clean up here.
241-253: LGTM!The error handling correctly translates aiohttp exceptions to the appropriate LiveKit API exception types, and properly re-raises already-translated exceptions without double-wrapping.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
davidzhao
left a comment
davidzhao
left a comment
There was a problem hiding this comment.
lg, just need to fix the version and add it here:
you'd also want to add it to the following toml files:
https://github.com/livekit/agents/blob/main/livekit-agents/pyproject.toml
https://github.com/livekit/agents/blob/main/pyproject.toml
livekit-plugins/livekit-plugins-camb/livekit/plugins/camb/version.py
Outdated
Show resolved
Hide resolved
|
awesome! how about the docs? is there a separate repo for that? and when would this go live? thanks! |
4 participants
Summary
Adds a new TTS plugin for Camb.ai powered by their MARS model family.
Features
Files Added
livekit-plugins/livekit-plugins-camb/- Complete plugin packagelivekit/plugins/camb/tts.py- Main TTS implementationlivekit/plugins/camb/models.py- Type definitionslivekit/plugins/camb/__init__.py- Plugin registration and list_voices()README.md- DocumentationTESTING.md- Testing guideTest plan
Summary by CodeRabbit
New Features
Documentation
Chores
Style
✏️ Tip: You can customize this high-level summary in your review settings.