Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 34 additions & 50 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,6 @@

<br />


<!-- <h1 align="center" style="display: none;">OpenRTC</h1> -->

<div align="center">
<strong>A Python framework for running multiple LiveKit voice agents in a single worker process with shared prewarmed models.</strong>
</div>
Expand Down Expand Up @@ -42,8 +39,8 @@ LiveKit worker per agent.
<li><a href="#greetings-and-session-options">Greetings and session options</a></li>
<li><a href="#provider-model-strings">Provider model strings</a></li>
<li><a href="#cli-usage">CLI usage</a></li>
<li><a href="#public-api-at-a-glance">Public API at a glance</a></li>
<li><a href="#project-structure">Project structure</a></li>
<li><a href="#migration-from-legacy-agent_-module-globals">Migration from legacy</a></li>
<li><a href="#contributing">Contributing</a></li>
</ol>
</details>
Expand Down Expand Up @@ -94,13 +91,15 @@ OpenRTC is built to reduce duplicate worker overhead.

## Installation

Install OpenRTC with the common runtime dependencies required for shared
prewarm:
Install OpenRTC from PyPI:

```bash
pip install openrtc[common]
pip install openrtc
```

`openrtc` depends on `livekit-agents[silero,turn-detector]`, so the runtime
plugins required by shared prewarm are installed with the base package.

If you are developing locally, the repository uses `uv` for environment and
command management.

Expand Down Expand Up @@ -220,6 +219,9 @@ For each incoming room, `AgentPool` resolves the agent in this order:
This lets one worker process host several agents while staying compatible with
standard LiveKit job and room metadata.

If metadata references an unknown registered name, OpenRTC raises a `ValueError`
instead of silently falling back.

## Greetings and session options

OpenRTC can play a greeting after `ctx.connect()` and pass extra options into
Expand Down Expand Up @@ -269,7 +271,7 @@ instead of strings.

## CLI usage

OpenRTC includes a small CLI for agent discovery.
OpenRTC includes a CLI for discovery-based workflows.

### List discovered agents

Expand All @@ -293,6 +295,28 @@ openrtc start --agents-dir ./agents
openrtc dev --agents-dir ./agents
```

Both `start` and `dev` discover agents first and then hand off to the underlying
LiveKit worker runtime.

## Public API at a glance

OpenRTC currently exposes:

- `AgentPool`
- `AgentConfig`
- `AgentDiscoveryConfig`
- `agent_config(...)`

On `AgentPool`, the primary public methods and properties are:

- `add(...)`
- `discover(...)`
- `list_agents()`
- `get(name)`
- `remove(name)`
- `run()`
- `server`

## Project structure

```text
Expand All @@ -302,50 +326,10 @@ src/openrtc/
└── pool.py
```

- `pool.py` contains the core `AgentPool` implementation
- `cli.py` provides agent discovery and worker startup commands
- `pool.py` contains the core `AgentPool` implementation and discovery helpers
- `cli.py` provides discovery and worker startup commands
- `__init__.py` exposes the public package API

## Migration from legacy `AGENT_*` module globals

Older prototypes used module-level constants such as `AGENT_NAME` and
`AGENT_STT`. OpenRTC now standardizes on `@agent_config(...)` plus
`AgentPool(...)` defaults.

Before:

```python
AGENT_NAME = "restaurant"
AGENT_STT = "deepgram/nova-3:multi"
AGENT_LLM = "openai/gpt-4.1-mini"
AGENT_TTS = "cartesia/sonic-3"
AGENT_GREETING = "Welcome to reservations."
```

After:

```python
from openrtc import agent_config


@agent_config(
name="restaurant",
greeting="Welcome to reservations.",
)
class RestaurantAgent(Agent):
...
```

Move shared provider settings into the pool:

```python
pool = AgentPool(
default_stt="deepgram/nova-3:multi",
default_llm="openai/gpt-4.1-mini",
default_tts="cartesia/sonic-3",
)
```

## Contributing

Contributions are welcome. Please read [CONTRIBUTING.md](CONTRIBUTING.md)
Expand Down
4 changes: 2 additions & 2 deletions docs/.vitepress/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,10 @@ export default defineConfig({
],
},
socialLinks: [
{ icon: 'github', link: 'https://github.com/mahimairaja/openrtc-python' },
{ icon: 'github', link: 'https://github.com/mahimailabs/openrtc' },
],
editLink: {
pattern: 'https://github.com/mahimairaja/openrtc-python/edit/main/docs/:path',
pattern: 'https://github.com/mahimailabs/openrtc/edit/main/docs/:path',
text: 'Edit this page on GitHub',
},
search: {
Expand Down
142 changes: 129 additions & 13 deletions docs/api/pool.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
## Imports

```python
from openrtc import AgentConfig, AgentPool
from openrtc import AgentConfig, AgentDiscoveryConfig, AgentPool, agent_config
```

## `AgentConfig`
Expand All @@ -17,19 +17,68 @@ class AgentConfig:
llm: Any = None
tts: Any = None
greeting: str | None = None
session_kwargs: dict[str, Any] = field(default_factory=dict)
```

`AgentConfig` is returned from `AgentPool.add()` and represents a registered
LiveKit agent configuration.

## `AgentPool()`
## `AgentDiscoveryConfig`

```python
@dataclass(slots=True)
class AgentDiscoveryConfig:
name: str | None = None
stt: Any = None
llm: Any = None
tts: Any = None
greeting: str | None = None
```

`AgentDiscoveryConfig` stores optional metadata attached to an agent class with
`@agent_config(...)`.

## `agent_config(...)`

```python
@agent_config(
name="restaurant",
stt="deepgram/nova-3:multi",
llm="openai/gpt-4.1-mini",
tts="cartesia/sonic-3",
greeting="Welcome to reservations.",
)
class RestaurantAgent(Agent):
...
```

Use `agent_config(...)` to attach discovery metadata to a standard LiveKit
`Agent` subclass.

## `AgentPool(...)`

Create a pool that manages multiple LiveKit agents in one worker process.

```python
pool = AgentPool()
pool = AgentPool(
default_stt="deepgram/nova-3:multi",
default_llm="openai/gpt-4.1-mini",
default_tts="cartesia/sonic-3",
default_greeting="Hello from OpenRTC.",
)
```

Constructor defaults are used when an agent registration or discovered agent
module omits those values.

## `server`

```python
server = pool.server
```

Returns the underlying LiveKit `AgentServer` instance.

## `add()`

```python
Expand All @@ -41,6 +90,8 @@ pool.add(
llm=None,
tts=None,
greeting=None,
session_kwargs=None,
**session_options,
)
```

Expand All @@ -52,6 +103,12 @@ Registers a named LiveKit `Agent` subclass.
- names must be unique
- `agent_cls` must be a subclass of `livekit.agents.Agent`

### Session options

- `session_kwargs` forwards a mapping of keyword arguments to `AgentSession`
- direct `**session_options` are also forwarded to `AgentSession`
- when the same key appears in both places, the direct keyword argument wins

### Returns

An `AgentConfig` instance for the registration.
Expand All @@ -61,6 +118,30 @@ An `AgentConfig` instance for the registration.
- `ValueError` for an empty or duplicate name
- `TypeError` if `agent_cls` is not a LiveKit `Agent` subclass

## `discover()`

```python
pool.discover("./agents")
```

Discovers Python modules in a directory, imports them, finds a local `Agent`
subclass, and registers it.

Discovery behavior:

- skips `__init__.py`
- skips files whose stem starts with `_`
- uses `@agent_config(...)` metadata when present
- otherwise uses the filename stem as the agent name
- falls back to pool defaults for omitted provider and greeting fields

### Raises

- `FileNotFoundError` if the directory does not exist
- `NotADirectoryError` if the path is not a directory
- `RuntimeError` if a module cannot be imported or defines no local `Agent`
subclass

## `list_agents()`

```python
Expand All @@ -69,6 +150,30 @@ pool.list_agents()

Returns registered agent names in registration order.

## `get()`

```python
pool.get("restaurant")
```

Returns a registered `AgentConfig`.

### Raises

- `KeyError` if the agent name is unknown

## `remove()`

```python
pool.remove("restaurant")
```

Removes and returns a registered `AgentConfig`.

### Raises

- `KeyError` if the agent name is unknown

## `run()`

```python
Expand All @@ -79,22 +184,33 @@ Starts the LiveKit worker application.

### Raises

`RuntimeError` if called before any agents are registered.
- `RuntimeError` if called before any agents are registered

## Routing behavior

`AgentPool` resolves the active agent in this order:

1. `ctx.job.metadata["agent"]`
2. `ctx.job.metadata["demo"]`
3. `ctx.room.metadata["agent"]`
4. `ctx.room.metadata["demo"]`
5. room-name prefix matching such as `restaurant-call-123`
6. the first registered agent

If metadata references an unknown agent, OpenRTC raises `ValueError`.

## Example

```python
from examples.agents.restaurant import RestaurantAgent
from pathlib import Path

from openrtc import AgentPool

pool = AgentPool()
pool.add(
"restaurant",
RestaurantAgent,
stt="deepgram/nova-3:multi",
llm="openai/gpt-5-mini",
tts="cartesia/sonic-3",
pool = AgentPool(
default_stt="deepgram/nova-3:multi",
default_llm="openai/gpt-4.1-mini",
default_tts="cartesia/sonic-3",
)

pool.discover(Path("./agents"))
pool.run()
```
Loading
Loading