OpenSpline

Add a live face to any voice AI agent in five lines. Turn a portrait and generated audio into synchronized video and speech on your own GPU.

Try Online (no GPU or installation needed)

• Bring your agent. Keep your prompts, tools, microphone, and voice provider.
• Python first, Node supported. Stream audio or use a ready-made connector.
• Ready to display. Built-in viewer, React component, and web component.
• Self-hosted. One active session per GPU worker; add GPUs for concurrency.

Quick start · Connectors · Configuration · Node SDK · Frontend · GPU workers

Avatars created with OpenSpline Cloud. Click a preview or Try online to open the live avatar.

Nikola Tesla	Meera	Lisa
Try online	Try online	Try online
Acharya	Elon Musk	Ye
Try online	Try online	Try online

Create, share, and embed your own avatar at openspline.dev, or explore the Cloud app source and embed examples.

Run the following command in a linux device/WSL2 with NVIDIA GPU:

curl -fsSL https://raw.githubusercontent.com/openspline/openspline/main/install.sh | sh

The installer creates a private Python environment and downloads the Low and High models.

sh ~/.local/share/openspline/run.sh

Open http://localhost:7860 to access the demo.

Install the client package where your agent runs:

pip install openspline

Stream generated audio to a portrait:

from openspline import Openspline
client = Openspline()
async with client.avatar("portrait.jpg", quality="low") as avatar:
    print(avatar.viewer_url)
    await avatar.stream(agent.audio_stream(), sample_rate=24000)

stream() accepts generated PCM audio and returns after playback finishes. WAV and MP3 files are also supported.

Avatars keep generating fresh idle video while listening and after interruptions, using near-silent audio with very low-level noise for model conditioning. Speech and idle video share the same motion state; idle clips are never looped or replaced with an animated portrait. The noise is not sent to speakers. Generation keeps at most one idle batch ahead of playback, and speech takes priority. The GPU stays active throughout a connected session; smooth continuous motion requires inference to keep up with playback.

GPU compatibility: PyTorch uses CUDA 12.8 for Blackwell (including RTX 50-series) and CUDA 12.6 for older NVIDIA GPUs. Pre-Ampere cards use FP32, which needs more VRAM and may be slower. AMD and Apple GPUs are not supported by this backend. Blackwell and pre-Ampere inference still need hardware validation.

These snippets plug into an existing async agent. Openspline owns output playback; disable your previous speaker output to avoid playing audio twice. Provider credentials stay on your backend. Each connector links to a complete runnable example.

Install pip install openai==2.54.0. Wrap your existing OpenAI Realtime connection; tools and transcripts still reach your event handler.

from openspline import Openspline
from openspline.integrations import OpenAIRealtime

async with Openspline().avatar("portrait.jpg") as avatar:
    print(avatar.viewer_url)
    await avatar.wait_for_viewer()
    adapter = OpenAIRealtime(avatar, connection)
    try:
        async for event in adapter.wrap(connection):
            await handle_event(event)  # your existing handler
    finally:
        adapter.close()

Run the example: OPENAI_API_KEY=... python examples/python/openai_realtime.py portrait.jpg.

Install pip install 'livekit-agents[openai]==1.8.3'. Add the avatar before starting your existing AgentSession; playback uses your normal LiveKit frontend.

from openspline import Openspline
from openspline.integrations.livekit import AvatarSession

avatar = AvatarSession("portrait.jpg", client=Openspline(quality="low"))
await avatar.start(agent_session, room=ctx.room)
await agent_session.start(room=ctx.room, agent=agent)

Run the example: PORTRAIT=portrait.jpg python examples/python/livekit_agent.py dev. Set OPENAI_API_KEY, LIVEKIT_URL, LIVEKIT_API_KEY, and LIVEKIT_API_SECRET. The native service includes LiveKit support.

Install pip install google-genai==1.75.0. Forward every audio part from your existing Gemini Live session, including interruption and turn-end events.

from openspline import Openspline
from openspline.integrations import GeminiLive

async with Openspline().avatar("portrait.jpg") as avatar:
    print(avatar.viewer_url)
    await avatar.wait_for_viewer()
    async for event in GeminiLive(avatar).wrap(live_session.receive()):
        await handle_event(event)  # your existing handler
    await avatar.end_turn()

Run the example: GEMINI_API_KEY=... python examples/python/gemini.py portrait.jpg. Repeat the receive loop for subsequent turns; set GEMINI_LIVE_MODEL to override the model.

Connect an existing ElevenLabs agent WebSocket. No additional provider SDK is needed. Pass decoded events, including the initial metadata, so the adapter uses the agent's negotiated audio format.

import json
from openspline import Openspline
from openspline.integrations import ElevenLabsAgents

async with Openspline().avatar("portrait.jpg") as avatar:
    print(avatar.viewer_url)
    await avatar.wait_for_viewer()
    adapter = ElevenLabsAgents(avatar)
    async for message in connection:
        event = json.loads(message)
        await adapter.handle(event)
        await handle_event(event)  # keep your input, pong, and tool handlers

The adapter supports PCM and μ-law audio, flushes final audio, and handles interruptions. It preserves transcripts and tool events for your existing handler. If attaching after connection setup, pass audio_format="pcm_24000" (or the negotiated format) to the constructor. Node exports ElevenLabsAgents from @openspline/node/integrations, with await adapter.handle(event) and an optional format as the second constructor argument.

Python example: ELEVENLABS_AGENT_ID=... python examples/python/elevenlabs_agents.py portrait.jpg. Node example: ELEVENLABS_AGENT_ID=... node examples/node/elevenlabs_agents.mjs portrait.jpg. Set ELEVENLABS_API_KEY for private agents; the examples request a signed URL on the backend. Each plays one response; the demo supports microphone conversations. Configure the agent's client events to include audio, interruption, and agent_response_complete. Custom client tools and required dynamic variables need your application's handlers and initiation data; the demo uses the agent's saved configuration.

Use any TTS provider or agent with an async PCM16 audio iterator. stream() buffers audio, flushes the final block, and waits for playback to finish.

from openspline import Openspline
client = Openspline(quality="low")
async with client.avatar("portrait.jpg") as avatar:
    print(avatar.viewer_url)
    await avatar.stream(agent.audio_stream(), sample_rate=24000)

For callbacks, await avatar.send_audio(chunk, sample_rate=24000), then await avatar.end_turn() when speech ends. Call await avatar.interrupt() on barge-in. Float32 uses encoding="pcm_f32le"; encoded streams use encoding="mp3". File example.

More connectors: OpenAI Agents, Pipecat, Google ADK, Vocode, and TEN. Python examples have locked dependencies; Vocode uses its own environment.

Pass settings directly when initializing, or reuse a typed config object:

from openspline import Openspline, OpensplineConfig

client = Openspline(url="http://localhost:7860", quality="low")
config = OpensplineConfig(quality="high", timeout=300, viewer_timeout=90)
client = Openspline(config=config)  # a dict works too

Explicit Python parameters override the config object; explicit settings override environment defaults. avatar(..., quality="high") overrides that session only. Invalid configuration fails before connecting. JSON Schema describes Python settings, with #/$defs/node and #/$defs/server for Node and worker configuration. Only avatar.session should be passed to a browser.

Use Node 20+ and install the SDK with one command:

npm install @openspline/node

import { Openspline, type OpensplineConfig } from "@openspline/node";
const config: OpensplineConfig = { quality: "low", timeout: 180000 };
const client = new Openspline(config);
await using avatar = await client.avatar("portrait.jpg");
console.log(avatar.viewerUrl);
await avatar.stream(agent.audioStream(), { sample_rate: 24000 });

Node examples: run npm ci there, then node universal.mjs portrait.jpg speech.wav. Without await using, call avatar.close() in finally.

Pass avatar.session from your backend to your frontend. Keep the publisher token on the backend.

import { Avatar } from "@openspline/react";

<Avatar session={session} />

For vanilla JS, Vue, or Svelte, call registerAvatarElement() from @openspline/browser/element, then set document.querySelector("openspline-avatar").session = session. Both players handle connection, autoplay permission, and cleanup. React also exports useAvatar().

Install the component you need with npm install @openspline/react or npm install @openspline/browser.

The built-in demo uses WebSocket playback through the server's HTTP port, including SSH port forwarding. For the same transport in your frontend, pass {...session, transport: "websocket"}; components otherwise use WebRTC.

Choose GPUs directly in the demo: one for Low, or one or more for High. The picker shows GPU names and memory; your selection is applied when you start and saved for restarts by the curl install. You can also set OPENSPLINE_GPUS=0,1 when installing. Explicit CUDA_VISIBLE_DEVICES limits are respected. /readyz reports the active devices and inference ranks.

Edit ~/.local/share/openspline/workers.yaml (example). Each worker loads weights once and serves one session at a time; device assignments cannot overlap.

workers:
  - { id: low-0, quality: low, devices: [0] }
  - { id: low-1, quality: low, devices: [1] }

The demo switches quality automatically for a single worker. For SDK sessions or multiple workers, configure a worker with quality: high, devices: [0, 1] to use a GPU pair, then run sh ~/.local/share/openspline/run.sh --prepare before starting. Busy workers return a typed CapacityError; an unconfigured quality returns ConfigurationError. IDs are relative to CUDA_VISIBLE_DEVICES when set. For multiple workers, remove OPENSPLINE_GPU, OPENSPLINE_GPUS, and OPENSPLINE_QUALITY overrides from the installed .env and configure devices in workers.yaml.

Set OPENSPLINE_HOST, OPENSPLINE_PORT, OPENSPLINE_PUBLIC_URL, and provider credentials in the installed .env. Use HTTPS for remote microphone access. WebRTC playback additionally needs reachable UDP ports or TURN configured in ice_servers. API reference: http://localhost:7860/docs. Health: /readyz; metrics: /metrics.

MIT. The vendored inference code retains its Apache-2.0 license and third-party notices; model weights have separate terms.

Openspline builds on open research and infrastructure, with thanks to the teams and contributors behind:

• SoulX-FlashHead for the audio-driven talking-head models and inference code that power avatar generation.
• LiveKit Agents for the real-time agent framework that Openspline integrates with to bring avatars into voice conversations.
• Wan for video generation research and VAE components used in the inference backend through SoulX-FlashHead.
• LTX-Video for video autoencoder components used in the inference backend through SoulX-FlashHead.

Openspline carries this work forward with a self-hosted runtime for interactive avatars: synchronized audio and video streaming, continuous idle motion, interruption handling, GPU worker orchestration, and a shared API across voice providers. Its Python and Node SDKs, agent connectors, and embeddable players make these capabilities available to developers building their own conversational experiences.

Pass an optional background file path or bytes when creating the avatar. Images stay still; videos loop silently during speech and idle playback. Upload a PNG portrait with its background removed: compositing uses the original portrait's alpha matte (a fixed silhouette, not automatic per-frame segmentation). Opaque portraits retain their original background. The background fills the portrait's aspect ratio with a centered cover crop.

async with Openspline().avatar("portrait.png", background="scene.mp4") as avatar:
    # Use any existing audio connector with this avatar.
    await avatar.play_file("speech.wav")

const avatar = await new Openspline().avatar("portrait.png", {
  background: "scene.jpg", // MP4/WebM or image; path or Uint8Array
});

LiveKit's Python AvatarSession and Node LiveKitAvatarSession, Pipecat's AvatarProcessor, and TEN's extension properties also accept background. OpenAI Realtime/Agents, Gemini Live, Google ADK, ElevenLabs, Vocode, and universal audio adapters use the background configured on their shared avatar session. Browser, React, web component, WebRTC, and native LiveKit viewers receive the already-composited video without extra configuration.

The HTTP API accepts an optional background file alongside portrait and quality in POST /v1/sessions. Limits: 25 MiB per background, 16 megapixels, and video up to 60 seconds / 120 fps. Video decoding retains only the current and next frame. Background selection lasts for the session; omit the field to keep the existing behavior.