Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ TeleFuser is a high-performance runtime for world model inference and multimodal

## News 📰

- ✨ **2026-07-27**: Unified streaming on LiveKit with room sessions, worker admission, reconnect-friendly browser
transport, and support for both server-push and bidirectional pipeline contracts.
- ✨ **2026-07-27**: Unified streaming on LiveKit with room sessions, retained multi-session admission, LingBot
chunk-boundary time slicing, reconnect-friendly browser transport, and server-push/bidirectional contracts.
- ✨ **2026-07-22**: **NEW** Added [**LingBot-Video**](examples/lingbot_video/README.md) support for Dense and MoE T2I/T2V/TI2V generation, native four-GPU CFG/SP execution, and in-memory MoE refinement.
- ✨ **2026-07-15**: Added [**LingBot-World v2**](https://github.com/Robbyant/lingbot-world-v2) support for offline generation, interactive WebRTC streaming, and multi-GPU inference.

Expand Down Expand Up @@ -140,9 +140,14 @@ telefuser stream-serve examples/lingbot/lingbot_world_v2_image_to_video_h100.py
--livekit-url ws://127.0.0.1:7880 \
--livekit-api-key devkey --livekit-api-secret secret \
--num-workers 1 --worker-gpu-map 0,1,2,3 \
--max-sessions-per-worker 2 --control-idle-timeout 10 \
--port 8088 --skip-validation
```

This is one four-GPU model worker and one loaded LingBot service instance, not four replicas. It can retain two
independent user sessions; the shared LingBot execution lease runs at most one session chunk at a time and yields at
a chunk boundary after the active controller becomes idle while another session waits.

Terminal 4 — serve the browser controller and proxy its session API:

```bash
Expand Down
2 changes: 1 addition & 1 deletion docs/en/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ telefuser stream-serve examples/lingbot/lingbot_world_fast_image_to_video_h100.p

<div class="tf-link-grid">
<a href="service/"><strong>Service Guide</strong><span>Batch serving, task APIs, and SDK.</span></a>
<a href="stream_server/"><strong>Stream Server</strong><span>LiveKit sessions, media, data topics, and bidirectional control.</span></a>
<a href="stream_server/"><strong>Stream Server</strong><span>LiveKit sessions, retained capacity, LingBot time slicing, and bidirectional control.</span></a>
<a href="stream_scheduler/"><strong>Stream Scheduler</strong><span>Actor ownership, bounded dataflow, lifecycle, metrics, and GPU placement.</span></a>
<a href="benchmark_aiperf/"><strong>AIPerf Benchmark</strong><span>Batch video and LingBot LiveKit workflows.</span></a>
<a href="configuration/"><strong>Configuration</strong><span>Runtime, attention, quantization, and offload settings.</span></a>
Expand Down
15 changes: 7 additions & 8 deletions docs/en/service.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,15 +45,11 @@ telefuser serve \
--port 8000 \
--parallelism 1

# LiveKit-backed world-model streaming
# Set TF_MODEL_ZOO_PATH and PPL_CONFIG["parallelism"] in
# examples/lingbot/lingbot_world_fast_image_to_video_h100.py
telefuser stream-serve examples/lingbot/lingbot_world_fast_image_to_video_h100.py \
# LiveKit-backed server-push streaming; start LiveKit Server separately
telefuser stream-serve examples/stream_server/stream_video_replay.py \
--livekit-url ws://127.0.0.1:7880 \
--livekit-api-key devkey \
--livekit-api-secret secret \
--num-workers 1 \
--worker-gpu-map 0,1,2,3 \
-p 8088 \
--skip-validation
```
Expand Down Expand Up @@ -93,9 +89,10 @@ Use for real-time world models, interactive generation, speech-driven animation,
- LiveKit server-push tracks for progressive video/audio output
- LiveKit bidirectional sessions for interactive control loops
- Stateful sessions with continuous chunk generation
- Worker admission, controller/viewer roles, and reconnect handling
- Worker admission, controller/viewer roles, and LiveKit transport reconnects

See the [Stream Server Guide](stream_server.md) for full streaming documentation.
See the [Stream Server Guide](stream_server.md) for the runtime topology, room roles, retained capacity, execution
lease, GPU placement, lifecycle, and complete local stack.

---

Expand Down Expand Up @@ -307,6 +304,8 @@ The request-response service is intentionally local and single-process by defaul
- `telefuser stream-serve` exposes LiveKit session routes under `/v1/stream/*` and service routes under
`/v1/service/*`. Media and reliable control messages travel through the configured LiveKit deployment. It does
not expose task, file-download, OpenAI-compatible request-response, or direct SDP routes.
- Stream runtime ownership, room topology, capacity, and replica boundaries are defined in the
[Stream Server Guide](stream_server.md); they are intentionally not shared with the request-response runtime.

### Artifact Storage and Cleanup

Expand Down
49 changes: 40 additions & 9 deletions docs/en/stream_scheduler.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,13 @@ stage groups; the streaming scheduler owns bounded, per-session dataflow and per

The scheduler executes a directed acyclic graph of typed artifacts:

```text
external input
|
v
encode -- condition --> denoise -- latent --> decode -- frames --> output
^
|
control
```mermaid
flowchart LR
I[External input] --> E[Encode actor]
E -->|condition| D[Denoise actor]
C[Control] --> D
D -->|latent| V[Decode actor]
V -->|frames| O[Output]
```

Each logical stage is represented by one long-lived actor. Independent actors may run concurrently even when their
Expand All @@ -45,6 +44,37 @@ Edges and outputs have explicit capacities. When a downstream stage cannot accep
backpressure rather than retaining unbounded tensors. Pipeline implementations must therefore treat submission as
admission-controlled, not as an unbounded queue.

## Relationship to stream-service scheduling

The [Stream Server Guide](stream_server.md) owns room, admission, and user-facing lifecycle semantics. This guide
starts after a pipeline session has been admitted. Three schedulers operate at different boundaries and must not be
treated as one queue:

```mermaid
flowchart TB
H[HTTP session request] --> A[Retained-session admission]
A -->|admitted pipeline session| L[LingBot execution lease]
L -->|one whole chunk| O[StreamingPipelineOrchestrator]
O --> E[Encode actor]
O --> D[Denoise actor]
O --> V[Decode actor]

Q1[HTTP admission FIFO] -. waits before .-> A
Q2[Execution-lease FIFO] -. waits before .-> L
Q3[Bounded artifact edges] -. backpressure inside .-> O
```

| Boundary | Owner | Purpose |
| --- | --- | --- |
| Retained-session admission | LiveKit runtime | Assign an HTTP session to capacity on a model worker, or place it in the bounded HTTP admission queue. |
| Cross-session model execution | LingBot service instance | Grant one execution lease so only one retained LingBot session submits a whole chunk at a time. |
| Intra-pipeline dataflow | `StreamingPipelineOrchestrator` | Schedule encode, denoise, and decode stage work with bounded artifacts and per-session ordering. |

`max_sessions_per_worker` changes only the first boundary. It does not change service-instance count, execution
leases, or graph-edge capacities. The second boundary is a LingBot service policy, not a generic orchestrator
feature: its lease surrounds one session chunk, while the orchestrator may still overlap independent stages within
that chunk. Other `BidirectionalService` implementations define their own cross-session policy.

## LingBot Condition Prefetch

LingBot condition encoding is independent of the corresponding control input. The session therefore keeps a fixed
Expand All @@ -65,7 +95,8 @@ session cleanup still runs through the owning actors.

## Actor Ownership and Session Lifecycle

A state-owning worker has exactly one actor owner for its entire lifetime. In particular, one `ParallelWorker` must
A state-owning stage worker has exactly one actor owner for its entire lifetime. This pipeline-level stage worker is
not the stream-server model worker that owns retained-session capacity. In particular, one `ParallelWorker` must
not be invoked directly by a session facade or shared by multiple stage actors. This preserves result ordering and
ensures that cache mutation and release occur in one well-defined execution context.

Expand Down
Loading
Loading