Skip to content

[FEATURE] Add ABot-World-0-5B-LF single-GPU interactive pipeline - #26

Merged
Kaimary merged 4 commits into
Tele-AI:mainfrom
youngmagician114514:feat/abot-world-single-gpu
Aug 7, 2026
Merged

[FEATURE] Add ABot-World-0-5B-LF single-GPU interactive pipeline#26
Kaimary merged 4 commits into
Tele-AI:mainfrom
youngmagician114514:feat/abot-world-single-gpu

Conversation

@youngmagician114514

@youngmagician114514 youngmagician114514 commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Description

Add a native TeleFuser single-GPU integration for the official
ABot-World-0-5B-LF checkpoint, including causal interactive generation,
a direct browser controller, and LiveKit streaming support.

Motivation

TeleFuser did not provide an ABot-World interactive pipeline. This change
enables image-conditioned, action-controlled world rollout while retaining
causal KV, scheduler, RNG, and VAE temporal decode state across control blocks.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Performance improvement
  • Code refactoring
  • Documentation update
  • Other

Changes Made

  • Add ABotWorldDiT, ABotWorldDenoisingStage, and TeleFuser pipeline classes.
  • Add persistent single-session interactive generation.
  • Add six sink latents and a twelve-latent rolling KV window.
  • Add bounded RoPE positions for long-horizon sessions.
  • Add direct HTTP browser demo with ordered 12 FPS playback and FIFO backpressure.
  • Add LiveKit streaming transport for interactive ABot-World serving.
  • Add unit, checkpoint-contract, and 30-block GPU smoke tests.

Testing

  • ABot-World unit tests pass
  • LiveKit service tests pass
  • Manual browser testing performed
  • Benchmarks added/updated

97 related CPU tests pass.

Test commands:

python -m pytest \
  tests/unit/pipelines/abot_world \
  tests/unit/models/test_wan22_video_vae.py \
  -q
ABOT_WORLD_MODEL_ROOT=/path/to/ABot-World-0-5B-LF \
python -m pytest -m filesystem \
  tests/integration/test_abot_world_checkpoint.py \
  -v
CUDA_VISIBLE_DEVICES=0 \
ABOT_WORLD_MODEL_ROOT=/path/to/ABot-World-0-5B-LF \
ABOT_WORLD_TEST_IMAGE=/path/to/initial.png \
python -m pytest -m "gpu and slow" \
  tests/integration/test_abot_world_smoke.py \
  -v -s

Checklist

  • Code follows the project's coding standards (ruff)
  • Pre-commit hooks pass (pre-commit run --all-files)
  • All repository tests pass (pytest tests/)
  • New tests added for new functionality
  • Documentation updated
  • Commit messages are clear and descriptive
  • PR title follows the convention: [TYPE] Brief description

Related Issues

N/A

Additional Notes

  • Scope is limited to one GPU and one retained interactive session.
  • LiveKit streaming transport is supported; the direct HTTP path is retained for local testing.
  • The six-sink KV policy and bounded RoPE positions are intentional stability behavior.
  • Full repository-wide pytest tests/ and pre-commit hooks were not run.

GPU Architecture Support

  • SM80 (Ampere, Ada Lovelace)
  • SM90 (Hopper H100)
  • SM100+ (Blackwell)

No new architecture-specific CUDA kernels are added.

Performance Impact

No formal benchmark is included. The runtime preloads model weights, retains
session state, consumes frames at 12 FPS, and applies producer backpressure
instead of dropping frames during normal playback.

@lzx1413

lzx1413 commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

How about serving this world model as the lingbot world with livekit.

@youngmagician114514

Copy link
Copy Markdown
Contributor Author

The ABot-World LiveKit integration and tests have been pushed.

Validation:

  • 97 related CPU tests passed
  • Real H100 continuous 30-chunk smoke test passed
  • Ruff and formatting checks passed

@Kaimary

Kaimary commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

LGTM, accept the pr

@Kaimary
Kaimary merged commit c99e1c4 into Tele-AI:main Aug 7, 2026
5 checks passed
@lzx1413 lzx1413 mentioned this pull request Aug 7, 2026
31 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants