LongShot

Anything in. A finished film out.

LongShot is a local-first production system for turning songs, prose, screenplays, and other source material into controllable long-form AI video. It plans a production as individual shots, keeps continuity assets between them, renders replaceable takes through model backends, and assembles approved clips against the original timeline.

The first usable workflow is a music video:

1. Import an audio file and plain or LRC-timed lyrics.
2. Build a shot plan over the exact song duration.
3. Edit prompts, timing, seeds, guide frames, and reference assets in manifest.json.
4. Render through a ComfyUI API workflow or the zero-GPU test backend.
5. Carry the last frame of each approved shot into the next shot.
6. Assemble approved takes and mux the untouched source audio with FFmpeg.

Why this is not another prompt-to-video demo

Models make clips. LongShot makes a production. The durable artifact is a versioned manifest containing the source timeline, creative direction, continuity rules, shots, render attempts, and approved takes. A failed shot can be rerolled without touching the other three minutes. A new model can replace an old one without rebuilding the screenplay or timeline.

flowchart TD
    A["Song / book / script"] --> B["Source adapter"]
    B --> C["Production manifest"]
    C --> D["Shot planner"]
    D --> E["Guide frames + references"]
    E --> F["Model backend"]
    F --> G["Take review"]
    G -->|reroll| F
    G -->|approve| H["Timeline assembly"]

Loading

Quick start

Requirements: Python 3.11+, FFmpeg/FFprobe, and optionally ComfyUI.

python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

longshot new-music \
  --name "My Song" \
  --audio ./song.wav \
  --lyrics ./lyrics.lrc \
  --style "epic old-world Spanish romance, deep reds and moonlit blues"

# Validate the complete pipeline without loading a model.
longshot render my-song --backend mock
longshot assemble my-song

# Launch the local studio.
longshot serve

Open http://127.0.0.1:8420.

ComfyUI rendering

Export a ComfyUI workflow in API format, replace the values you want LongShot to control with the tokens below, and render it:

longshot render my-song \
  --backend comfyui \
  --workflow ./my-wan-workflow-api.json \
  --comfyui-url http://127.0.0.1:8188

Supported recursive workflow tokens:

Token	Value
{{PROMPT}}	Shot prompt
{{NEGATIVE_PROMPT}}	Shot negative prompt
{{WIDTH}}, {{HEIGHT}}	Project dimensions
{{FPS}}, {{FRAMES}}	Timing converted to frames
{{SEED}}	Stable per-shot seed or reroll seed
{{START_FRAME}}, {{END_FRAME}}	Uploaded ComfyUI guide-image filenames

The repository intentionally does not ship a giant model-specific workflow. Workflows change much faster than the production manifest, so they are user-supplied adapters.

Recommended 48 GB model stack (July 2026)

Role	Model	Why it belongs
Default local shots	Wan 2.2 TI2V-5B	Officially supports 720p at 24 FPS on 24 GB; Apache-2.0; leaves headroom for planning and upscaling.
Quality / specialized shots	Wan 2.2 A14B	Strong T2V, I2V, speech-to-video, and character animation; use optimized FP8/GGUF ComfyUI workflows on 48 GB.
Native long continuation	LongCat-Video	MIT code; native T2V, I2V, and continuation; designed for minutes-long generation.
Infinite continuity experiment	Stable Video Infinity	MIT; Wan 2.1/2.2 error-recycling continuation with published multi-minute examples.
Synchronized audio/video	LTX-2.3	22B audio-video generation, fast distilled path, LoRA training, and ComfyUI support. Its community license is source-available, not OSI-open, and requires a paid license at $10M+ annual revenue.
Performance / lip-sync shots	LongCat-Video-Avatar 1.5	Audio-driven long human performance with video continuation.
Agent research reference	ViMax	MIT multi-agent idea/script/novel-to-video system; useful reference, not a dependency.

See docs/MODELS.md for the selection policy and licensing distinctions.

Project layout

longshot/
  api.py                 FastAPI application
  cli.py                 Local production commands
  models.py              Versioned production manifest
  planner.py             Music-video shot planner
  timeline.py            LRC/plain-text timing and media probing
  service.py             Resumable production orchestration
  render.py              Continuity frames and FFmpeg assembly
  backends/
    comfyui.py            Generic tokenized ComfyUI API adapter
    mock.py               Deterministic zero-GPU render backend
docs/
  ARCHITECTURE.md
  MODELS.md
  ROADMAP.md

Current boundary

Version 0.1 deliberately makes the production spine real before adding autonomous agents. It can create, persist, plan, render, resume, propagate continuity frames, and assemble a music video. The rule-based planner is a deterministic baseline; structured local/cloud LLM planners, reference image generation, beat analysis, take scoring, and book/screenplay source adapters are the next milestones.

Development

ruff check .
pytest

LongShot is MIT licensed. Model weights and third-party backends retain their own licenses.