# TWITCH-Generator Turn a Twitch VOD into short-form clips automatically. ``` Twitch VOD ──▶ Twitch API (metadata) ──▶ yt-dlp download ──▶ Whisper transcription (faster-whisper) ──▶ three highlight signals, fused: · TEXT Claude reads transcript (funny/emotional/exciting/skillful) · VIDEO Claude Vision scores a frame every 30s (explosion/pvp/ship_damage/landing/bug/action) · AUDIO ffmpeg ebur128 loudness peaks (explosions, shouting) highlight = wA·audio + wT·text + wV·video ──▶ FFmpeg cuts 9:16 clips (burned subtitles) ──▶ upload: YouTube Shorts · TikTok · Instagram Reels ``` The fusion is what makes this work for **Star Citizen**: many great moments are visual with no telling speech. Vision + audio surface those even when the transcript score is zero. Weights and the 30s sampling interval are in `config.yaml` (`vision`, `audio`, `score`). Per-clip component scores (a/t/v) are printed and saved in `moments.json`. Everything runs in one Docker container. Artifacts land in `./data//` (`transcript.json`, `moments.json`, `clips/*.mp4`). ## Quick start (web UI, standalone) ```bash cp .env.example .env # set USERS (logins) + SESSION_SECRET docker compose build docker compose up # web UI -> http://localhost:9443 ``` Log in with a user from `USERS`, then: 1. **Settings** — paste your keys (Anthropic, Twitch, upload tokens). Saved to `data/settings.json`, no need to touch `.env`. Optionally upload the YouTube `client_secret.json`. 2. **Select VODs** — type a streamer login → list their VODs → tick the ones you want (or paste a VOD url/id directly) → *Start processing*. Tick "render only" to skip upload. 3. **Jobs** — table auto-refreshes; click *log* for live pipeline output. Jobs run **serially** (one at a time — Whisper + ffmpeg are heavy). ## Auth Per-user login (username + password) with a signed session cookie. ```env USERS=admin:s3cret,tom:hunter2 # comma-separated user:password pairs SESSION_SECRET= # keep stable, else restart forces re-login ``` > Secrets entered in the UI are stored unencrypted in `data/settings.json`. Always run > behind TLS (the suite proxy provides it in prod). ## Production — co-hosted on the RDOC-Suite box (suite.raumdock.org/vod) This is a **separate compose project** at `/opt/TWITCH-Generator` on LXC 103 (same box / pattern as RDOC-LRC). It does not terminate TLS — the RDOC-Suite `caddy-rdoc` front door (host network, `:9443`) routes `/vod` to it. - `docker-compose.prod.yml` — app listens on `9443` in-container, published loopback-only as `127.0.0.1:9444`, `ROOT_PATH=/vod`. - The Caddy route block lives in `RDOC-Suite/deploy/caddy-rdoc/Caddyfile` (`handle_path /vod*` → `127.0.0.1:9444`). Deploy (inside LXC 103): ```bash cd /opt/TWITCH-Generator docker compose -f docker-compose.prod.yml up -d --build # bring up first (port 9444) cd /opt/RDOC-Suite # then reload the proxy docker compose -f docker-compose.prod.yml up -d caddy-rdoc ``` Then open `https://suite.raumdock.org/vod` and log in. ## CLI (no UI) ```bash # render + upload per config.yaml docker compose run --rm twitch-generator run --vod https://www.twitch.tv/videos/123456789 # render only, no upload (good first test) docker compose run --rm twitch-generator run --vod 123456789 --no-upload ``` The CLI reads the same `data/settings.json` the web UI writes, so keys entered in the UI work for CLI runs too. ## Configuration - `config.yaml` — tunables (Whisper model, clip length, categories, platforms). Mounted read-only. - `.env` — secrets. See `.env.example`. | Var | Needed for | |-----|-----------| | `TWITCH_CLIENT_ID` / `TWITCH_CLIENT_SECRET` | VOD title/duration metadata (download works without it) | | `ANTHROPIC_API_KEY` | LLM moment detection (required) | | `YOUTUBE_CLIENT_SECRETS` / `YOUTUBE_TOKEN_FILE` | YouTube upload | | `TIKTOK_ACCESS_TOKEN` | TikTok upload | | `INSTAGRAM_ACCESS_TOKEN` / `INSTAGRAM_USER_ID` | Instagram Reels upload | ## Performance Whisper is the slow stage. Defaults are CPU/`int8` — fine but slow on long VODs. For GPU: use a CUDA base image, set `whisper.device: cuda` + `compute_type: float16` in `config.yaml`, uncomment the `deploy.resources` block in `docker-compose.yml`, and run with the NVIDIA container runtime. Drop `whisper.model` to `medium`/`small` to trade accuracy for speed. The Whisper model is cached in the `whisper-cache` volume. Vision adds Claude image calls: ~`VOD_minutes·2 / max_frames_per_call` requests (a 3h VOD at 30s ≈ 360 frames ≈ 30 calls). Raise `vision.interval_seconds` or set `vision.enabled: false` to cut cost. Frames/transcript/audio results are cached per VOD under `data//`, so re-runs skip re-computing them. ## Upload caveats (read before expecting magic) - **YouTube** — fully working. First run needs an interactive OAuth consent: ```bash docker compose run --rm -it twitch-generator run --vod ``` Paste the consent code once; the token is saved to `YOUTUBE_TOKEN_FILE` for reuse. Use a Desktop-app OAuth client from Google Cloud Console. - **TikTok** — uses the Content Posting API. Requires an approved TikTok developer app and a user access token; unapproved apps can only post as `SELF_ONLY` (private). Wired up, no-ops with a message if `TIKTOK_ACCESS_TOKEN` is unset. - **Instagram Reels** — Graph API needs a *public* video URL, not a file upload. Host the clip on a CDN/bucket and pass its URL. Needs a Business/Creator IG account linked to a Facebook Page. No-ops with a message if token/URL missing. There is no official, frictionless "just upload a file" API for TikTok or Instagram — both gate behind app review. The code is structured so once you have valid tokens it works. ## Stages `src/twitch.py` download · `src/transcribe.py` Whisper · `src/analyze.py` text/Claude · `src/vision.py` Claude Vision · `src/audio.py` loudness · `src/score.py` fusion · `src/clip.py` FFmpeg · `src/upload.py` platforms · `src/main.py` orchestrator. # TWITCH-Generator