Release demo-vid-mcp v0.3.0 · sandraschi/demo-vid-mcp

Fleet Intro Video Pipeline v0.3.0

Subtitles & Closed Captions: Automatically generates WebVTT and SRT sidecars with native player subtitle track support.
Visual Cursor Highlights: Playwright cursor click-ripples on every button and action.
Aspect Ratio & Resolution Presets: Support for 16:9 desktop landscape and 9:16 vertical mobile reels in 720p / 1080p.
Automated Poster Extraction: Frame capture generating poster thumbnail images for instant video preview.
Background Audio Bed: Ambient music bed support with voiceover ducking.
Persistent Job Queue: File-backed background queue with live polling, queue scheduling, and restart recovery.
Dead Port Resilience: Graceful skipping and clear UI warning callouts for backend port 11134 and Ollama 11434.

2026-09-04 update — installer replaced (was completely broken)

The installer originally attached to this release never produced a working app. In order,
found and fixed:

Backend crashed on launch (ModuleNotFoundError: No module named 'fastapi') — the
PyInstaller entry point used a string-based uvicorn.run("module:app", ...) reference
that PyInstaller's static analyzer couldn't trace, so FastAPI/Starlette were never
bundled.
The app killed itself ~1.5s after every launch — free_port()'s cleanup logic did
Stop-Process -Name 'demo-vid-mcp-native' to clear stale processes, but that command
runs from inside the very process it's trying to clean up after, so on every fresh
launch it matched and killed itself (Windows process-name matching has no "not me").
Even past that, the window froze on launch (Windows showed it as
"Demo Vid MCP (Not Responding)") — the backend-startup logic ran directly on the
thread responsible for pumping the window's message loop, blocking the UI for the
whole time it took to spawn.

All three verified fixed by directly instrumenting the installed app (process lifetime,
Windows responsiveness state, and window class via pywinauto) rather than assuming a
green CI run meant it worked.

2026-09-04 update 2 — the app ran, but core features didn't work

Found by actually using the running app, not just checking it stayed open:

The whole webapp couldn't reach its own backend. Every page used
fetch("/api/...") with a relative path. In dev mode Vite's proxy makes that work;
in the packaged app the frontend is served from tauri://localhost, so a
relative fetch never reached http://127.0.0.1:11134 at all — this is why
Settings/Chat showed "No provider detected" even with Ollama and LM Studio running.
Fixed with an apiUrl() helper applied to every fetch and every video/poster/subtitle
element across all 11 pages.
Video generation failed — playwright-capture.js was never bundled into the
installer, and its path-resolution logic assumed a dev-repo layout that doesn't
exist inside the frozen backend. Bundled the script, added robust multi-candidate
path resolution, and a NODE_PATH fallback so a global Playwright install is still
discoverable from the installed resources/ folder.
Auto-drafted scripts navigated to fabricated URLs and 404'd — page routes were
guessed by lowercasing the component filename (ApiDocsPage.tsx → /apidocs)
instead of reading the target's real router config (/swagger, in one real case
hit). Now parses the actual <Route path="X"> table from the target's App.tsx.
Real intro title cards + explanatory narration — text_overlay (and most of the
Choreography editor's other step types) was accepted by validation but never
actually rendered by the capture script. Implemented it for real, and auto-drafted
scripts now open on a title card and narrate what each page does (from the
target's README "Webapp" table) instead of "The dashboard page."

2026-09-04 update 3 — recording worked, composition crashed

FFmpeg crashed compositing the final MP4 (FFmpeg exit 3221225477 — that's
0xC0000005, STATUS_ACCESS_VIOLATION, a genuine crash, not a normal error exit).
The drawtext filter burning the title into the video had no fontfile= set, so it
fell back to fontconfig's default-font lookup — Windows has no fonts.conf, and any
fontconfig-enabled FFmpeg build (the Gyan full_build from winget install ffmpeg,
which this repo's own ONBOARDING.md recommends) crashes outright on that lookup
instead of erroring cleanly. Fixed by passing an explicit fontfile= pointing at a
real Windows system font (Segoe UI, falling back to Arial), bypassing fontconfig
entirely; composes without the title overlay (not crashing) if neither font is found.
Composed videos showed 0:00 duration in the player — no -movflags +faststart,
so FFmpeg wrote the moov atom (container metadata, including duration) at the end
of the file; browsers can't report duration or seek until that's read. Fixed.

Verified end-to-end for real, not just in isolation: recorded and composed an actual
demo video for arxiv-mcp through the full pipeline — real webapp footage, title
overlay burned in, poster frame and WebVTT/SRT subtitles all produced from a genuine,
seekable .mp4. See BUILD_LOG.md in the repo for full root-cause detail on every item
above.

2026-09-04 update 4 — videos too short, no way to prioritize important pages

User feedback after watching a generated video: 15s total was too short, and important
pages (arxiv-mcp's search and depot) got the same one-liner treatment as everything
else — no way to tell the auto-drafter "linger here."

Per-page detail levels. Every page can now be classified Skip / Show / Detail —
keyword heuristics by default (search/depot/dashboard/chat/generate → Detail;
logs/swagger/settings → Skip), overridable per page via a new Skip/Show/Detail
checklist on the Generate page. Detail pages get 6-8s of dwell time and substantive
narration instead of a bare "The X page," even for repos whose README has no
## Webapp purpose table to draw from.
duration_target now reflects actual content — sum(step wait times) + 5s
with a 30s floor, replacing a step-count guess that capped out around 15s for
typical repos.
Found in passing while wiring page selection through the background queue:
enqueue() stored aspect_ratio/resolution on the job but the queue worker never
passed them to the generator — the Generate page's aspect-ratio/resolution
selectors were silent no-ops for anything sent to the background queue. Fixed
alongside.

Verified end-to-end in the rebuilt installer: reinstalled, confirmed a healthy
single backend+native process pair, hit the new page-listing endpoint directly against
the packaged backend, then drove the actual Tauri window (screenshot-verified via
pywinauto, since the packaged webview is tauri://localhost and not reachable from a
normal browser) — selected arxiv-mcp on the Generate page, saw the Skip/Show/Detail
checklist render with the correct defaults, and confirmed a drafted script came back
with duration_target: 55 (up from ~15s).

2026-09-04 update 5 — longer scripts immediately hit a stale hard timeout

Clicking "Generate Now" on the update-4 build failed with "Playwright capture timed out after 45s" - the very first real-world use of the new detail-level feature.

recorder.py's capture subprocess had a flat 45s timeout left over from when
scripts totaled ~15s of wait time. Once detail-level pages started producing 50s+
of narrated content, the capture was still legitimately running (browser launch +
real page navigation + the narrated wait time itself) when the timeout killed it.
_capture_timeout() now scales with the script's own content
(sum(wait times) + 60s buffer), extracted as a small pure function with direct
unit tests.

Verified with a real, non-mocked generation through the exact POST /api/generate
endpoint the Generate page's "Generate Now" button calls, against the rebuilt and
reinstalled packaged app: recording completed in 54.5s (previously would have hit the
old 45s cap and failed exactly as reported), composition succeeded, and the final
.mp4 in the app's own data/videos/arxiv-mcp/ came back at 52.2s duration - up
from the original ~15s, with the search/depot/chat/dashboard pages actually narrated in
detail rather than skimmed past.

(The voiceover stage still reports "speech-mcp not configured" in this test run -
that's expected and non-fatal, not a bug: no SPEECH_MCP_URL is set, so the pipeline
falls back to a silent video with timed subtitles rather than failing the whole
generation. Set SPEECH_MCP_URL in the app's .env to enable narrated voiceover.)

2026-09-04 update 6 — voiceover was silently broken from day one

.env was never actually loaded, anywhere, ever — despite every fleet-service URL
(SPEECH_MCP_URL etc.) being read via os.getenv(), and the app's own error message
saying "Set SPEECH_MCP_URL in .env", nothing in the codebase called load_dotenv().
A correctly filled-in .env (present the whole time, SPEECH_MCP_URL included) was
simply never read, so voiceover always failed with "speech-mcp not configured" even
with speech-mcp healthy and running. The packaged app's resources/ compounded this by
only ever getting .env.example copied in, never a real .env.

config.py now loads .env before its dataclass field defaults are evaluated,
trying the dev repo root, then the packaged app's install dir, then its resources/
folder. build.ps1 seeds a real resources/.env from .env.example on build
(skipped if one already exists, so a customized .env survives upgrades), and it's
now included in the NSIS bundle.

Verified with a real generation through the exact endpoint the Generate page's
button calls, against the rebuilt and reinstalled app: stages.voiceover came back
"Voiceover generated: 14 segments" (previously "speech-mcp not configured" on every
attempt), and the final .mp4 carries a genuine AAC audio track alongside the video
(confirmed with ffprobe) at 66.3s — an actually-narrated video, not a silent one.

2026-09-04 update 7 — real background music, dedicated Speech/Music pages

Choreography's "Desktop" checkbox had no effect - tracked in state, rendered as
a checkbox, never written into the generated script, so _is_desktop_capture()
never saw it. One-line fix.
Real background-music generation, not a stub. The "Background Audio Bed... with
voiceover ducking" bullet above described a feature that never actually existed -
bg_music was a field nothing read. Now calls songgeneration-mcp (which
aggregates Lyria 3 Pro / ACE-Step / Stable Audio / SongGeneration-Studio behind one
API, trying each in order - the right integration point, not a separate Lyria
client) and mixes the result under the voiceover with genuine sidechain ducking via
FFmpeg (music volume drops while narration speaks). stems-mcp, what the music
checkbox was previously wired to, does audio separation - the wrong tool
entirely.
New Speech and Music pages - voice picker (heart/sky/adam, shared with
Generate/Choreography via localStorage), health checks, and a real
generate-and-preview flow for both, backed by two new proxy endpoints
(/api/speech/preview, /api/music/preview) so the previews work without the
webview needing direct cross-origin access to speech-mcp/songgeneration-mcp.

Verified: both new health endpoints respond correctly in the packaged app (speech
connected, music cleanly "not detected" - no crash) and both pages render correctly in
the live Tauri window. The speech preview endpoint was verified twice - directly (95KB
real WAV, HTTP 200) and by running the exact fetch→blob→<audio> JS pattern used in the
page against the live backend from an actual browser (metadata resolved
duration=2.16s correctly). A full generation afterward with voice: "sky" completed
successfully end-to-end and the saved script confirmed the override took effect - no
regression from the rewiring. Not verified: the actual music-generation success path
itself, since no songgeneration-mcp backend was running locally to generate a real track
against - only its failure/unconfigured path and the mixing logic (tested with a
stand-in audio file) are confirmed.

2026-09-04 update 8 — CSP had no media-src, all audio/video silently blocked

"Speech page says Connected but the preview button does nothing." Root cause:
the CSP declared connect-src/img-src but no media-src, which falls back to
default-src 'self' - 'self' is the packaged app's own origin
(tauri://localhost), covering neither blob: (Speech/Music preview audio) nor
http://127.0.0.1:11134 (Depot/Detail's <video> player). Both silently broken the
whole time - CSP blocks a media src with no exception and no visible error.
Diagnosed by reproducing the exact JS flow from a real browser dev-server session
against the live packaged backend (worked perfectly there, proving the code was
right and the packaged app specifically was the problem). Fixed:
media-src 'self' blob: http://127.0.0.1:11134 added to the CSP; also added
onError handlers so a future media failure is visible instead of silent.

Verified: clicked Preview on the Speech page in the rebuilt, reinstalled app -
audio now plays through to completion. This also fixes Depot/Detail's video player,
silently affected by the identical gap.

2026-09-04 update 9 — "let's get fancy": real transitions + sound effects

Real per-page video transitions via vfx-mcp. Recording now captures one clip
per page visit instead of one flat continuous capture, then joins them with real
crossfade/wipe/slide transitions (vfx-mcp) when available, falling back to a plain
hard-cut concat otherwise. Single-page scripts are unaffected - still exactly one
clip, same as before.
Real sound effects via sfx-mcp. Choreography's "Sound Effect" step type existed
in the UI with no backend behind it since before this session - now resolves a CC0
sound (FreeSound) for each sfx step and mixes it in at its exact timestamp.
Getting vfx-mcp working at all took five rounds of debugging, each fix revealing
the next: its /mcp endpoint was completely unreachable (wrong mount + missing
FastAPI lifespan wiring - fixed in that repo), its "crossfade" transition used a
FFmpeg filter name that doesn't exist and was missing the offset needed to blend
at the actual clip boundary rather than frame 0, a wrong-JSON-key bug on this side
was swallowing the real error message, that error was also being truncated to
just FFmpeg's useless version banner, and the true root cause underneath all of
that: relative output paths resolved against vfx-mcp/sfx-mcp's own process cwd
instead of this app's - now resolved to absolute before crossing the process
boundary. .env.example's SFX_MCP_URL/VFX_MCP_URL also pointed at the wrong
ports entirely (pre-existing copy-paste errors, never caught before now).

Verified for real: a generation with an sfx step and three page transitions came
back "Stitched 4 clips (transitions=True)" in the backend's own logs through the
rebuilt, reinstalled packaged app - no fallback, real crossfades the whole way through.
sfx-mcp itself wasn't running (no FreeSound key configured), so its step correctly
degraded to a skipped-with-warning rather than failing the generation - the
graceful-degrade path was exercised for real, not just unit-tested.

Still an open, deliberate scope decision, not resolved here: the packaged app's
video recording depends on Node.js + a reachable Playwright install as an external
prerequisite (same category as FFmpeg) — nothing bundles Playwright/Chromium
(~300MB+) into the installer for a fully zero-setup experience. Dedicated SFX/VFX
settings pages (mirroring Speech/Music) also aren't built yet - reachable only via
Choreography's raw script fields for now.

All assets on this release have been replaced with the fixed build.

Navigation Menu

demo-vid-mcp v0.3.0

Choose a tag to compare

Sorry, something went wrong.