Repository navigation
vid2context 0.2.0 #12
nirholas
announced in
Announcements
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Install or update:
npx vid2context@latest(CLI) andnpx vid2context-mcp@latest(MCP server). Both are on npm as 0.2.0.Added
Playlist view in the web app: rows show each video's title and length from the list page (new optional
VideoRef.titleanddurationSeclisting hints, read by the YouTube lister; also inGET /itemsand MCPlist_items). Repeated entries are left out. A video waiting on a retry shows a live countdown and "Retry now", which cancels that job and starts it again. Stop and Retry now cancel the server job (DELETE /jobs/:id) instead of leaving it to retry for nobody.Job
progressevents carryretryAtwhile a failed attempt waits to be retried.X provider: a third lane for reading posts, x.com's GraphQL
TweetResultByRestIdwith a guest token, the lane xactions runs for xactions.app/video. Tried after syndication and FxTwitter, only whenX_BEARER_TOKEN(orTWITTER_BEARER_TOKEN) is set;X_GRAPHQL_QUERY_IDfollows X when it rotates the operation id. Recorded fixtures and a gated live test.Fixed
Speech-to-text decoder loops no longer cost the speech under them. A 9-minute interview on the keyless OVH lane came back with "issuance of" repeated 39 times over 45 seconds of real speech: the long-audio path never ran the repetition guard, and the guard alone would only have cut the junk back. Now every piece is checked; a looped piece is sent to the other rungs, then split in two at new cut points (once per piece), and only when every retry still loops is it cut back, with a
loop-keptattempt saying the speech is missing. The single-request path re-transcribes a looped answer in two pieces the same way, and the stitched long-audio transcript gets the repetition guard too. See docs/long-audio.md, "Decoder loops".The keyless OVH rung declares its 60 s audio cap (
maxDurationSec, now a field ofSttRung), so long audio is split before the first upload instead of after refused ones, and audio of known length over every rung's cap goes straight to the long-audio path.Long-audio attempts no longer list every cooldown skip while a chunk waits out rate limits: the 429s stay, and one
rate-limit-waitattempt gives the total wait.X links work on Cloudflare Workers (the public vid2context.com instance). Workers' fetch sends no user-agent, and without one X's syndication endpoint answers 400 and FxTwitter 401, so every X link failed there while Node, which always sends one, worked. The X provider now names its client on every lane.
Long X videos can be transcribed on the keyless lane. X audio is m4a, which no built-in splitter cuts, and the ffmpeg splitter in
@vid2context/ffmpegwas never passed to the pipeline, so anything over 60 s ended in "no splitter for audio/mp4".Registrynow carries optionalsplitters, tried after the built-in MP3 and WAV ones, and the Node registry addsffmpeg-silence, so the CLI, HTTP server and MCP server all get it.Long audio no longer stalls behind a rung with a bad key. A rejected key puts a rung on a 10-minute cooldown, and the chunk wait loop counted those cooldowns as rate limits, so a chunk waited its whole 10-minute budget and then failed. Cooldowns now remember their cause (
cooldownReason), and only rate limits, 5xx errors and timeouts are waited for. A chunk planned for keyed rungs that all failed is split again for the length cap of the rungs that are left (OVH's 60 s).A rung skipped up front for a declared limit now counts as a too-large refusal, so audio still reaches the long-audio path when the only other rungs fail on auth (seen live: a bad Cloudflare key plus OVH skipped for length ended in "every rung failed").
Web playlist view: the progress line no longer counts failures as done ("1 of 3 done, 1 failed"), and long status messages wrap instead of being cut off.
The no-transcript hint follows what was tried: it suggests installing yt-dlp only when no source could download the audio, and otherwise says whether the download was blocked or speech-to-text failed.
Web app Captions page (
?view=captions, in the sidebar and linked from every result): list every caption track of a video, preview one, save it as SRT, WebVTT, text or JSON, save all tracks as one .zip (tracks the platform refuses are listed, not fatal), or translate into a language the video has no track for. A playlist or channel saves every video's captions in one .zip. Captions only: no media is downloaded or transcribed.Web app: in-browser Whisper runs in a Web Worker, so transcribing a long file no longer freezes the page and Cancel really stops it. The worker reads the app's 16 kHz WAV without Web Audio, which workers lack.
Stall protection in the rung chain: a hosted rung that does not answer within
VID2CONTEXT_RUNG_TIMEOUT_MS(default 3 minutes) is abandoned, recorded astimeout, skipped for a minute, and the next rung gets its turn. Rate-limit cooldowns are capped at 10 minutes, so one hugeRetry-Aftercannot disable a provider for a day.Web app: a run shows the server's progress and, after 30 seconds, offers other ways to get the result (the import panel, or downloading the video) without stopping the run; whichever finishes first is shown. Runs without the user's own keys go through a server job, so long transcriptions are not cut off by proxy response timeouts.
Video links inside a playlist (
watch?v=...&list=..., what YouTube's share button gives) now surface the playlist instead of dropping it. The video stays the default. The web app shows "This video is part of a playlist: <title> (N videos)" above the result and on the Download page, with a link that opens every video. The CLI takes--playlistonlist,pack,bundleanddownloadand prints a note otherwise;GET /itemsreturnslistand takesplaylist=1; MCPlist_itemsreturnslistand takesplaylist: true. Providers report it through the optionalSourceProvider.matchList,Registry.resolvereturns it aslist, andexpandInputtakes{ list: true }. Mixes are never offered.Failures are never dead ends.
suggestRecoveries(corerecovery.ts) turns any error into ranked ways to still get the result: download the audio and transcribe it, use a local file, get the captions from your browser, paste or upload a transcript, add a free key, wait and retry, switch to transcription, run locally. The HTTP API returns them asrecoveryin every error body, the CLI prints them with ready-to-run commands (also in--json), and MCP tool errors include them.Web app: a failed run opens a "Let's get this another way" panel with those options usable in place (including an inline form to add a key and retry), and the raw error, hint and attempts folded under "Technical details".
Provider keys per request:
X-Vid2Context-Keyslets a caller retry with its own key, used for that request only and never stored. Only registered provider credential names are accepted;GET /healthlists them underrequestKeys.VID2CONTEXT_REQUEST_KEYS=0turns it off. In the web app, sending saved keys to the server is opt-in (Settings, or the recovery form).Rung cooldowns are keyed by credential, so one caller's rejected or exhausted key never cools a rung down for others.
@vid2context/core: data model, provider contracts, the free-first rung chain with cooldowns and key pools, the video-to-context pipeline (auto,captions,sttandvideomodes),textandjsonformatters, in-memory storage.@vid2context/registry: provider registries generated per runtime (node, workers, browser).@vid2context/cli: minimal command line withtextandjsonoutput, transcript modes and language selection.youtubeprovider: URL matching, metadata, description chapters, creator and auto captions. No API key.media-fileprovider: local audio and video files as speech-to-text sources.stt-openai-compatibleprovider: local servers, Groq, OVH (keyless) and OpenAI speech-to-text.Runnable examples in
examples/(library, CLI, HTTP API, MCP, LangChain.js, Python, LangChain and LlamaIndex) with a docs section indocs/examples/. Code blocks and the output of offline examples are synced bynpm run examples:docsand checked bynpm test;LIVE=1runs the live examples too.llm-openai-compatibleprovider: Ollama, LM Studio, local servers, Groq, Cerebras, Mistral, GitHub Models, Gemini, OVH and Pollinations (keyless), and OpenAI.Full changelog: v0.1.0...v0.2.0
This discussion was created from the release vid2context 0.2.0.
All reactions