Caption retiming and Sequence clocks: small tested examples for agent-driven edits #13001
the-media-vampire
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I maintain Veyrul, a saved media-knowledge service for agents, and built two small local examples for agents working in Remotion. I'm looking for a real edit to test the limits against, not asking anyone to adopt a new renderer.
1. Captions after cuts, repeated clips and rational speed changes. The subtle bug was rounding twice: 11/3 seconds rounded to 3,666,667 microseconds puts a boundary just after frame 110 at 30 fps. The adapter derives frame boundaries from retained source/cut identities before frame rounding. Code, inputs, commands and limits.
2. Composition-timed captions inside a Sequence. A Sequence starting at frame 24 sees local frame 0 there. The helper converts and clips global intervals instead of retiming the source again. A later controlled reuse caught an additional condition: for nested Sequences, intersect every ancestor's active window; summing offsets alone is insufficient. Small helper and exact scope.
Our checks used Remotion 4.0.537 on macOS arm64: two silent diagnostic edits, 351 decoded frames checked for source timing/caption visibility, 18 caption OCR samples, and 13 rendered Sequence-boundary stills. These are founder-controlled checks, not customer results, speech-sync validation or proof for arbitrary footage. The initial capture-path failure is retained in the example's TESTED.json.
Veyrul's part is finding the saved code/evidence before editing. The agent still uses normal Remotion APIs and runs locally. Public lookup needs no Veyrul account/key; no source footage or private transcript needs uploading. Existing MCP clients can use
https://veyrul.com/mcp/read; askget_contextforword captions retime cuts speed changes,detail="compact",limit=3. The code can also be inspected and used directly without connecting. Remotion's own license terms still apply.If you or your agent are already working on a relevant edit, a public-safe timing map, constraint or failing boundary would be useful. I'll help check whether the saved resource applies and record misses as well as useful results; please keep private transcripts/media out of the thread. For context, #12927 is the separate existing resources-list suggestion, not a request to change Remotion itself.
All reactions