Repository navigation
Automation
Use the running desktop engine's loopback URL, or the development engine at http://127.0.0.1:8040. The desktop port is assigned by the OS and appears in the engine startup log. POST JSON-RPC 2.0 to /api/rpc with Content-Type: application/json and x-formcut-local: 1.
{"jsonrpc":"2.0","id":1,"method":"capabilities"}capabilities lists current methods, editable properties and command names. catalogue lists effect IDs. project.list, project.create, project.get, project.edit, project.undo, project.redo and project.export operate on persisted documents. job.status and job.cancel operate on jobs in the same engine process. captions.export writes a project track's captions as SRT or WebVTT text; captions.import turns SRT or WebVTT text into cues (start, end, text) for clip.add.
{"jsonrpc":"2.0","id":2,"method":"project.create","params":{"title":"Example"}}Use the returned project id and revision for subsequent changes:
{"jsonrpc":"2.0","id":3,"method":"project.edit","params":{"id":"PROJECT_ID","revision":1,"commands":[{"op":"track.add","name":"Titles"}]}}A command can add/update/remove/move/split clips, add/remove tracks, add timelines, update project settings, or add, update and remove markers on the active timeline (marker.add with t, colour, note; marker.update; marker.remove). clip.update accepts ordinary document properties. Multi-clip commands act on a set in one call with the interface's semantics: clips.move (clipIds, deltaStart, optional deltaTrack; refused whole when any member would land on a locked track or before zero), clips.remove, clips.duplicate (copies land after their sources), clips.add (entries of track and clip) and clips.mute; capabilities lists them. A batch of commands is committed as one undo step. Supply the current revision to detect conflicts. REST PUT /api/projects/{id} also checks revision. Undo history is retained for the engine session only.
An external system can identify itself with session.hello and pass the
returned session id as session in later calls; every call is recorded against
that session and shown in the app under Settings, Connections, so the person in
front of the editor can see who is driving. media.list and media.import
move assets in (import copies the file rather than consuming it), and
job.result returns the finished render's path, name, media type and a URL to
fetch it.
GET /api/orchestration/events is a server-sent event stream carrying session
opens and closes, each orchestrated call, and job state with progress every five
percent through to done, cancelled or failed. It keeps a short backlog, so a
subscriber that connects mid-render still sees the job it cares about, and sends
a keepalive comment every fifteen seconds. Jobs can be cancelled from either
side: job.cancel over RPC, or the cancel control in the Connections tab.
Rendering: POST /api/render/export with { "project": DOCUMENT, "format": "mp4", "quality": "high" }. The response contains job_id, not id. Over JSON-RPC, project.export returns the same job_id, and job.status and job.cancel accept it as either id or job_id. Poll GET /api/jobs/{job_id}, cancel with DELETE, and download GET /api/jobs/{job_id}/file after status done. WebM is also supported; qualities are high, medium and low. The old /api/export route remains a compatibility renderer; use /api/render/export for current features.
The backend CLI accepts one JSON-RPC request per line on stdin and emits responses on stdout. From backend/:
python -m app.cli
python -m app.cli --tcp 18051
python -m app.cli --unix /tmp/formcut.sock
python -m app.cli --grpc 18052TCP binds to 127.0.0.1 and uses newline-delimited JSON. Unix sockets require OS support; remove a stale socket path after an interrupted server. The generic gRPC method is /formcut.Automation/Call, unary request/response with UTF-8 JSON bytes. It is not a generated protobuf service, and no compatibility with any other product's protocol is claimed. The frozen entry point accepts --headless before these CLI options.
WebSocket /api/rpc/ws sends the same JSON objects over a loopback connection. HTTP, stdin, TCP and gRPC have been exercised, most recently a full HTTP sweep that created a project, applied a four-command edit, rejected a stale revision, ran undo and redo, answered a batch, and drove an export to a finished file. WebSocket and Unix socket are implemented but still unverified. Do not use the GUI and a second standalone CLI process as concurrent writers to one data directory. Use HTTP RPC on the GUI's engine instead.
project.edit adds clips.split, clip.rejoin, clip.freeze-insert, clip.reattach; magnetic tracks settle after each batch; clip_properties includes reversed and audioStream (which of the media's audio streams a clip plays); per-timeline formats apply to project.export.
Title styles ride clip.text patches like any other style field, so the full property set is scriptable; /api/presets/titles lists the built-ins.
project.export and POST /api/render/export accept "gif" beside mp4 and webm, rendered through a two-pass palette. POST /api/media/capture remuxes a browser screen or camera recording into the bin. A clip's cutout.model accepts "auto" (person when the person model finds someone, else object; needs both models); a cutout stroke may carry region (a normalized outline the matte shaper fills instead of the painted disc) and t (the source time it was resolved on), which is what POST /api/render/smart-brush returns for a stroke's points: {project, clipId, time, points} in, {region, t} out, 409 until click-select is downloaded. clip.text on a caption clip refits its words in place. GET /api/models/runtime names the active ONNX Runtime execution provider and the available list.
POST /api/transcribe runs Whisper in the engine as a job (mediaId, sourceIn, duration, language, size from tiny to large, wordsPerCaption, optional audioStream); the job's result carries chunks with word times, byWord, model and provider, the same shape the Captions panel consumes. GET /api/transcribe/status lists providers, sizes and which models are cached. project.edit adds timeline.cut-range (start, end): the span leaves every track of the timeline, clips inside it are removed, clips straddling it are cut into a head and a tail, and everything after moves back by the length; it is the command behind the timeline's Extract. POST /api/speech/range (mediaId, start, end in source seconds) answers the span someone speaks in near those times, found in the audio.
The app carries an MCP server (formcut-engine --mcp, module backend/app/agent_mcp.py) with 34 tools over stdio,
each a thin, documented wrapper over the RPC above; the full listing is generated into
.claude/skills/formcut-mcp/sub-skills/mcp-workflows/references/tools.md. By purpose:
- Engine and projects:
formcut_status,formcut_capabilities,formcut_catalogue,formcut_list_projects,formcut_create_project,formcut_get_project,formcut_edit,formcut_add_clip,formcut_set_frame. - Media:
formcut_import_media,formcut_list_media,formcut_media_proxy,formcut_find_scenes,formcut_transcribe(sentences with their times; the small model by default),formcut_speech_range(RPCspeech.range:mediaId,start,endin source seconds; answersstart,endandsource, the span someone speaks found in the audio, refused when there is none). - Rendering:
formcut_render(format, quality, rate, size, codec, ten bit, exact, audio bitrate, a loudness target with a true-peak ceiling),formcut_render_audio,formcut_render_status,formcut_cancel_render. - Delivery checks:
formcut_qc,formcut_frame_check,formcut_speech_check. - Music and cutdowns:
formcut_music,formcut_plan_cutdown. - Device library:
formcut_devices(RPCdevices.list, optionalid;GET /api/devicesandGET /api/devices/{id}/model): the ten device models with their screens (inner and cover, size, aspect, suggested pixels, cut-outs), the parts that pose (lid, fold, stand, ranges in degrees), backlight and finishes. - Device clips:
formcut_deviceadds one on a new track (a clip of kinddevice, throughclip.add) or changes one (theclip.deviceedit: a patch merged into the clip'sdeviceblock,camera,poseandscreenskey by key, null removing a key, the result checked against the library). The block:id,finish,screens.innerandscreens.cover(eachmediaIdortimelineId,offsetin seconds,fitcover or contain),pose(lid, fold, stand in degrees, within the device's ranges),camera(yaw, pitch, distance with 1 framing the device, fov, roll, panX, panY),backlight,exposure,light,shadow,screenBrightness. Each number is keyframable asdevice.<name>(device.yaw, device.fold, device.backlight); otherdevice.names are refused. - Camera moves:
formcut_focus_motion(theclip.focus-motionedit:areasas[x0, y0, x1, y1]on the clip's picture in visiting order,speedfast, medium or slow,loop, optionalseedandmargin; the clip's scale, posX and posY keyframes replaced in one edit;POST /api/focus-motionreturns the keyframes without applying them). On a device clip the areas are in stage fractions and the move drives the camera: device.yaw, pitch, distance (down to 0.1), panX and panY keyframes, each stop centred on the front-most device point under its area through the renderer's own camera, with an alternating turn; the route takesmodecamera,camera,deviceandpose. - Effect packages:
formcut_package_guide,formcut_add_package,formcut_preview_package,formcut_list_packages,formcut_remove_package.
formcut_render_status opens every reply with state: and a numeric progress: (then file:, or reason: when a job
failed or was interrupted). Tools that need a measurement of a source first (the frame, speech and cutdown checks)
start a job on the first call and say so; poll formcut_render_status with its id, then call again. The cutdown
sub-skill in .claude/skills/formcut-mcp walks a whole cutdown through these tools, and the remove-speech sub-skill
removes the footage of sentences a person names (transcribe, formcut_speech_range, then timeline.cut-range).
project.export passes every export option: codec, tenBit, exact, size, rate (23.976 to 60), audioBitrate
(kbps), loudness (integrated LUFS) and truePeak (dBTP, default -1): the mix gets one measured gain that reaches the
target without passing the ceiling. A rate more than 0.2% from the project's is sampled from the timeline at that rate.
project.export_audio renders the mix alone. Jobs are kept in jobs.json in the data folder at each status change;
after a restart finished jobs are found again and an export caught mid-render is queued again under its own id
(interrupted, with the reason, when it cannot be). job.result carries a failed job's error and returns a finished
job's result whole when it is not a file.
-
media.qc(joborid, optionaledits,loudness,truePeak): duration, frames, rate, codecs; exact and perceptual (mpdecimate) repeats; freezes; black and flash frames; the first and last frame against the sharpest of their own shot's first and last second; loudness, range and true peak; clicks at the given edits;passedandreasons. -
timeline.frames(id,rate,fix): each video clip's predicted repeated frames from its source's measured cadence; withfix, the smallest change that removes them (an in-point shift up to 3 source frames and a speed-up up to 2%, else a push of 0.10% scale an output frame) as one edit. -
timeline.cuts(id): every sounding clip's in and out point checked against its source's voice map (Silero VAD, thevoice-activitymodel): unsafe with speech on both sides within 100 ms, the nearest pause given. -
media.transcribe(id,size,language): a whole library item transcribed as a job. -
media.music(id, optionaloutFrom,outTo): tempo, beat, bar, first beat, final hit; with an out-point range, joins a whole number of bars apart whose ends are outside speech.media.beatstakes its tempo from the same grid. -
media.cutdowns(id,length,top): ranked scripts of the length with their exact source segments, bars and pauses (whole sentences, an opener that can stand first, bar-aligned joins with natural pauses, music bridges, the call to action last on the button); one job first measures what is missing (sentences, voice map, music map).
form-functions.json at the repository root declares formcut's Form functions for NodeMode's hub (format
form-functions, version 1): typed files in, typed files out, no window and no engine server. Each entry is a
callable in backend/app/functions/, fn(inputs, params, out_dir) -> dict: inputs maps each input name to a local
file, the result maps each output name to a file written inside out_dir, and a refusal or failure raises
FunctionError with a short message. Params are checked against the manifest's schema first, defaults filled in.
-
transcribe(media, a video or audio file) writestranscript.json(model, size, language, provider, duration, and segments of start, end, text and words, each word with start and end) andsubtitles.srt(the caption chunks the Captions panel lays out). Params:modeltiny, base, small, medium or large (default tiny),languageen, auto or a language code (default en; anything but en runs the multilingual model). It is the engine's own transcriber (transcribe.py), with the model fetched once by the pinned hub fetcher into the models folder. -
render_project(project, a.fcpackage or a.fmcfile) writesvideo.<format>. Params:formatmp4, webm or gif (default mp4),qualityhigh, medium or low (default high). It is the export behindPOST /api/render/export(render_api.export_media) with no engine server. A.fcbrings its media, LUTs, shaders and fonts; a.fmcbrings only its clips' media, each read at its path relative to the.fmc, which must stay inside the.fmc's folder (Collect Files makes such a folder). Any media, background image, LUT, shader or font the project names that did not come with it is refused, and an mp4 is refused rather than fetching OpenH264 when the FFmpeg has no H.264 encoder.
From a shell, inside backend/:
python -m app.functions run transcribe --inputs '{"media": "talk.mp4"}' --params '{"model": "base"}' --out out
python -m app.functions run render_project --inputs '{"project": "cut.fc"}' --params '{"format": "webm"}' --out out
python -m app.functions manifest --writerun prints the result as one JSON line, or the error on stderr with exit status 1. Every call, from the shell or
through an entry, checks its params and inputs and then runs in a child process whose FORMCUT_DATA_DIR is a fresh,
empty temporary folder, removed afterwards: a function can neither read from nor add to a person's library, and the
caller's environment is left as it was. The models folder is shared, so a Whisper model is fetched once. After changing a function's declaration in
backend/app/functions/contract.py, run manifest --write; tests/test_form_functions.py fails while the file
drifts.
Pages
- Home
- Architecture
- Creator workflow
- Automation
- Extensions
- Packaging
- Translation
- Mobile notes
- Models
- Licences
In the repository
These pages are published from .docs/wiki/ in the repository. Edit them there;
a push to main updates this wiki. Editing a page here is overwritten by the
next publish.