Skip to content

v0.2.0 — rotoscoping, a real interface, and on-demand models

Choose a tag to compare

View all tags
@QuantumWars QuantumWars released this 27 Aug 09:24
· 8 commits to main since this release
v0.2.0
8aedc4e

Point at an object in a video. Get back either a matte that tracks it, or the
plate with it removed and the background reconstructed.

Source release. There is no downloadable application yet — see
Not in this release below. Install from source as shown.

What's new since 0.1.0

Rotoscoping mode. The SAM 2 track was already the expensive half of a removal
run, and it is exactly what rotoscoping needs. It is now a first-class output:

Format
ProRes 4444 the footage with the matte as a real alpha channel
ProRes 422 a standalone greyscale matte, white where the object is
PNG sequence per-frame mattes and RGBA cut-outs

Both modes share one cached track. The cache key covers the video, the clicks
and the SAM settings, and ignores every inpaint and encode knob — so pulling a
matte and then removing the same object pays for tracking once, and tuning
--soften or --crf never re-tracks.

Track, then scrub. Propagate the selection across the clip and check it
before rendering. Masks appear frame by frame while the track runs, because
each one is written to the cache the moment it is produced. Finding out a track
drifted should cost thirty seconds, not an hour of inpainting.

A real interface. Rebuilt in React: canvas viewer with left-click include /
right-click exclude, a 4× loupe for edge work, frame scrubbing and playback,
per-mode settings, and a live preview of exactly which files a run will write.

Desktop shell. Electron main process that reserves a port, starts the
backend, health-polls it, and cancels running renders on quit rather than
orphaning a multi-GB torch process.

Models download on demand. Pick SAM 2.1 Tiny / Small / Base+ / Large in the
app and it fetches the checkpoint — resumable, size-verified, written through a
.part file so an interrupted download can never be mistaken for a usable model.

New CLI subcommands: roto (export a matte) and track (warm the cache).

Notable fixes

  • The source frame rate is carried as an exact fraction. Passing the float to
    -r wrote 23.976023976023978 as the timebase, which an NLE conforms — and
    the matte then drifts against the plate it was cut from.
  • ffmpeg and ffprobe resolve through one module with VOR_FFMPEG / VOR_FFPROBE
    overrides, and the compositor no longer assumes libx264 exists.
  • Readiness reports what is actually missing. It used to go green on a
    downloaded checkpoint alone, then fail on the first click because the sam2
    package was never installed.

Install

git clone https://github.com/QuantumWars/video-object-remover.git
cd video-object-remover
pip install -e ".[web]"

# install PyTorch for your platform first (https://pytorch.org), then:
./setup_sam.sh                 # the SAM 2 package
./setup_propainter.sh          # ProPainter + weights

video-object-remover web       # http://127.0.0.1:8765

Model weights are downloaded from the picker inside the app.

Requires Python 3.9+, ffmpeg on PATH, and PyTorch. Apple Silicon (MPS) is the
validated path; CUDA works, CPU is impractically slow.

Not in this release

No .app or .dmg. The desktop shell runs from source (npm --prefix electron start) but expects a Python environment an installer would create, and
that installer does not exist yet. A packaged build needs to bundle its own
Python and ffmpeg and fetch ~2.5 GB of PyTorch on first run; that is the next
release.

It will also be unsigned until a Developer ID Application certificate is in
place, so macOS will require right-click → Open the first time.

Use responsibly

Only remove objects or watermarks from footage you own or are authorised to
modify.

Assets 2
Loading