Skip to content

Releases: stoatworks-labs/residual

Release list

v0.1.0

Choose a tag to compare

@github-actions github-actions released this 23 Sep 21:34

Datamosh, built as a real codec, as an FFGL effect for Resolume Arena and Avenue, shown as SW Residual. Datamosh is not a glitch: it is a video decoder doing exactly its job with the wrong information. So this is the codec — a hierarchical block-matching motion estimator, closed-loop prediction from the decoder's own reconstructed frame, a quantised 8×8 DCT residual in reversible YCoCg-R, and a GOP with scene-cut detection — and then controls that break exactly one stage at a time.

Drop I takes away the I-frame at a cut, so the decoder keeps painting the old picture with the new scene's motion: the classic mosh. Residual Gain at zero is a decoder that never corrects. Vector Hold and Vector Scale reuse and stretch one frame's vectors: the bloom. Q quantises the correction into blocks and drift. Refresh is one I-frame, now. Drop I can also be armed by an audio onset, and Show Vectors draws the field the search found.

What is verified

By tools/verify.sh on an M4 Max, macOS 26.4, on 2026-09-23 — 296 assertions across nine suites, most of them bitwise, at two rasters:

  • With the I-frame dropped and the residual at zero, the decoded frame after a hard cut is the previous decoded frame block-copied by the estimator's own vectors: 0 of 2,592,000 samples differ across six cases (--mosh).
  • At Q 0 the codec is lossless: 0 of 49,766,400 samples differ (--lossless). YCoCg-R round-trips all 16,777,216 RGB triples exactly.
  • A texture translated by (dx, dy) returns exactly (dx, dy) from all 29,961 interior blocks over 63 cases (--vectors).
  • An I-frame is bitwise a function of its source alone (--gop); closed-loop drift grows by exactly one code a frame for 31 frames under the quantiser's dead zone (--drift).
  • Seven deliberate perturbations of the model make those checks FAIL, and one character changed in the shipped predict shader fails --mosh in all six cases.
  • All 14 sweepable controls change the picture; oxbow loads it as SW Residual / RS01 / effect and renders 120 frames.
  • The harness's --pipe mode (the fleet's filming format) round-trips at Q 0 bit for bit, and takes cues for Refresh and for an audio onset.
  • Render cost at the defaults 5.1 ms/frame at 720p, 6.1 at 1080p and 14.0 at 4K, about 1 ms of it the one-integer scene-cut readback. The motion search is the cost: 8-pixel blocks at ±32 run at about 50 ms a frame, which is not a setting for a show. macOS figures only.
  • Windows, in Resolume Arena 7.27.1 (win-lab, Mesa llvmpipe, no GPU, 2026-09-23): a CI build of this source loads from Extra Effects, registers as SW Residual / RS01 / effect, all 22 host controls match the declaration, it renders and Arena's log stays clean: 8 of the fleet gate's 9 checks. The ninth is controls: Search Range, Half Pel, Scene Cut, Scene Threshold and Drop I read dead, because the gate holds a still picture and a codec's motion search and scene-cut detector have nothing to find in one — no motion, no cut. The harness sweep, which feeds moving pictures and cuts, proves all of them live. Software rendering says nothing about a GPU or about speed.

What is not

  • Never loaded into Resolume on macOS.
  • Chroma is coded at full resolution, not 4:2:0, so the chroma-block look of a real mosh is not here yet.
  • The scene-cut detector only judges frames whose vectors are fresh; under Vector Hold a cut can pass unnoticed.
  • The GPU memory figures in the guide are worked out from the buffers the plugin allocates, not measured.
  • The audio path has only seen synthetic spectra; nobody has measured what Resolume's 64 FFT bins are.
  • No OpenFX port, no browser demo, no presets. There is a user guide at https://stoatworks-labs.com/software/residual/guide/.

The macOS build is now Developer ID-signed and notarised (signed on 2026-09-24, after publication); a copy downloaded before then is unsigned, so if macOS refuses it, download it again. The Windows build is unsigned, so SmartScreen warns once.