Skip to content

v4.1.0 - Prop plate tracked and erased per frame across camera pans, with palette exclusion and contact warnings (video-frames --prop-plate)

Choose a tag to compare

@aldegad aldegad released this 07 Nov 00:33
· 11 commits to main since this release

Upgrading from 4.0.0: runs without --prop-plate take the same code path as 4.0.0 (every new argument is None) and give the same keyed frames, frames report, stdout, and stderr byte for byte as 4.0.0, with --decontam palette and --spill auto as well as by default. Runs with --prop-plate: keyed frames and every cut from them change (the plate is erased in each frame where it is found, and the 3.12.0 median rule no longer erases anything), the frames report's prop_plate records per-frame tracking instead of a fixed box, and the edge pass learns the subject palette without the plate. The canvas side (video-canvas --prop-plate) is unchanged. No new dependency, no model call.

  • Fix: video-frames --prop-plate tracks and erases the plate per frame on decoded frames instead of inside one fixed box. In 4.0.0 and since 3.12.0, erasure was confined to the canvas report's prop_plate.box scaled to the frame plus a 3 px rim (erase_box). When camera pan or push-off turned the plate, it left that box after initial frames, leaving the plate in keyed frames and video-loop cuts (strip, GIF, WebP). The 3.12.0 whole-clip channel median rule (within 40) is removed because it assumed a stationary plate. Now, start_box maps the canvas box to frame coordinates rounded outward without a rim. Each frame searches within a window extending the plate's diagonal (on start_box) past the reference box on every side: start_box on frame 1, and the last found box on later frames (retained if not found). Plate pieces are connected components (joined side by side or at corners) within 48 of (255, 0, 255) on every channel (PLATE_TOLERANCE), each holding at least 1% of the frame's plate area (PIECE_SHARE); the plate spans all matching pixels within the pieces' bounding box, finding plates cut in two by the subject whole. Erasure applies within 4 px (RIM, was 3) each way and diagonally on decoded frames: a pixel goes if (a) red and blue lead green by more than 35 and are within 90 of each other (plate, edge, codec-tinted subject edge); (b) it sits on the plate-to-key blend line within 2 px (BLEND_REACH) of the key colour (each channel within 48 of key), Rec. 601 luma within 40 (LUMA_SLACK) of the plate-to-key luma span, and red and blue within 40 (BLEND_BALANCE) plus 0.35 (BLEND_SLOPE) of their mean's distance from green; or (c) it forms an opaque piece wholly within the rim, removing blue-grey codec ringing that colour rules cannot tell from denim. Foreground subjects keep their pixels: skin, cloth, and hair lie off the plate-to-key axis, dark outlines fall below the luma span, white exceeds it, and parts crossing the rim are not wholly inside it. Limits: subject grey touching the plate between plate and key goes with the blend line; subject fragments cut off wholly within the rim go with the plate; and frames where no plate is found (subject covers it or it moved further than its diagonal) keep visible plate pixels. Refused runs keep the plate in their refused folder; video-loop cuts of erased frames carry no plate; video-set does not paint or erase a plate.
  • Fix: the edge pass (--decontam) learns the subject palette without the plate. In 4.0.0, the plate stood as opaque as the subject on the first frame, causing the edge pass (default auto in video-frames) to learn the plate's magenta as a subject colour. With --prop-plate, the first-frame plate region (its pixels and the 4 px rim) is excluded from the 32-colour subject palette sample and from the interior check deciding whether the palette is key-free. Without --prop-plate, the pass learns as before.
  • New: --prop-plate prints warning lines for undetected plates and subject contact. A run prints video-frames: warning: --prop-plate: no plate found in N frame(s) (...) naming frames where no plate was found (nothing erased), and video-frames: warning: --prop-plate: the plate meets the subject in N frame(s) (...) naming frames where opaque pixels remain within the rim ("look at those frames"). Each lists the first six frames and counts the rest.
  • The frames report records prop_plate: canvas_report, start_box, rim (4), frames, frames_changed, erased_px, not_found, touching, and rows. Replaces erase_box with start_box and per-frame entries in rows: frame, found, and where found box (frame px, right and bottom one past), plate_px, tilt (degrees the plate's long axis is turned from upright, clockwise on the frame positive), erased_px, and touch_px.
  • Python callers: sprite_gen.video.prop_plate: new find(files, plate), region(row, size), not_subject(row, size), warnings(done); erase(files, decoded, rows, plate) (was erase(files, plate)); frame_box no longer adds the rim; new constants KEY, BLEND_REACH, PLATE_TOLERANCE, PIECE_SHARE, LUMA_SLACK, BLEND_BALANCE, BLEND_SLOPE, NAMED; RIM is 4; STILL_TOLERANCE removed. sprite_gen.frames.decontam: not_applicable(..., not_subject=None), subject_palette(..., not_subject=None), decontaminate(..., not_subject=None). sprite_gen.frames.extract.remove_chroma_background(..., decontam_not_subject=None); sprite_gen.frames.cutout.cutout(..., decontam_not_subject=None) and extract_route(..., decontam_not_subject=None). None (the default) keeps 4.0.0's behaviour.
  • Docs: video-pipeline.md "Prop plate: --prop-plate" subsection rewritten; chroma-alpha.md decontam section points to it; SKILL.md plate guidance.
  • Tests: tests/video/test_prop_plate.py (19 synthetic tests: fixed plate shifted 1 px in two frames; panned clip sliding 4 px a frame with one frame turned 12 degrees and one frame without the plate; crossing figure in both; clip with a pink patch on the figure). Verifies no plate or magenta pixels remain in keyed frames; each frame row (box within 2 px of drawn plate, plate_px within 5% and tilt within 1 degree in clear frames, not_found, touching); the two warning lines; crossing figure losing at most 2 codec-tinted pixels a frame; byte-identical frames away from the plate compared to runs without --prop-plate (edge pass off); search initialized from the canvas report box (unrelated box finds and erases nothing); direct image check where plate edge and blue-grey blend line go while a denim bar crossing the rim keeps all pixels; early refusal of canvas reports without plates; frame_box arithmetic; palette exclusion ensuring no palette colour within 48 of the plate and preserving pink within 24 (versus plate colour included without --prop-plate); and clean video-loop cuts. Red first on the 4.0.0 engine: 9 of 18 tests failed. The fixed clip kept 376 and 188 plate pixels across the two shifted frames; the panned clip kept the plate in 13 of 15 frames (up to 3572 px, the whole plate); the turned frame kept 667 px (404 magenta). The median rule erased 4, 89, 100, and 104 untinted figure pixels in four frames. The learned palette contained a colour 3 away from the plate. Under the new rules: without rule (c) the blue-grey line beside the bar remained (column 76); with a 3 px rim a line 4 px from the plate remained (column 56); with both, all 19 passed. When the plate region is not excluded from palette learning, the test fails by including a colour within 48 of the plate (nearest was 3 away on its farthest channel); with exclusion, it passes.

Install or update:

pip install --upgrade "sprite-gen @ git+https://github.com/aldegad/sprite-gen.git@v4.1.0"

The wheel, the source archive and SHA256SUMS are attached. Full notes in CHANGELOG.md.