Skip to content

H3 Editor 0.8.0

Choose a tag to compare

@igorls igorls released this 15 Nov 05:02
· 52 commits to main since this release

H3 Editor 0.8.0 is about what happens where two clips meet. Joins hold a take's own latents and cut where the two files match, bridges land on the next clip, and the cuts report now listens to a cut and watches how things move across it.

H3 Editor 0.8.0

Highlights

  • Joins hold a take's own latents, from any cut, with its sound. A chain of extensions no longer gets darker and softer with every join.
  • Joins cut where the two clips match, 9 frames inside the frames both files show.
  • Finals join finals the same way, in the refine pass too.
  • A bridge lands on the next clip. It used to cut to it.
  • The cuts report listens and watches. Every cut gets a sound score, the camera's move across it, and what stops or starts moving in the frame. The editor lists what it finds.
  • The cast catalog. Save a cast member once and bring them into any project.
  • A project list you can see, with a frame of each project.

Joins

A clip made with Extend forward, Extend back or Bridge to next holds frames of the clip it joins. Until now those frames were read back from the source's video file and encoded again, except at a take's two ends. Each trip through the video decoder and encoder cost a little: over a chain of extensions the held frames came out 1.2 to 2.7 percent darker per join.

A take now saves all of its latents beside its video, and a linked clip holds them as they are.

What is held When it is the take's own latents Otherwise
Picture The cut is on a multiple of 17 frames, the grid of H3's picture latent. A whole clip always is. Read from the video, as before
Sound The cut is also on a multiple of 3 frames, as the end of every extension is. Encoded from the file, as before
A final's refine pass The source is a final too, refined at the same size. Read from the video, as before

Extending from a trimmed clip moves the cut onto the grid by 16 frames at most (two thirds of a second), and the clips after it close up. A whole clip is already on the grid. Imported footage keeps its cut, and Merge with next still holds its frames from the video.

The file beside a take is about 1.6 MB for a 5 second draft and about 22 MB for a 5 second final.

Where the cut sits

Both files show the held frames, and the timeline has to cut from one to the other somewhere. It used to cut at their edge, which is the worst place: a video decoder draws the last frames of a file with nothing after them and the first with nothing before them, and the held sound is softened next to the new material. New extensions, bridges and merges cut 9 frames inside the held frames. The source stops 9 frames earlier and the new clip plays those 9 itself, so the scene is as long as before.

Measured At the edge 9 frames inside
Picture at the end of a file, against the same frames in the other file 1.5 dB under the rest on the last frame, 5 dB at worst Level with the rest
Picture at the start of a file 6 to 9 dB under on the first frame Level with the rest
Step in the sound at the cut, against the steps around it 7 to 22 times Under 3 times

Joins made before this release keep their cut. Render a linked clip again after trimming its source and it is joined at the new cut.

Bridges

A bridge used to hold the next clip's first 39 frames at its end, as it holds the first clip's last 39 at its start. H3 treats a run that long from another shot as another shot: in 8 of 9 test pairs it followed the prompt and then cut to the next clip a few frames early. Changing the prompt did not help.

Bridge to next now holds a short run of the next clip, 22 frames with their sound, and the timeline cuts 9 frames inside it. That is enough to say where things are and how they move, and too little for H3 to take for another shot.

At the cut into the next clip One pinned frame (tried first) 22 frames held (now)
Picture, of 100 71 to 97 97 to 99
Sound, of 100 0 to 90, with drops of up to 36 dB 78 to 97
What moves in the frame Stopped at the cut in one pair Carried across every time

Two test pairs, three seeds each, drafts.

Bridge to next Bridge into its movement
Holds of the next clip 22 frames 39 frames, as before
Use it for A change of framing or of place A next clip that starts in mid-movement
Default length 6.4 seconds 4.25 seconds

What to expect:

  • A bridge needs a way from one picture to the other. It works best when the two shots share a place, or when the move passes through something that hides the change: into the dark of a cup, through a window. Two shots with nothing in common still cut.
  • What happens on the way depends on the seed. The same bridge may push in, dissolve or cut. Look at the take, and render it again when it does not move as you asked.
  • A move needs room. At 4.25 seconds a push-in from a wide shot to a close-up never came out as one shot in testing, so a new bridge is 6.4 seconds unless you set its seconds.
  • The held sound can step a little at the cut. It is encoded from the next clip's file, because 22 frames is not a whole number of sound steps.

Bridges made before this release keep their way of arriving.

The cuts report

The report already scored the picture at every cut. It now scores four more things, and the editor lists what they find under Checks in the top bar.

Score What it catches How
Sound at a cut The level jumping, a swell chopped off, sound into silence, a click Loudness on each side after ITU-R BS.1770, against how much the sound moves by itself around the cut; the change in the spectrum's shape; the step in the wave
Camera move at a cut A push-in or pan that stops dead, starts or reverses at the cut The optical flow of the whole frame (Lucas and Kanade's method, one pan and one zoom per frame)
Movement in the frame Something that moves before the cut and is still after it: rain, a wave, a beam of light What is left in each of 72 parts of the frame once the camera's move is taken out
Sound inside a take A sound that is cut off, a hole in the sound, a take that ends 20 dB louder or quieter than it began The level along the take. A sound that starts at once is left alone: words and waves do that

Each score is judged against what the sound or the picture does by itself around the cut, so a handheld shot or a swelling sea is not flagged for being itself. A cut to a new shot has its camera move and movement described, not scored.

python joins.py PROJECT_FOLDER is a separate measure for joins: it compares the two copies of the frames every join holds, latents, picture and sound.

The cast catalog

Action Where
Keep a cast member for other projects Save beside Catalog on the cast member
Bring one into a project Add, From the catalog. The project gets its own copy of their pictures and voice.
Pick up a change After a change is saved to the catalog, other projects show Newer in the catalog and an Update button.
Update every project at once From the catalog. Copies changed in their own project are left alone.

The project list

The project menu in the top bar is now a list of the editor's own: each project with a frame of its first rendered clip, its shape, how many clips are on its timeline, how long it runs and when it last changed. Type to narrow it by name, move with the arrow keys and open with Enter.

LoRA starts

H3 samples with a shift that keeps most of a render near the start of the scale, so a start of 0.75 was already four fifths of the way in. The presets moved to match.

Preset Was Now
After framing 0.75 0.93, about halfway
Details only 0.4 0.7
The Realism Slider on finals 0.75 0.93: the last four of HyperFlow's six steps, where it used to work on the last three

A start you set yourself stays as it is.

Fixed

  • The side panel's tabs no longer make the panel scroll sideways at its usual width.
  • Space plays again after you pick from a menu and then scrub the timeline.

For agents

What Where
sound, movement and parts on each cut GET /project/{name}/cuts: what the cut sounds like, how the camera's move carries across it, and what stops or starts moving in the frame. Each has kind, score and notes.
sound_inside and arrivals The same report: sound faults inside a take, and whether a bridge pinned to a frame reached it. worst counts all of these.
"arrive" on bridge "hold" holds 39 frames of the next clip, "pin" pins its first frame alone. Left out, the bridge holds 22 frames, and its link says "arrive": "short".
handover on a link How many frames inside the held frames the cut sits: 9 on new links.
save_cast_to_catalog, add_cast_from_catalog, update_cast_from_catalog Operations for the cast catalog. GET /projects lists the entries under catalog, and each project's facts under cards.
end_hold_frames on H3 Take, final on H3 Save Hold For your own workflows: a bridge that holds a short run of the second clip, and a refine pass saved beside the first.
python listen.py a.mp4 b.mp4, python joins.py PROJECT_FOLDER --json The sound score and the join measurements from the command line.

Updating

Update the pack and restart ComfyUI. Projects made with 0.7 need no change.

Takes rendered before this release keep their old file and are held from their two ends, and finals from their video in the refine pass. To get exact joins on an existing cut, render the source clip again, then the clip that joins it.