Filmmaking from images, not from prompts

[ReneBoroughs]

Filmmaking from images, not from prompts

A film is basically a flipbook: a sequence of still images creating movement.

From a filmmaker’s point of view, a useful workflow would be:

Image 1 → selected movement → Image 2

In the ComfyUI workflows I tested, I could provide a starting image, but not a second image defining where the shot should end. The model therefore had to invent the continuation, often changing faces, hands, objects or camera position.

In CineAI-lab we have been developing a DIRECTOR approach inspired by the old Director/Lingo logic: the filmmaker selects and names elements of the image, defines which areas may move, which must remain locked, and assigns temporal behaviours to those selections.

For the example above, I may want only:

• one hand to move,
• the father to point,
• facial expressions to change slightly,

while the background, framing, other characters and camera remain unchanged.

The task would then be:

Create the missing frames between Image 1 and Image 2, animating only the selected elements.

This could also reduce generation time. Regenerating an entire frame when only a small region needs to change is costly and makes iteration very slow.

Useful controls would include:

• Start image
• End image
• Named selectable regions
• Motion / locked regions
• Temporal behaviours
• Camera lock
• Character and object preservation
• Duration / FPS
• Optional intermediate keyframes

The filmmaker defines the images and the movement. The AI creates the transition.

https://cineai-lab.org/