Need help with a very specific task V2V hip hop music video

[gizy843]

Hey guys, looking for some advice from experienced VFX / AI people on a music video transition setup (Russian hip-hop) where I need to take a static shot and replace a car in the background with a different one, then swap it for a pack of Dalmatians, clear the scene entirely, and have an elephant walk into frame, all while keeping my original body movements and lip sync completely intact without turning my face into mush. I've tried Minimax h3 (total mess), LTX 2.5 ripple (okay from afar, no detail up close), and Wan 2.2 Animate (better motion, but weak lip sync and prompt adherence), and recently saw someone rotoscope their face to protect it in Wan 2.2, so I'm wondering what the absolute best workflow or ComfyUI setup is for this right now since I'm down to install any model that gets clean, high-quality results.

[Aaron3312]

Since the camera is locked off and your performance must stay untouched, I'd stop asking a video model to regenerate you at all. Anything that re-renders the full frame (Wan 2.2 Animate, LTX…) will touch your face and lips. Treat it as a compositing job instead: the AI makes only the background, and your original pixels go on top.

1. Matte yourself. Use SAM 2 in ComfyUI (kijai/ComfyUI-segment-anything-2): click points on frame 1 and it propagates through the clip. Refine the edges with BiRefNet/RMBG or a matting model for hair, and export the mask sequence. It's the "rotoscope the face" trick, but for the whole body.
2. Make a clean plate. Take one frame and remove yourself and the original car with an edit or inpaint model (Flux Fill, Qwen-Image-Edit, or LaMa for a quick removal). With a static camera, that one empty plate works for the whole shot.
3. Make the key states as stills. From the plate, make edited stills: new car, Dalmatians, empty, elephant entering. Edit models keep the rest of the plate stable.
4. Animate between states, with nobody in frame.
   • Use Wan 2.2 14B I2V for "an elephant walks into frame from the left, static camera".
   • Use first–last frame (WanFirstLastFrameToVideo, or the FLF2V template) for car → Dalmatians → empty.
   • Clips are about 5 s (81 frames at 16 fps), so plan the beats around that, then interpolate to your frame rate with RIFE or FILM.
5. Composite. Put the original footage over the generated background through your mask, in Resolve/AE/Nuke or with ImageCompositeMasked in ComfyUI. Match grain and color, and keep or fake a contact shadow so you don't float. If something has to pass in front of you, matte that clip too and layer it above.

Your face and lips are never generated, so the lip sync stays exact, and you can redo any background beat without re-rendering yourself. The weak spots are hair edges and lighting: if the new background is much brighter or darker, you'll need a color pass or a light wrap.

[gizy843]

Hello, thank you so much for your advice bro 😁 It really works well. But now I had another problem. Wan generates only 5 seconds video, and I create music snippets 15 secs and longer. When I'm finishing first 5 sec generation, I take the last frame and put it as a new reference image. And Wan does his work well, but when I put all three 5 sec videos to davinci resolve, there were still visible transitions between the videos. You could tell that the frame had changed, though I couldn't pinpoint exactly what was different just some sort of micro-adjustments. Regardless, the seam between one video and the next is still noticeable. Do you happen to know how this issue could be resolved? 🧐 пн, 2 нояб. 2026 г., 08:56 Aaron Hernández Jiménez ***@***.***

…

: Since the camera is locked off and your performance must stay untouched, I'd stop asking a video model to regenerate you at all. Anything that re-renders the full frame (Wan 2.2 Animate, LTX…) will touch your face and lips. Treat it as a compositing job instead: the AI makes only the background, and your original pixels go on top. 1. *Matte yourself.* Use SAM 2 in ComfyUI ( kijai/ComfyUI-segment-anything-2 <https://github.com/kijai/ComfyUI-segment-anything-2>): click points on frame 1 and it propagates through the clip. Refine the edges with BiRefNet/RMBG or a matting model for hair, and export the mask sequence. It's the "rotoscope the face" trick, but for the whole body. 2. *Make a clean plate.* Take one frame and remove yourself and the original car with an edit or inpaint model (Flux Fill, Qwen-Image-Edit, or LaMa for a quick removal). With a static camera, that one empty plate works for the whole shot. 3. *Make the key states as stills.* From the plate, make edited stills: new car, Dalmatians, empty, elephant entering. Edit models keep the rest of the plate stable. 4. *Animate between states, with nobody in frame.* - Use Wan 2.2 14B I2V for "an elephant walks into frame from the left, static camera". - Use first–last frame (WanFirstLastFrameToVideo, or the FLF2V template) for car → Dalmatians → empty. - Clips are about 5 s (81 frames at 16 fps), so plan the beats around that, then interpolate to your frame rate with RIFE or FILM. 5. *Composite.* Put the original footage over the generated background through your mask, in Resolve/AE/Nuke or with ImageCompositeMasked in ComfyUI. Match grain and color, and keep or fake a contact shadow so you don't float. If something has to pass in front of you, matte that clip too and layer it above. Your face and lips are never generated, so the lip sync stays exact, and you can redo any background beat without re-rendering yourself. The weak spots are hair edges and lighting: if the new background is much brighter or darker, you'll need a color pass or a light wrap. — Reply to this email directly, view it on GitHub <#17472?email_source=notifications&email_token=CTVWIZHEKE4YYZXK26OR75T5XAV3PA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCOJTGQ2DOMBYUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVRTG633UMVZF6Y3MNFRWW#discussioncomment-19344708>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/CTVWIZGVHE4HXE757ALZQGL5XAV3PAVCNFSNUABIKJSXA33TNF2G64TZHM2TQOJYGMYTOMJYHNCGS43DOVZXG2LPNY5TCMJRHE4DKNZYUF3AE> . Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS <https://github.com/notifications/mobile/ios/CTVWIZFSV3M4G2KO2S3OL5D5XAV3PA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCOJTGQ2DOMBYUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVJTG633UMVZF62LPOM> and Android <https://github.com/notifications/mobile/android/CTVWIZDYKJT3AZKUJLHXYW35XAV3PA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCOJTGQ2DOMBYUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVZTG633UMVZF6YLOMRZG62LE>. Download it today! You are receiving this because you authored the thread.Message ID: ***@***.***>

[mrhy7176]

I work on CloudGPU. For the remaining seam, I'd keep your original performance as one continuous foreground layer and overlap/crossfade only the generated background clips. A reproducible option is FFmpeg's xfade: first make both inputs constant-frame-rate with matching resolution, pixel format, frame rate and timebase, then set offset to the overlap start and duration to its length. That treats the join as an edit rather than asking the next generation to preserve the entire motion state from a single frame. I haven't tested your footage; a dissolve may still show ghosting or discontinuous motion.