Seamless camera loop

[devwearsprada]

Hi everyone,

Thanks for the great work on this repo, I love playing around with NeRFs 👍🏼

I've been trying to render a seamless loop around an object. My idea was to add the same object from the start in the path array (within base_cam.json) to the end of the array. This seem to somewhat make a circle around the object, but it is not seamless (not the same start position and end position). It seems that the camera moves around a spline based on the paths given as an input. Does anyone have an idea on how to control these splines in order to make a seamless circle around an given object? Thanks in advance!

[Tom94]

Hi there, the spline that is used is a cubic b-spline, so if you'd like the camera to perfectly match the pose from a given keyframe, that keyframe needs to be duplicated (so it occurs twice in a row). If you do this for both the start and end frame, you should get a perfect circle. Cheers!

[erich666]

I believe you'd actually need to duplicate the first three keyframes and put them at the end, or vice versa, not just two. This is needed because a cubic spline interpolates among four keyframes. You'll also need to trim off the repeated parts of the generated video manually. No fun, but possible.

I'm working on adding a simple "make this loop" video option for run.py, I hope to get its approval soon.

[devwearsprada]

Thank you both for your suggestions, I'll try them out soon and report back with my findings.

@erich666, the "make this loop" option sounds great, hope you get the approval!

[erich666]

It's in! With help from @Tom94 to get it just right, you can now easily make a smooth, repeating loop. When using run.py to make a video, just add:

--video_loop_animation

to the command line. What it does is acts as if your keyframe list is now continuous. It internally repeats the first keyframe to also act like the last one (in other words, you don't copy your first keyframe to be last), and properly interpolates using the neighboring keyframes along the way. Does that make sense?

The commit: 25dec33

All this said, there appears to be a little bug in instant-ngp, where the very first frame (of any animation) is weirdly blurred. You can see this here: https://youtu.be/uW92VodbpMQ and here https://www.youtube.com/watch?v=NO5Vw3dLzrE (so, it's not just me). Scrub to the first frame and it's blurred. This doesn't matter much if you're not looping a video (the first frame whips by and never repeats), but becomes obvious if you are, e.g., https://youtu.be/uW92VodbpMQ - we're working on it (well, I tried, but didn't figure it out; on to the experts).

[erich666]

The little bug I mentioned, where the first frame of an animation is blurred, has also now been fixed. So, this feature request is done! Fixed loop example here: https://looptube.io/?videoId=x1bJ5TF7B6Y