Does rest shape have to fit any specific frame in video ?

[vhvkhoa]

Hello, thank you for your great work.

I am wondering if the rest shape has to fit a pose in a specific frame of the video, e.g., first frame or last frame ?
If so, can I know which part in the source code does that ?
If not, how can you control the rest shape to be in rest pose like human or cat standing straight like in the visualization ?

[gengshan-y]

Hi, we didn't enforce the rest shape to be consistent with a specific video frame (that is actually a good idea to get good skinning weights, but we wanted to reduce manual inputs in the project).

The rest shape in the visuals are automatically figured out by the optimization. We also found it to be data-dependent -- if there are more standing poses in the video, the rest shape tends to be standing.

[vhvkhoa]

Thank you very much for your clarification. But before I close this issue, I'd like to have an additional question.

If the rest shape is not required to fit poses of any frame, how do you constrain it to be at the same scale with them without being distorted in any axes, or rotate, distorted in an arbitrary angle ? Because by learnt forward warping, the pose at a certain frame can be recovered.

[gengshan-y]

I'm not sure if I understand the question. Do you mean, why the deformed shape and the rest shape happens to converge to the same scale?

This is because the se3 bone transformations are initialized as very small values, approximating a single rigid body. During optimization to fit the video observations, the bone transformations grow but doesn't suddenly become arbitrarily large that would cause distortion / global scale change.