Usage of First Order Motion Model

[matanby]

Hi Aliaksandr,

Thanks for sharing this awesome project :-)
I dug into the code and tried to understand how is the "First Order Motion Model" used when adding the --first_order_motion_model CLI argument.
I saw that when using this argument, an instance of DenseMotionNetwork is created in PartSwapGenerator and this instance is used in the forward pass of this module in order to calculate the dense motion map between the source and the target objects.
My question is, in this case, does this module use segmentation maps (as described in the Motion-Supervied Co-Part Segmentation paper) or discrete keypoints (as described in First Order Motion Model)?
I would guess that keypoints would be used instead of segmentation maps so that we will be able to align the keypoints of the source and target faces, otherwise they wouldn't be aligned and so blending their features would result in an undesired result. Am I correct?

Thanks again!

[AliaksandrSiarohin]

Yes it is correct. Inside the dictionary that contain segmentation there is also affine transformations that used to warp these segmentations. So in case of first_order_motion_mode just this transfrormations is used in dense motion.

[matanby]

Thanks for the answer, makes sense now :-)