how to apply pytorch3d to face 3d reconstruction?

[xingmimfl]

In this situation, i have got vertex/angles/scale, how can i project this vertex onto 2d images? like this

[nikhilaravi]

@xingmimfl it sounds like you just need to transform the vertices and then project them onto a 2D image. For this you probably don't need a renderer. #125 is a related issue.

You can initialize a Rotation/Translation matrix based on the angles (do you mean euler angles?). Have a look at the RotateAxisAngle class for how to do this. We also have a number of helper functions for initializing rotation matrices in other ways - see the rotation helper functions and the look at transform. The Transforms3D class also allows you to easily compose transforms.

You can then create a Cameras class which takes the R/T matrices as input - you can chose from several different camera types and the type of projection you want to apply (orthogonal/perspective).

You can then call the transform_points method on a camera which will apply the transformation matrices and project the points onto 2D.

e.g. pseudo code would look like this:

from pytorch3d.transforms import RotateAxisAngle, Scale
from pytorch3d.renderer import OpenGLPerspectiveCameras

# vertices (N, V, 3) tensor
# angles  (N, )  e.g. one angle per batch element 
# scales (N, 3) xyz scaling factor per batch element
# given H, W, of images

aspect_ratio = H/W
scale = Scale(scales)
scaled_points = scale.transform_points(vertices)
R = RotateAxisAngle(angle=angles)
camera = OpenGLPerspectiveCameras(R=R, aspect_ratio=aspect_ratio)
projected_points = camera.transform_points(scaled_points)

Have a look at one of the rendering tutorials for example usage of cameras and rotation function.

[gkioxari]

Hi @xingmimfl
I marked your question with a how to label, as it's not an issue or question regarding an implementation in the codebase, but it's a question about how to use PyTorch3D in your project. @nikhilaravi has been kind to provide some tips to help you, which can hopefully get you started. I will be closing this issue though as it's not pertinent to the codebase. Good luck with your project!