Inverse rendering problem (From edge to vertex)

[fasogbon]

I know that the software can render image of a mesh. Is it possible to inverse project the image back to get the corresponding vertices. I have some detected silhouette edge of the mesh image and will like to get the corresponding vertex indexes. Thank you for your response

[pmh47]

Yes, but it is a little complicated. You re-render the mesh using the face indices as the vertex colours (with a suitable mapping from the integer indices to float colours); then, look up the colours of the pixels you care about, which tells you the corresponding face indices. For this to work, you need to ensure each vertex is used in at most one face. Then, to find which two vertices of the face correspond to your edge, you can do another rendering pass where for each face, one vertex is pure red, the second green and the third blue. What colour appears under your pixel tells you which of the vertices you are near; at an edge, one of the channels will be roughly zero, and the other two correspond to the vertices you're looking for.

If you care about speed, this can be done with just one additional pass instead of two, by putting the face indices into the red channel for all vertices, and setting green = 1, blue = 0 for the 1st vertex, and green = 0, blue = 1 for the 2nd in each face.

(For completeness, note you can also do this with less engineering but much more maths, by inverting the projection operation to give vectors corresponding to rays shot into the scene from each relevant pixel, then checking every face for intersection with those rays)

[fasogbon]

This repsonse is exactly what I need! Thanks