Generating a heatmap of annotated failures using annotations from different object viewpoints. In practical use cases, annotated data from users come from different viewpoints because not all failures can be photographed and labeled from the same viewpoint.
All vertices (3rd image) are compared to the depth map (1st image) to display only the visible vertices (4th image).
In order to create a detailed heatmap, we need to refine the mesh into smaller faces. This is done by subdividing all triangles with an edge longer than a defined threshold.
Next, we calculate the location of all vertices in pixels for any given object pose and check which vertices are inside the bounding boxes.
The more often a face's vertices appears in a bounding box, the redder its color becomes.
