How to get per vertex rgb color vector from textured Mesh

[Pooya448]

Hi everyone, I am relatively new to this field.

Using pytorch3d I have a textured mesh, which I added the texture information from png texture maps, and after rendering everything seems fine. But I want to extract per vertex rgb color value (rgb features) from this mesh.

When I use verts_uvs_padded() method it returns texture info corresponding to texture maps, But I want actual rgb values for every vertex: Tensor of shape (V, 3)

Is there a way to do this?
Thank you in advance

[bottler]

See issue #854 and my first comment. (Note that asking for a TexturesVertex is equivalent to a color per vertex).

[Pooya448]

See issue #854 and my first comment. (Note that asking for a TexturesVertex is equivalent to a color per vertex).

Thank you for your quick reply.

Ok, but my ultimate goal is that convert the textured mesh to a colored point cloud, so I can render it with a point renderer.
I thought that taking vertices and per vertex colors from the mesh and then creating a pointcloud object would do the trick.
Is there a better way to do this?

Please take a look at my code which is based on your comment:

verts_colors_packed = torch.ones_like(mesh.verts_packed())
verts_colors_packed[mesh.faces_packed()] = mesh.textures.faces_verts_textures_packed()  # (*)

verts = mesh.verts_packed()
norms = mesh.verts_normals_packed()
rgb = verts_colors_packed

colored_ptc = Pointclouds(points=[verts], normals=[norms], features=[rgb])

Which I can render this pointcloud, but the render results are not good enough.

[bottler]

If you want more than just the vertices, then maybe you could use the sample_points_from_meshes operation to make a pointcloud from many points on the mesh?

https://github.com/facebookresearch/pytorch3d/blob/v0.6.0/pytorch3d/ops/sample_points_from_meshes.py

[Pooya448]

Is this a built-in method which I can use?

[bottler]

yes. from pytorch3d.ops import sample_points_from_meshes

[Pooya448]

Thank you very much for your quick and effective answers. I am closing this.

[Pooya448]

I am trying to sample points from a textured mesh, returning both normals and colors. the built in ops.sample_points_from_meshes samples from the points which are on the surface of the mesh.
I need to sample points from a normal distribution with a standard deviation of 5 cm from the mesh surface (i.e. sampling from points which are 5cm outer or inner the mesh surface)

is there a function in pytorch3d to do this? If not, is there a way?

Thanks.

[bottler]

This is a new question. You mean to exactly sample uniformly over the volume which is within 5cm of the mesh? I can only think of rejection sampling as a solution (i.e. sample from an axis-aligned cuboid surrounding 5cm around the mesh and select the points in the volume e.g. using pytorch3d.loss.point_mesh_face_distance).

[Pooya448]

What about I sample points from the mesh surface and then add a gaussian noise to the points? can this achieve the goal?

[bottler]

I didn't read your previous comment precisely. You aren't looking for uniform over a volume. Still, it all depends on your exact goal, which isn't fully explained. E.g. the distribution can't both have a distance from the mesh which is "normal with a standard deviation of 5 cm" and be within 5cm outer or inner the mesh surface, because lots of a normal distribution is beyond 1 standard deviation.

If you sample from the surface and then add noise, then a small volume which is 1cm from two different flattish parts of the mesh will have about twice the probability as an equal-sized small volume which is 1cm from a single flattish part of the mesh. Is that good? Acceptable?

[Pooya448]

I think yes.

Also, I'm trying to render normals of a mesh(or poitcloud) into an image. Can this be achieved by pytorch3d?

[bottler]

Do you mean you want to visualize the mesh along with normals as little arrows? If so, and you don't care about differentiability, that sounds useful but isn't in PyTorch3D. I think you can make this happen with plotly or matplotlib, and if you get this working it would be interesting.

[bottler]

In future please make sure you open a new issue for an unrelated question.

[nikhilaravi]

@Pooya448 closing this issue as @bottler has answered all your questions in a lot of detail! Please reopen or create a new issue if you have any further questions.

[8bitnand]

Im having issues in displaying textured mesh in plotly graph,
can you share me your code