NoneType error on rendering a mesh with texture initialized with verts_rgb

[rahuldey91]

I am facing problems with rendering a mesh with Texture initialized using verts_rgb. As a toy example, I used the same code as in ##15 (comment) with the additional definition of a renderer.

from pytorch3d.utils import ico_sphere
from pytorch3d.structures import Meshes, Textures
from pytorch3d.renderer import (look_at_view_transform, OpenGLPerspectiveCameras, PointLights, DirectionalLights, Materials, RasterizationSettings, MeshRenderer, MeshRasterizer, TexturedSoftPhongShader)

# Initialize two ico spheres of different sizes
mesh1 = ico_sphere(3)  # (42 verts, 80 faces)
mesh2 = ico_sphere(4)  # (162 verts, 320 faces)
verts1, faces1 = mesh1.get_mesh_verts_faces(0)
verts2, faces2 = mesh2.get_mesh_verts_faces(0)

# Initalize the textures as an RGB color per vertex
tex1 = torch.ones_like(verts1) 
tex2 = torch.ones_like(verts2)
tex1[:, 1:] *= 0.0  # red
tex2[:, :2] *= 0.0  # blue

# Create one mesh which contains two spheres of different sizes.
# To do this we can concatenate verts1 and verts2
# but we need to offset the face indices of faces2 so they index
# into the correct positions in the combined verts tensor. 

# Make the red sphere smaller and offset both spheres so they are not overlapping
verts1 *= 0.25  
verts1[:, 0] += 0.8
verts2[:, 0] -= 0.5
verts = torch.cat([verts1, verts2])  #(204, 3)

#  Offset by the number of vertices in mesh1
faces2 = faces2 + verts1.shape[0]  
faces = torch.cat([faces1, faces2])  # (400, 3)

tex = torch.cat([tex1, tex2])[None]  # (1, 204, 3)
textures = Textures(verts_rgb=tex)

mesh = Meshes(verts=[verts], faces=[faces], textures=textures)

# Create a renderer
R, T = look_at_view_transform(dist=2.7, elev=0, azim=0)
R = R.repeat(batch_size, 1, 1)
T = T.repeat(batch_size, 1)
cameras = OpenGLPerspectiveCameras(device=self.device, R=R, T=T)

raster_settings = RasterizationSettings(image_size=224, blur_radius=0.0, faces_per_pixel=1, bin_size=None, max_faces_per_bin=None)
lights = PointLights(device=self.device, location=[[0.0, 0.0, 1.0]])

renderer = MeshRenderer(rasterizer=MeshRasterizer(cameras=cameras, raster_settings=raster_settings), shader=TexturedSoftPhongShader(device=self.device, cameras=cameras, lights=lights))

# Render the mesh
image = renderer(mesh)   # (1, H, W, 4)

When I try to render, I get the following error:

-> image = renderer(mesh)   # (1, H, W, 4)
(Pdb) c
Traceback (most recent call last):
  File "main.py", line 93, in <module>
    main()
  File "main.py", line 68, in main
    loss_train = trainer.train(epoch, loaders)
  File "~/3DMM/pytorchnet_3d/train.py", line 369, in train
    image = renderer(mesh)   # (1, H, W, 4)
  File "~/miniconda3/envs/pytorch3d/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "~/miniconda3/envs/pytorch3d/lib/python3.6/site-packages/pytorch3d/renderer/mesh/renderer.py", line 69, in forward
    images = self.shader(fragments, meshes_world, **kwargs)
  File "~/miniconda3/envs/pytorch3d/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "~/miniconda3/envs/pytorch3d/lib/python3.6/site-packages/pytorch3d/renderer/mesh/shader.py", line 269, in forward
    texels = interpolate_texture_map(fragments, meshes)
  File "~/miniconda3/envs/pytorch3d/lib/python3.6/site-packages/pytorch3d/renderer/mesh/texturing.py", line 43, in interpolate_texture_map
    faces_uvs = meshes.textures.faces_uvs_packed()
  File "~/miniconda3/envs/pytorch3d/lib/python3.6/site-packages/pytorch3d/structures/textures.py", line 147, in faces_uvs_packed
    return list_to_packed(self.faces_uvs_list())[0]
  File "~/miniconda3/envs/pytorch3d/lib/python3.6/site-packages/pytorch3d/structures/utils.py", line 116, in list_to_packed
    N = len(x)
TypeError: object of type 'NoneType' has no len()

However, the rendering works only when the Texture is initialized with verts_uvs, faces_uvs and texture_maps. From the error trace, it seems that Texture structure needs to be initialized with both verts_uvs as well as faces_uvs. Is that the case or am I missing something? Logically, verts_rgb alone should be able to fully define a texture too.

[nikhilaravi]

@rahuldey91 as explained in #112, there are different shaders for each texture type - the HardPhongShader calls a function interpolate_vertex_colors whereas the TexturedPhongShader calls a function interpolate_texture_map which expects the faces_uvs and verts_uvs.

For this case, you need to use the HardPhongShader or the SoftPhongShader.

We are making some updates to the texturing API and will add better error messaging for the different texturing options. The Shaders are the most flexible part of the rendering API, and you can easily create and customize your own so the ones we have provided are only examples.

See this table for an explanation of the shader options available currently.