Some wired pink-green artifact

[YotamWolf]

Hi thanks for the amazing project, the best I've seen so far.

On some of the 3D models I'm trying I receive som weird pink-green artifact like salt and pepper noise, any idea why is that

happening?

[AmanKishore]

I'm getting a similar issue:

[pengwangucla]

+1

[YuliangXiu]

+1

[qzq2514]

Differentiable rendering has reconstruction error under the api of kaolin，this is the main reason to cause this green artifact. Pink artifact is caused by its inaccurate method to calculate trimap.

[pengwangucla]

Differentiable rendering has reconstruction error under the api of kaolin，this is the main reason to cause this green artifact. Pink artifact is caused by its inaccurate method to calculate trimap.

any way to fix these issue ? e.g. for the first issue, should we use pytorch3d differential renderring or there is a corrected way.

[qzq2514]

Differentiable rendering has reconstruction error under the api of kaolin，this is the main reason to cause this green artifact. Pink artifact is caused by its inaccurate method to calculate trimap.

any way to fix these issue ? e.g. for the first issue, should we use pytorch3d differential renderring or there is a corrected way.

I haven't try pytorch3d differential renderring, but I think this error may still exist for any texture reconstruction under differentiable rendering, but you can try to map the view image to texture space directly, just like the inverse of kaolin's texture_mapping

[reynoldscem]

I am also experiencing this. Has anyone found a solution?

[YuzhiChen001]

Differentiable rendering has reconstruction error under the api of kaolin，this is the main reason to cause this green artifact. Pink artifact is caused by its inaccurate method to calculate trimap.

"inaccurate method to calculate trimap",I want to know why the author's method is inaccurate?How do you think it should be calculated?thanks.

[qzq2514]

Differentiable rendering has reconstruction error under the api of kaolin，this is the main reason to cause this green artifact. Pink artifact is caused by its inaccurate method to calculate trimap.

"inaccurate method to calculate trimap",I want to know why the author's method is inaccurate?How do you think it should be calculated?thanks.

author's method to get uncolored mask is to calculate difference between textured uvmap and base color, this will be mostly affected by the color of texture. The correct method is to use reverse process of rasterizing.

[smha1012]

This problem occurs because the mesh color cannot be optimized perfectly in trainer.py, i guess. However, I think this is not a fundamental reason and it seems closely related to the normal parameters from the mesh(=self.mesh_model.get_params() in Adam optimizer) and the train/eval_grid_size configs.

Therefore, since each person uses different kinds of mesh, it will be very difficult to find a perfect solution.

[qzq2514]

Differentiable rendering has reconstruction error under the api of kaolin，this is the main reason to cause this green artifact. Pink artifact is caused by its inaccurate method to calculate trimap.

"inaccurate method to calculate trimap",I want to know why the author's method is inaccurate?How do you think it should be calculated?thanks.

author's method to get uncolored mask is to calculate difference between textured uvmap and base color, this will be mostly affected by the color of texture. The correct method is to use reverse process of rasterizing.

We have in part solved this problem, you can find this in https://github.com/OpenTexture/Paint3D