Open3D visualizer like visualization quality

[salmanmaq]

Cross-posting from the model-viewer github discussions: google/model-viewer#3098

I am currently doing RGBD integration based reconstruction using Open3D. When I visualize the reconstructed models with Open3D, the results are great! The colors are accurate as they were in the video source. See below:

However, when I visualize using Google's model-viewer, the results are different especially the colors are not as rich and there are geometrical artifacts that do not seem to be present in the Open3D based visualization (for example above the red line in the image below). I can adjust the lightning in the model viewer a bit but the results are almost never as accurate as Open3D. Maybe experimenting more with lightning here will help, but it won't resolve the geometrical artifacts. Screenshot from model-viewer below:

I am using GLTF file extension. I would like to get results as close to the Open3D visualizer as possible and be able to display them in the browser. Any tips and suggestions on how to do that or get there? I don't know too much about computer graphics but I am willing to learn more if that helps me with this.

[errissa]

I am not familiar with what Google's model-viewer is. Would you please include a link? I don't know why are you seeing a red line in one viewer but not the other. However, it looks to me like the difference between the two images is that one (Open3D) is rendered with sRGB colors and the other is not.

[salmanmaq]

Oh, sorry I should have pasted a link. Here it is: https://github.com/google/model-viewer and https://modelviewer.dev/

The red line was drawn by me in the image actually to depict the portion of the scan where I see the artifacts in model-viewer. I don't have much idea about how either Open3D and model-viewer render the models. But that is what I am trying to understand to be able to hopefully get Open3D like rendering in model-viewer as well. I will have a look at sRGB too. Thank you for your comment.

[errissa]

Thanks for the links and for clarifying about the red line - I misunderstood originally!

It appears that model-viewer uses three.js as its rendering engine. Open3D uses Filament. I took a quick look at three.js and model-viewer but it's very hard to tell why model-viewer is rendering some parts of your model differently, there are many possible reasons why. As far as the color difference goes it looks like three.js does tone mapping as does Open3D but they probably don't use the same tone mapping algorithms so the colors may never match exactly. I also still believe it could be an sRGB-related issue.

How are you visualizing the reconstruction with Open3D? Another possibility is that you're visualizing in Open3D with no lighting so you are only see the colors of the point cloud but model-viewer is applying lighting. As far as I can tell, there is no way to disable lighting in model-viewer.

[salmanmaq]

Thank you so much! This is very useful. For Open3D, I am simply doing this via the Python API:

import open3d as o3d
mesh = o3d.io.read_triangle_mesh('path_to_mesh.gltf')
o3d.visualization.draw_geometries([mesh])

I am not sure if the problem is with sRGB. I have tried other web viewers as well that offer sRGB colors as well, but the results are barely different. Filament, however, seems really interesting! I will try to use it for rendering natively. Since they have a JavaScript API, we might be able to use that for rendering in the browser for better results (if we obtain them). Thanks for your comments and help! Any other tips that might b helpful for browser-based visualization could be helpful too. :)

[errissa]

Please try o3d.visualizaiton.draw([mesh])

[salmanmaq]

Sorry for the late response. I am using WSL2 and even though I am able to view Linux windows through X-server, using o3d.visualization.draw([mesh]) results in a segmentation fault. I suppose I may get back to this thread in the future when I am able to test this. Thank you for the guidance.