3d scanner with Reconstruction system and measurement - Question

[giginap]

Hi,
Is it Possible to reconstruct an object from a depth camera(realsense D400) and convert into a mesh model? and then make a mesurement process on it.

Thanks

[yxlao]

Please refer to this release video for an example on reconstruction with realsense camera: https://www.youtube.com/watch?v=6FkbxLLZyFQ

[giginap]

thank you yxlao,

are you planning to develop a real-time reconstruction and also do you have any advice to optimize in spread the reconstruction system?

[qianyizh]

We don't have plan for a real-time reconstruction system yet. The main reason is that the current system is built completely on C++ and without GPU acceleration it is very hard to make it real time.

The best practice in terms of optimization is to sample less frames (e.g., sample 10fps), and use larger fragments (e.g., 3s fragments). This way you reduce the size of the graph. And the accuracy does not drop much. Many very good previous work does similar things too (e.g., RGB-D odometry).

For global registration, FastGlobalRegistration is much faster than RANSAC but it sacrifices some accuracy. So be careful of it.

For ICP, always use a HybridSearchParameter, it is much faster than KNN and RadiusSearch. But I think this has been made default in our tutorial.

For integration, use ScalableTSDF instead of UniformTSDF. The former is much faster.

There are some parameters to tune. I think the ones in the tutorials have been tuned to achieve a good balance between accuracy, robustness, and speed.

[rafael2k]

Hi all,
I don't really understand the need of the fragment. I'm writing an offline scanner, and I'm capturing from Kinect2 using freenect, then aligning the PCs created from the captured RGB+D using FastGlobalRegistration. The transformation matrix from the registration result I feed the volume.Integrate extrinsic. Is this correct?
It seems almost working here.
Btw, I'm coding in C++, so the python examples does not really help me, and the odometry example which uses the Jacobian only accept RBGDImage with 1 channel and 4 bytes per channel in both color and depth components, my RGBDImage uses color Image with 3 channels and 1 byte per channel.

[qianyizh]

The purpose of fragment is to integrate a number of consecutive RGB-D frames to create geometry with less noise. If in your use case the raw RGB-D frames work well then there is no need for fragments.

[ttsesm]

@qianyizh or anyone else that might knows. I am want to create an online (for the moment offline is also fine) volumetric 3D map of an environment using the realsense. I followed the instructions from the Reconstruction System tutorial but my output is quite bad as you can see below:

I have recorded the scene quite a few times but the result is always quite bad. Thus, I was wondering if you guys have any hints regarding how I can improve the reconstruction. I am using the default settings as provided in the tutorial scripts.

Also another question is how the open3d approach compares to the fastfusion solution in terms of accuracy (my guess is that open3d is better here) and speed (I guess that fastfusion is better here) and whether these two could be combined somehow in order to provide a cpu-based real time solution in open3d as well.

[qianyizh]

I haven't got much first hand experience of Realsense. (TBH, I think its accuracy is pretty bad, at least in its DS400 series.) @syncle has some success with realsense cameras. He may be able to help you or give you a few pointers to start with.

[germanros1987]

Issue closed due to lack of activity. Feel free to open a new issue if you still consider this is a critical matter.