Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kinect Fusion under opecv wrappers #3406

Closed
vladsterz opened this issue Mar 6, 2019 · 10 comments

Comments

@vladsterz
Copy link

@vladsterz vladsterz commented Mar 6, 2019


Required Info
Camera Model D415
Operating System & Version Win 10
Platform PC/Raspberry Pi/ NVIDIA Jetson / etc..
SDK Version 2.17+

Issue Description

Has anyone built and run kinfu.exe ? I have much trouble with the latest realsense.dlls , in a way that I cannot run it. It crashes whenever a pipeline is created or is started.

@dorodnic

This comment has been minimized.

Copy link
Collaborator

@dorodnic dorodnic commented Mar 6, 2019

Hi @vladsterz
We will check. Its somewhat of a headache to build, requiring specialized build flags for both opencv and librealsense, so I admit we haven't really tested it lately.
@AnnaRomanov please take a look
Personally I found kinfu reconstruction somewhat disappointing (we also tried open3d) and I'm wondering if making small modifications to D435i+RTabMap (removing reliance on IMU and adjusting for D415) would produce better results.

@vladsterz

This comment has been minimized.

Copy link
Author

@vladsterz vladsterz commented Mar 7, 2019

Yes, I managed to build it after all, with librealsense 2.18.0. Results are indeed disappointing, even after I tried to run it with high accuracy preset. Fused data stop to make sense shortly after the recording starts and after a while it resets.

  • KinFu has some parameters, has anyone messed around with them?
  • Were the results from open3D better?
    Thx
@dorodnic

This comment has been minimized.

Copy link
Collaborator

@dorodnic dorodnic commented Mar 7, 2019

Were the results from open3D better?

Not sufficiently enough to publish. These methods seem to be extremely sensitive to rotations.
I also tried OpenCV RGBD module. As a cheap pose estimator it performs pretty well, but does not help you to stitch resulting point-clouds.

@vladsterz

This comment has been minimized.

Copy link
Author

@vladsterz vladsterz commented Mar 8, 2019

Kinfu is not that bad under the configuration of small distance between foreground and background actually.
image
This is with high accuracy preset and the result is better than I imagined.

@Jie2World

This comment has been minimized.

Copy link

@Jie2World Jie2World commented Mar 28, 2019

Hello @vladsterz Do you improve the recontruction? The poor result drive me crazy. If move the camera, the kinfu will reset soon which caused by getting zero matrix or nan/inf matrix A from getAB() function in icp->estimateTransformT()(fast-icp.cpp in opencv_rgbd). I try disable the optimization by adding setUseOptimized(false), which avoid getting a nan/inf matrix. But the zero matrix still exist. I am disappointed for these result. Have you try any library like PCL to reconstruct object with realsense d435?
Best regards,
Jie

@vladsterz

This comment has been minimized.

Copy link
Author

@vladsterz vladsterz commented Mar 28, 2019

Hello @Jie2World. I actually didn't looked in too much into the code or method itself. I made 2 modifications. i) I did change it to high accuracy in order have higher quality depth maps (laser power helps as well) and ii) I made it kind of offline process, reading depth maps from file.

@Jie2World

This comment has been minimized.

Copy link

@Jie2World Jie2World commented Mar 29, 2019

@vladsterz Sorry, I can't understand the meaning of offline process. Do you mean that the program save the depth maps when kinfu fail and then read them after kinfu reset? Or you just read the depth maps from other place? I set it high accuracy but also get disappointing result. If move the camera, the point cloud will become blurred. Have you try other library such as PCL? PCL has same kinect fusion, but it works for kinect and I have no idea to use it with RealSense D435.

@vladsterz

This comment has been minimized.

Copy link
Author

@vladsterz vladsterz commented Mar 29, 2019

Hello @Jie2World. No , I haven't tried PCL implementation, actually I didn't know there is one.Offline process means that I read depth files from a directory and use them as an input to kinfu, it may be helpful if you want to pick the frames you like.

@RealSenseCustomerSupport

This comment has been minimized.

Copy link
Collaborator

@RealSenseCustomerSupport RealSenseCustomerSupport commented May 5, 2019


@vladsterz Any other questions about this ticket? Looking forward to your update. Thanks!

@RealSenseCustomerSupport

This comment has been minimized.

Copy link
Collaborator

@RealSenseCustomerSupport RealSenseCustomerSupport commented May 9, 2019


@vladsterz Any other questions?

@vladsterz vladsterz closed this May 9, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.