Kinect v2 / Kinect One

[SchmidL]

I've got a Kinect One and will rtabmap work with this version? I've read that the resolution is much better than the Kinect 360

[matlabbe]

To use with ROS: see http://official-rtab-map-forum.206.s1.nabble.com/rtabmap-for-kinect-v2-on-ROS-Indigo-tp61p260.html

Install driver
Follow ALL installation directives of the iai_kinect2 package. Make sure it is calibrated for better mapping results!

 Run

$ roslaunch rtabmap_ros rgbd_mapping_kinect2.launch
$ rosrun kinect2_bridge kinect2_bridge  _publish_tf:=true

To use with RTAB-Map standalone, libfreenect2 should be installed and RTAB-Map should be built with it (the cmake output should show "With Freenect2 = Yes"). Then under the source menu, just select "Kinect v2" -> "freeenect2" driver.

[fondastorker]

there is new driver for the kinect v2 on open ni2 :
https://github.com/kaorun55/OpenNI2-Kinect2Driver
i used this driver myself and its work good!
did your team work on it to support open ni2 for kinect v2?

[matlabbe]

Hello,

Not in the short term. I assume that you are on Windows. I would like to support it but at this time I don't have a computer to be able to test Openni2+K4W2+Kinect v2 on Windows 8 (the main issue on my side is that Windows doesn't recognize the Kinect with the USB 3 card installed, but it works on Linux with freenect2 above).

Well, hope I will have some functional hardware compatible with K4W2 soon.

[fondastorker]

Hello
Thanks for answering
yes im on windows 8.
and i tested openni2 on my windows and that's great !
im on project that need slam ... as i find out kinectv2 has 2x rang on depth sensor : about 8 meter and that's perfect on slam!
because of my project cant switching to Ubuntu .
im hardly waiting for your new version with openni2 support.

[matlabbe]

I am looking the official repository of OpenNI2, it seems that a driver for Kinect2 already exists (see here). If it is compatible with same OpenNI2 interface than Kinect or XTion, rtabmap can almost support Kinect2 already. The only thing I see is that image width/height in CameraOpenNI2 is hard coded to 640x480. I'll check if I can automate it.

[matlabbe]

The Kinect for XBOX 360 (model 1414) has the following formats (as shown by rtabmap when starting OpenNI2):

CameraOpenNI2: Depth video mode 0: fps=30, pixel=100, w=640, h=480
CameraOpenNI2: Depth video mode 1: fps=30, pixel=100, w=320, h=240
CameraOpenNI2: Depth video mode 2: fps=30, pixel=100, w=80, h=60
CameraOpenNI2: Color video mode 0: fps=12, pixel=200, w=1280, h=960
CameraOpenNI2: Color video mode 1: fps=30, pixel=200, w=640, h=480
CameraOpenNI2: Color video mode 2: fps=15, pixel=201, w=640, h=480

I'll check tomorrow at the lab for XTION Pro Live and Kinect for Windows (the first version).

[fondastorker]

Thanks
I will wait for your test.
And our time try to work on your open source to!

[fondastorker]

*team

[matlabbe]

More info:

Xtion Live Pro

Depth video mode 0: fps=30, pixel=100, w=320, h=240
Depth video mode 1: fps=30, pixel=101, w=320, h=240
Depth video mode 2: fps=60, pixel=100, w=320, h=240
Depth video mode 3: fps=60, pixel=101, w=320, h=240
Depth video mode 4: fps=30, pixel=100, w=640, h=480
Depth video mode 5: fps=30, pixel=101, w=640, h=480
Depth video mode 6: fps=30, pixel=100, w=160, h=120
Depth video mode 7: fps=30, pixel=101, w=160, h=120
Color video mode 0: fps=30, pixel=200, w=320, h=240
Color video mode 1: fps=30, pixel=201, w=320, h=240
Color video mode 2: fps=30, pixel=205, w=320, h=240
Color video mode 3: fps=15, pixel=200, w=320, h=240
Color video mode 4: fps=15, pixel=201, w=320, h=240
Color video mode 5: fps=15, pixel=205, w=320, h=240
Color video mode 6: fps=60, pixel=200, w=320, h=240
Color video mode 7: fps=60, pixel=201, w=320, h=240
Color video mode 8: fps=60, pixel=205, w=320, h=240
Color video mode 9: fps=30, pixel=200, w=640, h=480
Color video mode 10: fps=30, pixel=201, w=640, h=480
Color video mode 11: fps=30, pixel=205, w=640, h=480
Color video mode 12: fps=15, pixel=200, w=640, h=480
Color video mode 13: fps=15, pixel=201, w=640, h=480
Color video mode 14: fps=15, pixel=205, w=640, h=480
Color video mode 15: fps=30, pixel=200, w=1280, h=1024
Color video mode 16: fps=30, pixel=202, w=1280, h=1024
Color video mode 17: fps=30, pixel=201, w=1280, h=1024
Color video mode 18: fps=30, pixel=205, w=1280, h=1024
Color video mode 19: fps=30, pixel=200, w=1280, h=720
Color video mode 20: fps=30, pixel=202, w=1280, h=720
Color video mode 21: fps=30, pixel=200, w=1280, h=960
Color video mode 22: fps=30, pixel=202, w=1280, h=960

Kinect for Windows (Model 1517)

Depth video mode 0: fps=30, pixel=100, w=640, h=480
Depth video mode 1: fps=30, pixel=100, w=320, h=240
Depth video mode 2: fps=30, pixel=100, w=80, h=60
Color video mode 0: fps=12, pixel=200, w=1280, h=960
Color video mode 1: fps=30, pixel=200, w=640, h=480
Color video mode 2: fps=15, pixel=201, w=640, h=480

I can do something to select the right depth/color mode automatically but it would be nice if I can see for Kinect v2 the information as above (I can't test it here). Can you try to start RTAB-Map from a command window, open Preferences and set debugging level to INFO, select OpenNI2 as the driver then start. It may fails on start but check if you can see information like the following in the command window:

[ INFO] (2015-07-23 17:20:41) CameraRGBD.cpp:542::rtabmap::CameraOpenNI2::init() CameraOpenNI2: Depth video mode 0: fps=30, pixel=100, w=640, h=480
[ INFO] (2015-07-23 17:20:41) CameraRGBD.cpp:542::rtabmap::CameraOpenNI2::init() CameraOpenNI2: Depth video mode 1: fps=30, pixel=100, w=320, h=240
[ INFO] (2015-07-23 17:20:41) CameraRGBD.cpp:542::rtabmap::CameraOpenNI2::init() CameraOpenNI2: Depth video mode 2: fps=30, pixel=100, w=80, h=60
[ INFO] (2015-07-23 17:20:41) CameraRGBD.cpp:553::rtabmap::CameraOpenNI2::init() CameraOpenNI2: Color video mode 0: fps=12, pixel=200, w=1280, h=960
[ INFO] (2015-07-23 17:20:41) CameraRGBD.cpp:553::rtabmap::CameraOpenNI2::init() CameraOpenNI2: Color video mode 1: fps=30, pixel=200, w=640, h=480
[ INFO] (2015-07-23 17:20:41) CameraRGBD.cpp:553::rtabmap::CameraOpenNI2::init() CameraOpenNI2: Color video mode 2: fps=15, pixel=201, w=640, h=480
[ INFO] (2015-07-23 17:20:41) CameraRGBD.cpp:574::rtabmap::CameraOpenNI2::init() CameraOpenNI2: Using depth video mode: fps=30, pixel=100, w=640, h=480, H-FOV=1.082104 rad, V-FOV=0.848230 rad
[ INFO] (2015-07-23 17:20:41) CameraRGBD.cpp:578::rtabmap::CameraOpenNI2::init() CameraOpenNI2: AutoWhiteBalanceEnabled = 1
[ INFO] (2015-07-23 17:20:41) CameraRGBD.cpp:579::rtabmap::CameraOpenNI2::init() CameraOpenNI2: AutoExposureEnabled = 1
[ INFO] (2015-07-23 17:20:41) CameraRGBD.cpp:581::rtabmap::CameraOpenNI2::init() CameraOpenNI2: Exposure = 0
[ INFO] (2015-07-23 17:20:41) CameraRGBD.cpp:582::rtabmap::CameraOpenNI2::init() CameraOpenNI2: GAIN = 100
[ INFO] (2015-07-23 17:20:41) CameraRGBD.cpp:597::rtabmap::CameraOpenNI2::init() depth fx=532.569458 fy=531.541077
[ INFO] (2015-07-23 17:20:41) CameraRGBD.cpp:605::rtabmap::CameraOpenNI2::init() CameraOpenNI2: Using color video mode: fps=30, pixel=200, w=640, h=480, H-FOV=1.082104 rad, V-FOV=0.848230 rad

thx

[fondastorker]

our team hardly worked on your request but we just get this :
ERROR] (2015-07-24 05:46:25) CameraRGBD.cpp:538::rtabmap::CameraOpenNI2::init() CameraOpenNI2: Cannot open device

[fondastorker]

we work on your 0.8.10 version
i think this version s openni2 doesn't support kinect v2.
if you can update rtab s openni2 and build the test version with
https://github.com/kaorun55/OpenNI2-Kinect2Driver
or official as you say
https://github.com/occipital/OpenNI2/tree/kinect2/Source/Drivers/Kinect2
i will test again.
tanks.
so what next?

[fondastorker]

we continue the test out of rtab and just by openni 2 viewer
after pasting driver (https://github.com/kaorun55/OpenNI2-Kinect2Driver )
i executed simpleviewer.exe and got this:

After initialization:

Error - expect color and depth to be in same resolution: D: 512x424, C: 1920x108
0

i hop at least this can help ..

[matlabbe]

Thx for the info,
this is what I was wondering, if OpenNI2 does the registration. With the Kinect1, you can enable registration like that:

    if(!_device->isImageRegistrationModeSupported(openni::IMAGE_REGISTRATION_DEPTH_TO_COLOR))
    {
        UERROR("CameraOpenNI2: Device doesn't support depth/color registration.");
        _device->close();
        openni::OpenNI::shutdown();
        return false;
    }
    _device->setImageRegistrationMode(openni::IMAGE_REGISTRATION_DEPTH_TO_COLOR );

Then the registration is done by the sensor. With the kaorun55's Kinect2 driver, can you get two images of the same size registered together? If no, rtabmap can do calibration/registration of the kinect2 but it requires to modify a little more the CameraOpenNI2 code, and do something like what I did with the libfreenect2 driver. With libfreenect2, you don't even need to do calibration, it provides a default registration. Well if the OpenNI2 driver cannot give registered color/depth images, you may want to give a try with libfreenect2 (Windows install here).

[fondastorker]

tanks.
if i see correctly , you tell that libfreenect2 is beter that openni2 in calibration and what about result ?
then one of our problem is must of our team's people was working on Ubuntu and now we have to switch to windows , and our programer have problem with libfreenect2 installs steps. that's was long and complicated.
is there any way to install rtab and libfreenect in single exe file?

[fondastorker]

for appreciation
our webdesign team are working on your skin web page
please give us your email to send a previous .

[matlabbe]

RTAB-Map needs registration between the color and the depth image. libfreenect2 does this registration by default.

I've tested RTAB-Map on Windows 8 with libfreenect2 and my Kinect v2 worked out-of-the-box (no need of the Windows Kinect SDK, which didn't detect the Kinect on my computer). However, getting all dependencies correctly installed is quite long (if we start only with Visual Studio: install OpenCV, build PCL and its dependencies, build libfreenect2 with its dependencies, then built RTAB-Map). I'll check next week if I can package the binaries so only libusbK step of libfreenect2 driver installation should be done.

Oh thank you for looking for the design of the web page of the project. My email is my GitHub username + @gmail.com.

cheers

[matlabbe]

Just released new Windows binaries with libfreenect2 included. I didn't test on a fresh Windows 8 computer, but you would only need to install Kinect v2 with the libusbK driver. See here for instructions.

[fondastorker]

Thanks my friend
Our developers will test it and send you the results.

[eric-schleicher]

@matlabbe was Kinect2 working out of the box on windows ever verified?

I initially got it setup running with ROS on linux, but was attempting to build a few maps from windows.
the kinect2 is verified working on this same installation (though MS SDK samples).

I get the following error on windows 8.1 system.

[ERROR] (2017-03-24 18:31:09) CameraRGBD.cpp:1547::rtabmap::CameraFreenect2::init() CameraFreenect2: no device connected or failure opening the default one! Note that rtabmap should link on libusb of libfreenect2. Tip, before starting rtabmap: "$ export LD_LIBRARY_PATH=~/libfreenect2/depends/libusb/lib:$LD_LIBRARY_PATH"

[matlabbe]

I've found a computer with USB3 that I can safely format and do a fresh install of Windows 10. I updated the Installation errors section in Windows section of the Installation page. It seems that the current binaries don't work with latest version of usbDK. Version v1.00-12 should be installed to work (this is the version installed on my Windows development machine).

If it doesn't work, what is the log info just before that error on the terminal?

cheers,
Mathieu

[eric-schleicher]

I've been able to confirm that the version @matlabbe listed above (v1.00-12) works with RTABMAP on Windows 10x64 version 0.12.4 and the kinect v2.

Woot!

[Abduoit]

Hi @matlabbe

I am using ros ubuntu and kinect v2 to do mapping with rtabmap, the kinect v2 works fine, however when I try to do new mapping and odometry by writing in terminal

$ rtabmap

then, under the source menu I can't select "Kinect v2" -> "freeenect2" driver. It is dis-activated.

I checked my catkin_ws/src I found that I have libfreenect2 but I don't know how can I get freenect2.
In addition, I already have both folders libfreenect2 and freenect2, they are out side the catkin_ws/src.

Could you please tell me how to activate the options under the source menu?

Thanks for your great help

[matlabbe]

The standalone rtabmap version installed with ros doesn't have freenect2 available built-in. You can use the ros interface to use kinect v2, follow this tutorial (iai_kinect2 package should be installed).

If you want the option enabled in the standalone, you should build rtabmap from source. See instructions here (make sure freenect2 is detected in cmake status).

cheers