Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenNI2 driver cleaned up #523

Merged
merged 12 commits into from Jan 25, 2016
Merged

OpenNI2 driver cleaned up #523

merged 12 commits into from Jan 25, 2016

Conversation

xlz
Copy link
Member

@xlz xlz commented Jan 4, 2016

Cleaned up from #518. Previous PR #302. Discussion #243.

Build tested on Debian. Tested on MacOSX.

Still need Windows testing, and docs for installation on Windows.

@floe floe added this to the 0.2 milestone Jan 4, 2016
@hanyazou hanyazou mentioned this pull request Jan 14, 2016
@xlz
Copy link
Member Author

xlz commented Jan 21, 2016

This should be merged pretty soon after the required testing.

From libfreenect 89f77f6d2c23876936af65766a4c140898bc3ea8
Test with /opt/OpenNI2/Tools/NiViewer.
@xlz
Copy link
Member Author

xlz commented Jan 21, 2016

v2: Add a .so.0 suffix for the library so OpenNI2 can find it. Fix compiler warnings.

Tested with NiViewer2. Did not test with pcl_openni2_viewer because it is not packaged.

Still needs Windows testing.

@xlz
Copy link
Member Author

xlz commented Jan 22, 2016

@hanyazou There are two problems I found in your code. You are including extra headers in .hpp files for functions that are only referenced in .cpp files. If there is no reference to a function in the hpp file, the hpp file should not include that header.

Also, VideoStream.hpp contains method definitions, and this header is include twice in different cpp files. This will produce duplicate object code. You should not put method definitions in header files.

Try to avoid the problems next time.

I found these problems exist in the original code, so they are not your problems.

@xlz
Copy link
Member Author

xlz commented Jan 22, 2016

@hanyazou Can you provide some instructions on where and how to install the built libfreenect2-openni2.dylib on Mac OSX?

@xlz
Copy link
Member Author

xlz commented Jan 22, 2016

v3: Fixed MSVC FTBFS. Added Windows installation instructions.

TODO: think through proper Linux and MacOSX installation instructions.

Because substantial amount of code is changed, this needs retesting on Linux and Mac OSX.

@hanyazou
Copy link
Contributor

@xlz

Can you provide some instructions on where and how to install the built libfreenect2-openni2.dylib on Mac OSX?

I believe $OPENNI2_REDIST/OpenNI2/Drivers/ is the driver repository.
('brew install openni2' sugests OPENNI2_REDIST=/usr/local/lib/ni2.)

While /usr/local/lib/ni2/OpenNI.ini might be able to specify the repository path, the default is OpenNI2/Drivers/. You should install the library to the default driver repository path.

@xlz
Copy link
Member Author

xlz commented Jan 23, 2016

v4:

  • Moved more method definitions out of headers
  • Install RPATH in libfreenect2-openni2.so if libfreenect2.so is not installed in /usr/local
  • Now use make install-openni2 to install libfreenect2-openni2.so. This driver can't be in the default install target for non-root Linux installation and Windows installation.
  • Finished installation docs.

@xlz
Copy link
Member Author

xlz commented Jan 23, 2016

@hanyazou How does this work on Mac OSX now?

@hanyazou
Copy link
Contributor

@xlz it seems not to work. The .dylib is installed to /usr/local/lib/openni2/. Please refer full build and test log attached.

build.log.txt

@xlz
Copy link
Member Author

xlz commented Jan 24, 2016

Now use make install-openni2 to install libfreenect2-openni2.so.

Have you done that?

@hanyazou
Copy link
Contributor

@xlz 'make install-openni2' copies libfreenect2-openni2.dylib into /usr/local/lib/ni2/OpenNI2/Drivers/ and the NiViewer works fine!
(make install-openni2 did not require sudo in my env. the target directories were owned by me.)

@xlz
Copy link
Member Author

xlz commented Jan 24, 2016

One last change (v5): make install will copy libfreenect2-openni2.so to ${CMAKE_INSTALL_PREFIX}/lib/OpenNI2/Drivers/ instead of .../lib/openni2. This name is easier to understand when compared with /usr/lib/OpenNI2/Drivers.

I'll test this on linux and windows on Monday. If eveything is already, I'll merge this then.

@xlz
Copy link
Member Author

xlz commented Jan 25, 2016

Tests OK on Linux and Windows.

xlz added a commit that referenced this pull request Jan 25, 2016
@xlz xlz merged commit 7aa52cf into OpenKinect:master Jan 25, 2016
@xlz xlz deleted the openni2 branch January 25, 2016 18:08
@kruss84
Copy link

kruss84 commented Jan 26, 2016

I'm trying to convert an image from openni::VideoFrameRef to cv::Mat.
When I use the examples given for color and depth, I just get gray images.
Resolution of the images is correct, only the data is bad.

http://stackoverflow.com/questions/15966722/capture-rgb-from-kinnect-with-openni-and-show-with-opencv
http://answers.opencv.org/question/33513/opencv-asus-xtion-sensor/

When I start the program SimpleRead of Openni2, the depth image looks very good.
The readout of the Kinect v2 is working perfectly.

I would appreciate any suggestions or ideas to solve this problem.

@xlz
Copy link
Member Author

xlz commented Jan 26, 2016

@hanyazou is responsible to OpenNI2 related questions.

@GerardusHuang
Copy link

I have followed the windows instructions. Every thing is ok except I can not find any file named "libfreenect2-openni2.dll".
I have checked "BUILD_OPENNI2_DRIVER" and run in cmake GUI, also do not generated any project named with "openni2".

@xlz
Copy link
Member Author

xlz commented Jan 29, 2016

Post output of the CMake configure log.

@GerardusHuang
Copy link

The C compiler identification is MSVC 18.0.31101.0
The CXX compiler identification is MSVC 18.0.31101.0
Check for working C compiler using: Visual Studio 12 2013 Win64
Check for working C compiler using: Visual Studio 12 2013 Win64 -- works
Detecting C compiler ABI info
Detecting C compiler ABI info - done
Check for working CXX compiler using: Visual Studio 12 2013 Win64
Check for working CXX compiler using: Visual Studio 12 2013 Win64 -- works
Detecting CXX compiler ABI info
Detecting CXX compiler ABI info - done
Detecting CXX compile features
Detecting CXX compile features - done
using tinythread as threading library
Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE)
Found LibUSB: G:/DevelopInstalls/sdk/Kinect/libfreenect/libfreenect2/depends/libusb/x64/Release/dll/libusb-1.0.lib
Performing Test TURBOJPEG_WORKS
Performing Test TURBOJPEG_WORKS - Success
Found TurboJPEG: C:/libjpeg-turbo64/lib/turbojpeg.lib
Found GLFW3: G:/DevelopInstalls/sdk/Kinect/libfreenect/libfreenect2/depends/glfw/lib-vc2013/glfw3dll.lib
Found OpenGL: opengl32
Looking for CL_VERSION_2_0
Looking for CL_VERSION_2_0 - found
Found OpenCL: F:/Program Files (x86)/Intel/OpenCL SDK/4.6/lib/x64/OpenCL.lib (found version "2.0")
Performing Test COMPILER_HAS_DEPRECATED_ATTR
Performing Test COMPILER_HAS_DEPRECATED_ATTR - Failed
Performing Test COMPILER_HAS_DEPRECATED
Performing Test COMPILER_HAS_DEPRECATED - Success
Linking with these libraries:
G:/DevelopInstalls/sdk/Kinect/libfreenect/libfreenect2/depends/libusb/x64/Release/dll/libusb-1.0.lib
C:/libjpeg-turbo64/lib/turbojpeg.lib
G:/DevelopInstalls/sdk/Kinect/libfreenect/libfreenect2/depends/glfw/lib-vc2013/glfw3dll.lib
opengl32
F:/Program Files (x86)/Intel/OpenCL SDK/4.6/lib/x64/OpenCL.lib
Found Doxygen: C:/Program Files/doxygen/bin/doxygen.exe (found version "1.8.9.1")
Configurating examples
Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE)
Using in-tree freenect2 target
Found OpenNI2: C:/Program Files/OpenNI2/Lib
Configuring done
Generating done

@xlz
Copy link
Member Author

xlz commented Jan 29, 2016

Post a list of all files recursively under the build directory.

@GerardusHuang
Copy link

│ ALL_BUILD.vcxproj
│ ALL_BUILD.vcxproj.filters
│ CMakeCache.txt
│ cmake_install.cmake
│ file.txt
│ freenect2.pc
│ freenect2.vcxproj
│ freenect2.vcxproj.filters
│ freenect2Config.cmake
│ generate_resources_tool.vcxproj
│ generate_resources_tool.vcxproj.filters
│ INSTALL.vcxproj
│ INSTALL.vcxproj.filters
│ install_manifest.txt
│ libfreenect2.sdf
│ libfreenect2.sln
│ resources.inc.h
│ ZERO_CHECK.vcxproj
│ ZERO_CHECK.vcxproj.filters

├─bin
│ └─MinSizeRel
│ freenect2.dll
│ generate_resources_tool.exe
│ glfw3.dll
│ libusb-1.0.dll
│ Protonect.exe
│ turbojpeg.dll

├─CMakeFiles
│ │ cmake.check_cache
│ │ CMakeError.log
│ │ CMakeOutput.log
│ │ feature_tests.bin
│ │ feature_tests.cxx
│ │ generate.stamp
│ │ generate.stamp.depend
│ │ generate.stamp.list
│ │ TargetDirectories.txt
│ │
│ ├─20c456e0b6f062e6a8052eefdc0c43ec
│ │ resources.inc.h.rule
│ │
│ ├─3.2.2
│ │ │ CMakeCCompiler.cmake
│ │ │ CMakeCXXCompiler.cmake
│ │ │ CMakeDetermineCompilerABI_C.bin
│ │ │ CMakeDetermineCompilerABI_CXX.bin
│ │ │ CMakeRCCompiler.cmake
│ │ │ CMakeSystem.cmake
│ │ │
│ │ ├─CompilerIdC
│ │ │ │ CMakeCCompilerId.c
│ │ │ │ CompilerIdC.exe
│ │ │ │ CompilerIdC.vcxproj
│ │ │ │
│ │ │ └─Debug
│ │ │ │ CMakeCCompilerId.obj
│ │ │ │
│ │ │ └─CompilerIdC.tlog
│ │ │ cl.command.1.tlog
│ │ │ CL.read.1.tlog
│ │ │ CL.write.1.tlog
│ │ │ CompilerIdC.lastbuildstate
│ │ │ link.command.1.tlog
│ │ │ link.read.1.tlog
│ │ │ link.write.1.tlog
│ │ │
│ │ └─CompilerIdCXX
│ │ │ CMakeCXXCompilerId.cpp
│ │ │ CompilerIdCXX.exe
│ │ │ CompilerIdCXX.vcxproj
│ │ │
│ │ └─Debug
│ │ │ CMakeCXXCompilerId.obj
│ │ │
│ │ └─CompilerIdCXX.tlog
│ │ cl.command.1.tlog
│ │ CL.read.1.tlog
│ │ CL.write.1.tlog
│ │ CompilerIdCXX.lastbuildstate
│ │ link.command.1.tlog
│ │ link.read.1.tlog
│ │ link.write.1.tlog
│ │
│ ├─3c4aa58db9105a46851b08dc22b5a663
│ │ doc.rule
│ │ INSTALL_force.rule
│ │
│ ├─4a8a10df6077e5d195688f9f8301f111
│ │ generate.stamp.rule
│ │ INSTALL_force.rule
│ │
│ ├─b047920d80ab002551325f77e500032f
│ │ INSTALL_force.rule
│ │
│ └─CMakeTmp
├─doc
│ │ cmake_install.cmake
│ │ doc.vcxproj
│ │ doc.vcxproj.filters
│ │ Doxyfile
│ │ INSTALL.vcxproj
│ │ INSTALL.vcxproj.filters
│ │
│ └─CMakeFiles
│ generate.stamp
│ generate.stamp.depend

├─examples
│ │ ALL_BUILD.vcxproj
│ │ ALL_BUILD.vcxproj.filters
│ │ cmake_install.cmake
│ │ INSTALL.vcxproj
│ │ INSTALL.vcxproj.filters
│ │ libfreenect2_examples.sln
│ │ Protonect.vcxproj
│ │ Protonect.vcxproj.filters
│ │
│ ├─CMakeFiles
│ │ generate.stamp
│ │ generate.stamp.depend
│ │
│ ├─MinSizeRel
│ └─Protonect.dir
│ └─MinSizeRel
│ │ flextGL.obj
│ │ Protonect.log
│ │ Protonect.obj
│ │ viewer.obj
│ │
│ └─Protonect.tlog
│ cl.command.1.tlog
│ CL.read.1.tlog
│ CL.write.1.tlog
│ custombuild.command.1.tlog
│ custombuild.read.1.tlog
│ custombuild.write.1.tlog
│ link.command.1.tlog
│ link.read.1.tlog
│ link.write.1.tlog
│ Protonect.lastbuildstate

├─freenect2.dir
│ └─MinSizeRel
│ │ command_transaction.obj
│ │ cpu_depth_packet_processor.obj
│ │ depth_packet_processor.obj
│ │ depth_packet_stream_parser.obj
│ │ double_buffer.obj
│ │ event_loop.obj
│ │ flextGL.obj
│ │ frame_listener_impl.obj
│ │ freenect2.log
│ │ libfreenect2.obj
│ │ logging.obj
│ │ opencl_depth_packet_processor.obj
│ │ opengl_depth_packet_processor.obj
│ │ packet_pipeline.obj
│ │ registration.obj
│ │ resource.obj
│ │ rgb_packet_processor.obj
│ │ rgb_packet_stream_parser.obj
│ │ tinythread.obj
│ │ transfer_pool.obj
│ │ turbo_jpeg_rgb_packet_processor.obj
│ │ usb_control.obj
│ │
│ └─freenect2.tlog
│ cl.command.1.tlog
│ CL.read.1.tlog
│ CL.write.1.tlog
│ custombuild.command.1.tlog
│ custombuild.read.1.tlog
│ custombuild.write.1.tlog
│ freenect2.lastbuildstate
│ freenect2.write.1u.tlog
│ link.command.1.tlog
│ link.read.1.tlog
│ link.write.1.tlog

├─generate_resources_tool.dir
│ └─MinSizeRel
│ │ generate_resources.obj
│ │ generate_resources_tool.log
│ │
│ └─generate.FADD3BC8.tlog
│ cl.command.1.tlog
│ CL.read.1.tlog
│ CL.write.1.tlog
│ custombuild.command.1.tlog
│ custombuild.read.1.tlog
│ custombuild.write.1.tlog
│ generate_resources_tool.lastbuildstate
│ link.command.1.tlog
│ link.read.1.tlog
│ link.write.1.tlog

├─install
│ ├─bin
│ │ freenect2.dll
│ │ glfw3.dll
│ │ libusb-1.0.dll
│ │ Protonect.exe
│ │ turbojpeg.dll
│ │
│ ├─include
│ │ └─libfreenect2
│ │ config.h
│ │ export.h
│ │ frame_listener.hpp
│ │ frame_listener_impl.h
│ │ libfreenect2.hpp
│ │ logger.h
│ │ packet_pipeline.h
│ │ registration.h
│ │
│ └─lib
│ │ freenect2.lib
│ │
│ ├─cmake
│ │ └─freenect2
│ │ freenect2Config.cmake
│ │
│ └─pkgconfig
│ freenect2.pc

├─lib
│ └─MinSizeRel
│ freenect2.exp
│ freenect2.lib

├─libfreenect2
│ config.h
│ export.h

├─MinSizeRel
└─x64
└─MinSizeRel
├─ALL_BUILD
│ │ ALL_BUILD.log
│ │
│ └─ALL_BUILD.tlog
│ ALL_BUILD.lastbuildstate
│ custombuild.command.1.tlog
│ custombuild.read.1.tlog
│ custombuild.write.1.tlog

├─INSTALL
│ │ INSTALL.log
│ │
│ └─INSTALL.tlog
│ custombuild.command.1.tlog
│ custombuild.read.1.tlog
│ custombuild.write.1.tlog
│ INSTALL.lastbuildstate

└─ZERO_CHECK
│ ZERO_CHECK.log

└─ZERO_CHECK.tlog
custombuild.command.1.tlog
custombuild.read.1.tlog
custombuild.write.1.tlog
ZERO_CHECK.lastbuildstate

@xlz
Copy link
Member Author

xlz commented Jan 29, 2016

If you followed the instructions, there would not be "MinSizeRel".

@xlz
Copy link
Member Author

xlz commented Jan 29, 2016

rm -rf * everything under the build directory, and redo cmake.

@GerardusHuang
Copy link

I had a try on another computer today, but still no "libfreenect2-openni2.dll" :(
The difference is that I use libusb latest release, not your cmd script, because it always occurs compile error.
Maybe no support on windows 10 ?
I will have a try on window 8.1 soon.

@xlz
Copy link
Member Author

xlz commented Jan 30, 2016

What error?

If you don't follow the instructions, you are expected to be capable to follow and debug the script in spirit and not ask question here.

@mennayahia
Copy link

Hello,
great work guys.

I am trying to use Kinect v2 to build applications on linux to detect and track skeleton, using libfreenect2.
Your driver seems like the solution, but I am a bit confused, so if you can direct me to some instructions as how to install/build your driver with libfreenect2, i.e how to make it work, this will be really helpful.

thanks

@hanyazou
Copy link
Contributor

@mennayahia It it not so difficult. Read README.md on the top page https://github.com/OpenKinect/libfreenect2 and follow the instruction https://github.com/OpenKinect/libfreenect2#linux to install.
Please submit another issue if you have some problem.

@mennayahia
Copy link

@hanyazou
I already installed libfreenect2 from https://github.com/OpenKinect/libfreenect2 and Protect example is working fine, displaying depth maps.
I want to install/configure the openni2 for libfreenect2, to be able to track skeletons.
and you guys seem to be working on this, so how can I integrate your code with the libfreenect2 repository I cloned and installed a week ago.

Thanks for the help.

@hanyazou
Copy link
Contributor

@mennayahia Are you looking for a skeleton tracking library? This project libfreenect2 nor OpenNI2 does not contain any skeleton tracking function. You might be able to use the Nite binary for that purpose. Refer #243.
(Please note that the Nite is not a open source nor free software project and development was already discontinued.)

@mennayahia
Copy link

@hanyazou
I want to develop an application based on Kinect v2 that detects and tracks hands, then I want to extract features from the hand to use for hand gesture recognition.
I thought openNI2 with libfreenect2 would help, wouldn't it ?

@tofighi
Copy link

tofighi commented Feb 25, 2016

@mennayahia
OpenNI and Libfreenect2 don't provide hand detection or tracking. If you are looking for such features, you should use NiTE handtracker feature which is not opensource, however, it seems you can use it as a middleware on top of OpenNI and Libfreenect2.

@HenningJ
Copy link
Contributor

But keep in mind that you are not actually allowed to do that, since you are only allowed to use NiTE with Primesense devices.

@tofighi
Copy link

tofighi commented Feb 25, 2016

@HenningJ @mennayahia
That' true, Kinect V2 is not a Primesense device. I know an open source skeleton tracking implementation that may help:
https://github.com/joaquimrocha/Skeltrack

@mennayahia
Copy link

But does Skeltrack works with kinect v2 ?
it does not seem to be compatible with v2

@HenningJ
Copy link
Contributor

someone made it work with Kinect v2: joaquimrocha/Skeltrack#28

@mennayahia
Copy link

@hanyazou
Ok.
I have another question.
I clones the master branch of libfreenect2, installed it and Protonect is working fine. Does this mean I installed OpenNI also with libfreenect2 ? or should I download it and install it separately, and if so, how to integrate them together then ?

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants