Libfreenect2 not working on Jetson TX1 #581
Comments
How? Anything more specific? jetsonhacks was mostly using one of my branches which has been incorporated into the upstream libfreenect2 so there shouldn't be any functional difference. It's the a few small tweaks that made the difference. Frankly, several people have come to ask about problems with TX1. As I don't have a TX1, some of the investigation can only be done on the user side. And they did not follow up so the problems are never solved. |
|
I changed the wget line in jetsonhacks' install script to a new version of gstjpeg: http://developer.download.nvidia.com/embedded/L4T/r23_Release_v1.0/source/gstjpeg_src.tbz2 I also changed line 113 of CMakeLists.txt from "jpeg" to "nvjpeg" to fix a build error. Those are the only changes I made. However, I also ran the jetsonhacks version without CUDA or the Tegra acceleration in order to try to get it to work with the iai_kinect2 code and it still worked (the altered iai_kinect2 code crashes though). So I think something other than the optimizations (CUDA and Tegra) that you added are making it work. Are there any other commands I can run to help you help me debug? Or any issues you think it might be? |
I suppose you have done this? Also checked other issues here https://github.com/OpenKinect/libfreenect2/wiki/Troubleshooting#jetson-tk1-issues ? We definitely have one working version and one non-working version to compare with. Several things to look for
|
|
I maximized all CPU's and set the GPU clock speed to 10x what it was (76800 kHz -> 768000 kHz). This didn't fix anything. I have the correct udev rules in place and I've enabled usb3 with usb_port_owner_info=2 and disabled autosuspend with usbcore.autosuspend=-1. When I run the Kinect through a USB3 hub rather than a PCI Express USB 3.0 card I only get the following error: Using the libusb debs you linked to didn't fix anything either. However, I didn't have a depends/libusb* folder or file, I only have the install_libusb.sh script and the two deb files in the debs folder. There weren't any libusb errors or messages when run with I get the following errors in dmesg: Does that tell you anything? |
|
Everyone thinks
These are not errors. They just show inefficiency. They are irrelevant in determining if it works or not.
Not errors.
Not an error, but could indicate something indirect about your USB controller. It's not clear to me from your report which is working and which is not. Also, you really need to clarify your hardware configuration. Something unusual is going on with your
|
I had trouble testing the onboard USB 3 port because I wasn't able to run the program over ssh because of a GLFW error. Could it be because of a version mismatch between glfw on my workstation vs the TX1? I'm running the TX1 development board with a 4 port PCI Express USB 3.0 Card with 4 dedicated 5Gbps Channels as well as a USB 3.0 4 port hub plugged into the built-in TX1 usb port. |
|
You can run it over ssh using |
|
Previously there are several cases when people use external USB extension cards or USB hubs on Jetson and it didn't work. Once they use the built-in port it worked. So definitely check the built-in port. This hints at mostly USB problems. Yes, the upstream libfreenect2 should work even with the USB extension card. I don't recall any dramatic changes to the USB transfer code in libfreenect2 since the jetsonhacks version. Besides the test with the built-in port, can you do the test using the upstream version and the extension card port, and provide:
|
|
Running with built-in usb port (run over ssh -X with DISPLAY=:0.0)
|
|
I really need log of Even with Jetson TK1, TegraJPEG and CUDA processors provide 85Hz and 94Hz. Also, it seems your build didn't enable TegraJPEG support. A user #480 (comment) had the same issue but hasn't followed up. The problem is there should be |
|
Running upstream libfreenect with PCI express card (not CUDA or Tegra accelerated): I've attached my dmesg. |
|
Does the output of ? |
|
No but I wasn't sure how much more to do, by that point it was seeming like it was repeating. By the time I stopped it (after running for 5 - 10 seconds) this is what was approximately repeating: |
|
So I suppose you are still getting a stream of ? This points to data loss at kernel driver level (WARN Successful completion on short TX). It's probably some changes in libusb behaviors that expose bugs in the kernel driver for uPD720202. This can be verified by printing packet lengths here Actually here: https://github.com/OpenKinect/libfreenect2/blob/master/src/depth_packet_stream_parser.cpp#L72 Add LOG_DEBUG << "pkt len " << length;And record it. I still want to verify which libusb this test was run with. Can you provide the output of |
|
The output of I really think there is some difference in their libusb. |
|
In regards to the tegra headers, in the jetsonhacks version I changed the following line of the build file from to (changed jpeg to nvjpeg). For packet lengths in the jetsonhacks version (where all viewer windows work) there seem to be two variations: Libusb versions:
You were right, there is a difference: the jetsonhacks version uses one from it's depends directory. |
|
What happens if ? |
|
Also, what happens if jetsonhacks version uses the system libusb? (Remove the libusb in depends directory) |
Am I doing something wrong? My Kinect is connected as my lsusb shows. |
|
If I move the libusb directory out of the depends directory and try to build the jetsonhacks version I get a build error Here are the offending lines of the cmake I assume: |
|
You didn't do anything wrong. Maybe it's the code structure which won't allow my proposed test. I suspect the difference in libusb is this: jetsonhacks@e818be2 Can you |
|
If this commit indeed makes the difference, you can try to build libusb from source by patching it to change |
|
You're right! When I reverted that commit the jetsonhacks version also broke. So in the upstream version I ran install_libusb.sh in the depends directory, then edited Now the upstream libfreenect2 works on the TX1! However, now when I run the kinect2_bridge it outputs nanms/nanHz for the depth processing. This is better than before when it wasn't working at all, but I'd rather it worked all the way! |
|
I don't think you are supposed to need After you built libusb in |
|
Now there are two remaining issues:
|
|
You're right, I didn't need those two CMake additions. With the original CMakeLists.txt file here is my output: When I run the upstream libfreenect2 with libusb patched I don't get any errors, but no windows show up on my machine. I'm running I'll have to look into Tegra more tomorrow, I haven't disabled it, but CMake can't find the right headers. |
|
A few things that don't look right to me:
I don't know your setup. The viewer is supposed to show up on the monitor that is connected to the Jetson. If no monitor is connected to it, framerate information can also prove it works. I don't know if you still get nanHz or 0Hz, but There are some new commits these two days, remember to update. |
|
Sorry I misunderstood - the viewer does launch on the monitor connected to the TX1 when I run it over ssh -X with When running the Kinect through the built-in USB port the viewer appears but is only a black screen. I get lots of "skipping rgb packet!" messages and a color frame rate, but nothing for depth. For both of these tests I built using I was able to get the TegraJPEG decoder to build, but I had to replace the the current section in CMakeLists.txt relating to TegraJPEG with (adapted from the jetsonhacks CMakeLists.txt): I also had to download the tegra headers (adapted from jetsonhacks install script): I also disabled VA-API support to ensure the Tegra version would be run. However, after all this, when I run `./bin/Protonect' I get the following error: This error happens in src/packet_pipeline.cpp when the tegra processor is tested. The I'm going to continue to debug the Tegra and CUDA optimizations tomorrow. Do you notice anything wrong that you know how to fix? |
This will not work. And I'm unable to support the old CMake code. You can't simply mix the old code with new code and expect it to work. If you take a look, https://github.com/OpenKinect/libfreenect2/blob/master/cmake_modules/FindTegraJPEG.cmake#L38, CMake supposedly downloads the headers to Now the problem is somehow it fails to extract the headers. Some very trivial error happens here. It just needs someone with TX1 to take a look. |
|
OK, I know the problem now. |
|
Try this tree https://github.com/xlz/libfreenect2/tree/tegra-url |
|
@xlz, sorry for not replying to #480 I have been busy. I just tried your tree tegra-url and I confirm that now cmake downloads and extract the tarball. Here is the final output: Nevertheless, it seems that Protonect has still problems: |
|
After some reboots it seems that TegraJpeg started to work, but I still can't see any depth information from the output. @menlocaleb, how did you set your TX1 for max performance? My CPU speed is 1.9 GHz, the GPU clock speed is 998.4 MHz and the GPU memory clock is 1.6 GHz. Do these numbers make sense to you? |
|
@marketto89 Your setup is different from @menlocaleb @menlocaleb has been using a uPD720202 extension card with patched libusb. Right now both of you could not get libfreenect2 to work with the built-in USB 3 port. Regarding the built-in USB 3 port, there are several things to verify: (See the logs here #480 (comment))
|
Yes I see many of these errors!
Here is the tail of my dmesg:
I will inform you if this fix works! |
I followed the instructions and verified that Protonect was linked to the patched libusb but it is still not working: |
|
OK this confirms it's the same USB kernel driver issue as last time. There is no solution to that yet. I will try to contact Nvidia developers about this bug. In the meantime, I think the best workaround is to acquire a uPD720202 extension card as menlocaleb did. |
|
Given the answers, is there anything we can do to test the built-in USB 3.0 port? |
|
@marketto89 I think the easiest thing to do right now is get a uPD720202 extension card. |
|
I have obtained a TX1 and I have reproduced all the errors you've had. I'm working on it, but this one is really hard. |
|
@xlz sorry for not responding on this thread recently. I was just looking into this problem again and went to try your new branch https://github.com/xlz/libfreenect2/tree/tegra-url but it looks like it's deleted. Was the problem just changing a url for the jpeg headers to be compatible with the X1? Is there anything I should try to help, or should I wait for you to fix on your X1? |
|
@menlocaleb I thought your had it working #581 (comment), what is the new problem? tegra-url is merged into upstream so it's deleted. I have put the recommendation of "uPD720202 extension card with libusb 49152" into docs. I can't fix this issue because it's a problem with Nvidia's tegra kernel driver. |
|
Sorry for not being specific: I can run the upstream libfreenect, I was working on fixing the tegra acceleration, which it sounds like should work now that you've checked in the fix. Last time I tried the iai_kinect2 code was crashing, but maybe after the tegra acceleration fix and rebuilding it will work. If it doesn't I'll let you know. |
|
Hi, Here's the message from `Version: 0.2.0 [ 0.000005] [00000cac] libusb: debug [libusb_init] created default context |
|
Hello, I have read through a majority of this and other treads related to the Tegra TX1 and the Kinect v2 on the native built in USB3.0 port. has there been any progress? I have seen the progress at https://devtalk.nvidia.com/default/topic/919354/jetson-tx1/usb-3-transfer-failures/post/4836312/#4836312 I can reproduce all the known problems resulting in no viewer output. Attached are reproduced logs for consistencies sake. other_logs.txt Any progress apart from buying the suggested PCIe card? |
|
Nvidia's kay was looking at the issue but there is no update. So, no progress. |
|
NVIDIA has posted a USB firmware patch that appears to resolve the issue: The data appears to be transferred correctly, though the RGB color stream appears to have tint issues, suggesting a possible color decoding problem. |
|
Cool news. @jetsonhacks how does the tint look? Can you post a screen shot? |
|
I'm a monster, but one of a different color. The shirt is black as reference, painting is brown/gold/orange.
It's been several months, but I seem to recall that the Jetson TK1 has the same issue with coloring. |
|
Oh, this is because Jetson hardware JPEG decoder outputs image in RGBA format but the viewer is expecting BGRA. You can check this by checking frame->format. |
|
Yes, converting to BGRA solves the issue. Thank you very much for your help. |
|
@xlz @menlocaleb |
|
I believe this has been resolved by newer TX1 firmware. |
I'm trying to run libfreenect2 on an Nvidia Jetson TX1 Developer Kit and can't seem to get it working. Here's the output I'm getting:
I've ensured that USB 3 is enabled in /boot/extlinux/extlinux.conf and turned off usb autosuspend using
for i in /sys/bus/usb/devices/*/power/autosuspend; do echo -1 >$i; doneI've checked dmesg for all of the issues mentioned in the Troubleshooting wiki and couldn't find anything.
I did succeed in getting the jetsonhacks version of libfreenect2 to work after a few small tweaks so I know it's possible to run on the TX1, I just can't figure out how to get the main version of libfreenect2 to work. I need to do this because I want to use the iai_kinect2 code to integrate with ROS, and that code needs the main version of libfreenect2. Any help would be greatly appreciated!
The text was updated successfully, but these errors were encountered: