D435i is not providing IMU data on Jetson Xavier NX #2213
Comments
|
Hi @haquebd There was a similar sounding case yesterday at #2212 where a Jetson user with a D455 could not access the IMU in RealSense ROS. They also found that the IMU data was not accessible in librealsense programs such as realsense-viewer and rs-motion. They were able to confirm the presence of the /camera/imu topic using rostopic list in their particular case though. The IMU worked normally for them on a Windows computer, indicating that it was a Jetson-specific issue. An important difference in your own case is that you are including align_depth:=true filters:=pointcloud in your roslaunch instruction. Align and pointcloud instructions are known to currently have problems when used in the ROS wrapper on Jetson boards specifically (the same setup works correctly on desktop / laptop PCs). An official Intel bug report has been filed about this Jetson issue, and #1967 contains further information about Jetson pointcloud problems. On the case that the bug report is linked to, the Jetson user found a workaround for generating a pointcloud, described at #2201 (comment) Could you try removing align_depth:=true filters:=pointcloud from your roslaunch instruction please in order to eliminate the possibility that your IMU access problem is being caused as a side-effect from these commands, please? Thanks! |
|
We are still getting Hardware errors and are not getting /camera/imu data. The IMU data is showing on the realsense-viewer. I have already downgraded the Jetpack version from 4.6 to 4.5 thinking it might resolve the issue. Now I have Jetpack 4.5 and Realsense SDK 2.50.0 and realsense-ros version 2.3.2 Resource temporarily unavailable, number: 11 nx-bot@expert-nx:~/catkin_ws$ roslaunch realsense2_camera rs_camera.launch initial_reset:=true depth_width:=640 depth_height:=480 depth_fps:=30 color_width:=640 color_height:=480 color_fps:=30 unite_imu_method:=copy enable_gyro:=true enable_accel:=true gyro_fps:=250 accel_fps:=200 started roslaunch server http://expert-nx:42787/ SUMMARYPARAMETERS
NODES auto-starting new master setting /run_id to 49c871a8-6ea4-11ec-97b6-d8c0a6b11fcf [ INFO] [1641441248.599185357]: Device with physical ID 2-3.2-6 was found. [ INFO] [1641441255.072790994]: Device with physical ID 2-3.2-7 was found. |
|
In the other recent case of an IMU problem on Jetson, I posted feedback that Intel provided me at #2212 (comment) As mentioned in that advice, support for JetPack 4.6.1 was added as part of librealsense 2.50.0. You are currently using JetPack 4.6-b199. Before that 4.6.1 support was added, the most recent officially supported JetPack version was 4.5 (32.5)
|
|
As I mentioned I already downgraded the Jetpack version from 4.6.b199 to 4.5 now. So will librealsense 2.50.0 be supportive with Jetpack 4.5 and relasense-roc version 2.3.2? |
|
JetPack 4.5 should work with librealsense 2.50.0. Please let me know if the /camera/imu topic is now able to publish data in wrapper 3.2.2. In #2213 (comment) you said that it was not working when you downgraded to JetPack 4.5. |
|
I will update the ros-wrapper to 3.2.2 and run it again. |
|
Okay, thanks very much @haquebd - please do let me know the outcome of your wrapper update. Good luck! |
|
I have managed to work it with Jetpack 4.5.1 and ros wrapper 2.3.2 @MartyG-RealSense thanks for your feedback. |
|
That's excellent news that you found a solution, @haquebd - thanks very much for the update! |
|
@MartyG-RealSense I have following problem, but slightly different setup. I have jetpack 4.6 on host jetson agx and running realsense node in docker ubuntu focal. librealsense in docker is compiled with USB flag on. Still IMU messages are not published. What I should do in this case? Applying the patch on the host machine does nothing(which makes sense). |
|
Hi @AlexKaravaev I note that your log has a Motion Module Failure error. This error relates to a problem with starting the IMU streams. Does the error still occur if you add the instruction initial_reset:=true to the roslaunch instruction to reset the camera at launch? |
|
@MartyG-RealSense I can not test that, because in docker after initial_reset it can not update again device list in /dev |
|
Are you able to set gyro to 200 and accel to 63 to see whether the motion module error still occurs? Does your launch set unite_imu_method to linear_interpolation If unite_imu_method is not defined then the 'imu' topic would not be published, and two separate gyro and accel topics would be published instead. |
|
Still the same. accel and gyro are not published, but imu just appears empty. Attaching full config and full rosout |
|
This is what I get with 400 and 250 fps set on gyro and accel. No Motion Module failure, but still no data arrived on imu topic |
|
oh yes and important thing is that I am using ros2, latest commit on ros2 branch |
|
And also I have tested on another pc with x86 arch in realsense-viewer imu is displayed correctly with the same camera |
|
The warning control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11, if it is generating continuously in the ROS terminal, would indicate a communications issue where frames are not being reliably received from the camera on your Jetson AGX + Docker setup. In fact, you said earlier that IMU messages are not being published. Noting that you have align_depth set to True in a log above, can you confirm that you still have no IMU messages if align_depth is False. I know that it is not directly related to the IMU messages, but having align_depth enabled has been known to cause issues on Jetson specifically. |
|
Unfortunately, no( Stil the same |
|
Also output of config Output |
|
@AlexKaravaev Which installation instructions did you use to install the librealsense SDK and the RealSense ROS wrapper on Jetson, please? Have you also tested without Docker to determine whether Docker may be a factor in the lack of IMU data? |
|
@MartyG-RealSense This is the command: The wrapper is built just with usual Though if rs-data-collect report no imu too, I guess it's issue with librealsense, not the wrapper. I haven't tried on host, but I guess it's possible that it will be working because it's installable with apt-get . In docker it is not, because distrubution is focal I guess? But I don't see problem why it should not work with a source build |
|
Because you are using a Jetson that has an Nvidia graphics GPU, DBUILD_WITH_CUDA can be set to true. The CUDA support in librealsense offloads processing work from the CPU to the GPU when performing color conversion, depth-color alignment and point cloud operations. As you are using align_depth = true, enabling CUDA support in the librealsense build could considerably reduce CPU percentage utilization. This may not address the missing IMU data but is worth considering from a processing efficiency viewpoint. |
|
@MartyG-RealSense It's true, thank you for the tip! I had some problems with cuda in docker, but I will do that off course in the future. But yes, unfortunetely problem with imu lies somewhere else |
|
In your opening message in this case, you describe using rostopic as /camera/imu. Are you able to echo the 'imu' topic to check if messages are being published? rostopic echo /camera/imu |
|
It wasn't showing anything, as well as ros2 hz |
|
@AlexKaravaev I researched your problem from the beginning in search of fresh insights. I recalled reading that Nvidia's Jetson JetPack was compatible with Ubuntu 16.04 and 18.04 but not designed for 20.04 (Focal), which you are using in your Docker system. Nvidia's official JetPack installation instructions seemed to confirm my recollection of JetPack support only being for 16.04 and 18.04. https://docs.nvidia.com/jetson/jetpack/install-jetpack/index.html#jetpack-debian-packages I did observe in my research though that some progress was being made in developing workarounds for 20.04 JetPack compatibility, such as the JetsonHacks guide in the link below that was published in August 2021. https://www.jetsonhacks.com/2021/08/25/native-boot-for-jetson-xaviers/
|
|
@MartyG-RealSense But I am using JetPack on 18.04. 18.04 is installed on my host machine, in my docker it's 20.04(focal),which should not cause a problem. Also given that imu was working in docker previously(though I last tested it very-very long time ago, so I can not tell exactly which update causes this error with imu not published is happened) |
|
In librealsense 2.50.0 Intel released a tutorial guide for Librealsense Docker. The guide states that Librealsense Docker does not currently support use of IMU devices. https://github.com/IntelRealSense/librealsense/tree/master/scripts/Docker Is your own Docker system using Librealsense Docker? |
|
I have the exact same issue in jetson xavier, the imu is working in realsense viewer, but nothing is publishing in the rostopic, not even the individual message from gyro and accel. I've checked my realsense2-camera version, librealsense and jetpack version are exactly the same as the one that op managed to get working. Any more idea on what do i need to do |
|
Hi @blue-ring-octopus What is the roslaunch instruction that you are using, please? The creator of this case used the following launch instruction:
If you are using the above launch instruction then the IMU topics would be published on the 'imu' topic instead of individual gyro and accel topics because of the 'unite_imu_method' parameter being used to combine the two separate IMU topics into one. |
|
i used both |
|
Does it publish any depth or color topic data if you use the most basic launch instruction, roslaunch realsense2_camera rs_camera.launch |
yes, it's working perfectly -update: the imu suddenly stopped working even in the realsense-viewer and rs-motion today. it is still working on my window laptop |
|
If the camera works normally on your Windows computer then we can conclude that the camera hardware, firmware driver and USB cable are likely fine and that it is a problem related to Linux (such as the kernel) or to the Linux version of the librealsense SDK on the particular computer that you are using. What method do you use to install librealsense on your Jetson Xavier, please? For example, if you were using Method 1 of the ROS wrapper instructions to install librealsense and the wrapper together at the same time from packages then that method would not be suitable for Jetson's Arm architecture and it would be better to install librealsense and the wrapper separately (librealsense first and then the wrapper secondly after that). |
i was using method 1. i purged it and switched to RSUSB method and installed realsense-camera from source this morning. It worked the first time I launched it, then the same problem is happening again after i relaunched everything |
|
If you are using RSUSB then it is likely not a kernel related problem, as an RSUSB build bypasses the kernel. Does the IMU behave correctly again if you do a complete shutdown and reboot of the Jetson when it stops working after the first launch? |
no, it worked exactly once after the first launch, and neither rebooting or resetting can get it to work again |
|
Are you able to reload the udev device handling rules by inputting the commands below into the Ubuntu terminal? udevadm control --reload-rules |
|
|
A purge will not clear that particular problem (multiple udev rules detected) but it has a simple solution. If librealsense is installed with both packages and source code on the same computer then you can end up with two different udev rules that conflict, etc/udev/rules.d/99-realsense-libusb.rules and /lib/udev/rules.d/60-librealsense2-udev-rules.rules The rule at the etc location is the one that should be deleted. The multiple udev rules warning and problems associated with it should then automatically disappear. |
just did, and ran the two line for reloading too, still not fixing the imu problem, |
|
Having eliminated a camera hardware / firmware problem or kernel problem as possibilities and tried most other solutions, the remaining unpleasant option would be a complete wipe and reinstall of the Jetson (including the Ubuntu OS) to remove any hidden problems that might be affecting the IMU. When everything else has been tried, a total wipe and reinstall does sometimes correct a problem, even if you never find out what the cause of the problem actually was. |
very unpleasant indeed, many things were setup by my teammates long time ago so it could be a lot of work to retrace every step to get it back to the current state. I will think about it and try to hunt for more ways to check what's wrong with it, thanks for your help |
|
Hi @blue-ring-octopus we have the exact same issue, we reinstall Jetson , but when we run ros examples we caouldn't get IMU datas, did you solve ur problem? If it is can you write down here one by one. |
My goal is to use the D435i in Rtabmap with camera depth and IMU. I also tried with unite_imu_method:=linear_interpolation But still the same error. I dont see any "rostopic as /camera/imu"
The camera is connected with the supplied USB cable directly connected with the USB port in Jetson Xavier NX.
realsense_ros version 2.3.2 or latest and librealsense is 2.50.x
The Jetpack and Ubuntu info as below:
ROS: Melodic
Ubuntu: 18.04
Jetson Package: nvidia-jetpack
Version: 4.6-b199
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 194
Depends: nvidia-cuda (= 4.6-b199), nvidia-opencv (= 4.6-b199), nvidia-cudnn8 (= 4.6-b199), nvidia-tensorrt (= 4.6-b199), nvidia-visionworks (= 4.6-b199), nvidia-container (= 4.6-b199), nvidia-vpi (= 4.6-b199), nvidia-l4t-jetson-multimedia-api (>> 32.6-0), nvidia-l4t-jetson-multimedia-api (<< 32.7-0)
`remote-eng@xavier-nx:~$ roslaunch realsense2_camera rs_camera.launch initial_reset:=true align_depth:=true filters:=pointcloud depth_width:=640 depth_height:=480 depth_fps:=30 color_width:=640 color_height:=480 color_fps:=30 unite_imu_method:=copy enable_gyro:=true enable_accel:=true gyro_fps:=250 accel_fps:=200
... logging to /home/remote-eng/.ros/log/9c1d2e46-6dda-11ec-b7ba-d8c0a6b11fcf/roslaunch-xavier-nx-23024.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.
started roslaunch server http://xavier-nx:42683/
SUMMARY
PARAMETERS
NODES
/camera/
realsense2_camera (nodelet/nodelet)
realsense2_camera_manager (nodelet/nodelet)
auto-starting new master
process[master]: started with pid [23034]
ROS_MASTER_URI=http://localhost:11311
setting /run_id to 9c1d2e46-6dda-11ec-b7ba-d8c0a6b11fcf
process[rosout-1]: started with pid [23047]
started core service [/rosout]
process[camera/realsense2_camera_manager-2]: started with pid [23064]
process[camera/realsense2_camera-3]: started with pid [23065]
[ INFO] [1641354626.842334499]: Initializing nodelet with 4 worker threads.
[ INFO] [1641354627.238120701]: RealSense ROS v2.3.2
[ INFO] [1641354627.238257533]: Built with LibRealSense v2.50.0
[ INFO] [1641354627.238346878]: Running with LibRealSense v2.50.0
[ INFO] [1641354627.353258880]:
04/01 22:50:27,438 WARNING [546803020160] (messenger-libusb.cpp:42) control_transfer returned error, index: 300, error: Success, number: 0
[ INFO] [1641354627.697529733]: Device with serial number 045322074531 was found.
[ INFO] [1641354627.697784838]: Device with physical ID 2-3.2-9 was found.
[ INFO] [1641354627.697912903]: Device with name Intel RealSense D435I was found.
[ INFO] [1641354627.700014065]: Device with port number 2-3.2 was found.
[ INFO] [1641354627.700142257]: Device USB type: 3.2
[ INFO] [1641354627.700248786]: Resetting device...
[ INFO] [1641354633.842230794]:
[ INFO] [1641354634.161278901]: Device with serial number 045322074531 was found.
[ INFO] [1641354634.162921885]: Device with physical ID 2-3.2-10 was found.
[ INFO] [1641354634.163335903]: Device with name Intel RealSense D435I was found.
[ INFO] [1641354634.165729098]: Device with port number 2-3.2 was found.
[ INFO] [1641354634.165855562]: Device USB type: 3.2
[ INFO] [1641354634.177713859]: getParameters...
[ INFO] [1641354634.674126839]: setupDevice...
[ INFO] [1641354634.674287960]: JSON file is not provided
[ INFO] [1641354634.674410104]: ROS Node Namespace: camera
[ INFO] [1641354634.674526489]: Device Name: Intel RealSense D435I
[ INFO] [1641354634.674609753]: Device Serial No: 045322074531
[ INFO] [1641354634.674671577]: Device physical port: 2-3.2-10
[ INFO] [1641354634.674897051]: Device FW version: 05.13.00.50
[ INFO] [1641354634.674982555]: Device Product ID: 0x0B3A
[ INFO] [1641354634.675044059]: Enable PointCloud: On
[ INFO] [1641354634.675150620]: Align Depth: On
[ INFO] [1641354634.675252060]: Sync Mode: On
[ INFO] [1641354634.675517021]: Device Sensors:
[ INFO] [1641354634.777893475]: Stereo Module was found.
[ INFO] [1641354634.861425904]: RGB Camera was found.
[ INFO] [1641354634.862906743]: Motion Module was found.
[ INFO] [1641354634.863500218]: (Confidence, 0) sensor isn't supported by current device! -- Skipping...
[ INFO] [1641354634.864987137]: Add Filter: pointcloud
[ INFO] [1641354634.869699671]: num_filters: 2
[ INFO] [1641354634.870251002]: Setting Dynamic reconfig parameters.
[ INFO] [1641354639.237687847]: Done Setting Dynamic reconfig parameters.
[ INFO] [1641354639.239303982]: depth stream is enabled - width: 640, height: 480, fps: 30, Format: Z16
[ INFO] [1641354639.241110807]: color stream is enabled - width: 640, height: 480, fps: 30, Format: RGB8
[ WARN] [1641354639.244200902]: No mathcing profile found for gyro with fps=250
[ WARN] [1641354639.244338662]: Using default profile instead.
[ INFO] [1641354639.244465191]: gyro stream is enabled - fps: 200
[ WARN] [1641354639.244642248]: No mathcing profile found for accel with fps=200
[ WARN] [1641354639.244744328]: Using default profile instead.
[ INFO] [1641354639.244831689]: accel stream is enabled - fps: 63
[ INFO] [1641354639.244914953]: setupPublishers...
[ INFO] [1641354639.258937868]: Expected frequency for depth = 30.00000
[ INFO] [1641354639.430975739]: Expected frequency for color = 30.00000
[ INFO] [1641354639.545222777]: Expected frequency for aligned_depth_to_color = 30.00000
[ INFO] [1641354639.653979612]: Start publisher IMU
[ INFO] [1641354639.659942809]: setupStreams...
04/01 22:50:39,837 WARNING [547035201920] (ds5-motion.cpp:473) IMU Calibration is not available, default intrinsic and extrinsic will be used.
[ WARN] [1641354640.217446699]:
[ WARN] [1641354640.320831510]: Hardware Notification:Motion Module failure,1.64135e+12,Error,Hardware Error
04/01 22:50:40,484 WARNING [546459083136] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[ INFO] [1641354640.535796144]: SELECTED BASE:Depth, 0
[ INFO] [1641354640.596498480]: RealSense Node Is Up!
04/01 22:50:40,608 WARNING [546459083136] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
04/01 22:50:40,658 WARNING [546459083136] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
04/01 22:50:40,709 WARNING [546459083136] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
04/01 22:50:40,760 WARNING [546459083136] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
04/01 22:50:41,031 WARNING [546459083136] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
04/01 22:50:41,081 WARNING [546459083136] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
04/01 22:50:42,708 ERROR [546056429952] (uvc-streamer.cpp:106) uvc streamer watchdog triggered on endpoint: 132
`
The text was updated successfully, but these errors were encountered: