Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem with 2 D415 cameras | RS2_USB_STATUS_BUSY | Not detected after realsense-ros in other tools #1187

Closed
mjsobrep opened this issue May 6, 2020 · 55 comments

Comments

@mjsobrep
Copy link

mjsobrep commented May 6, 2020

I just made the mistake of updating software (dumb) and am now having trouble with a setup running two D415 cameras. My launch file had been working well for sometime. Rather than just downgrading, I would like to understand the source of the problem and fix it.

The system seems to think that the cameras are busy on startup, I do not know why. If I am running in rosmon, I kill the nodes, and then restart them, then it seems to work, although CPU usage goes crazy. Simple relaunching does not solve the problem.

Weirdly, when I turn on the computer, if I launch realsense viewer, there are no problems. If I run the ros launch file above and then open realsense viewer, the cameras do not show up.

If downgrading software is the way to go. How do I downgrade librealsense, the ros stack, and the firmware all at once? Or do I only need to downgrade one? Where should I be downgrading to?

console output:

nuc-admin@flo-nuc:~$ roslaunch flo_telepresence realsense.launch 
... logging to /home/nuc-admin/.ros/log/a622f598-8fb9-11ea-98e5-e470b83e61ec/roslaunch-flo-nuc-15444.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
WARNING: disk usage in log directory [/home/nuc-admin/.ros/log] is over 1GB.
It's recommended that you use the 'rosclean' command.

started roslaunch server http://10.153.99.5:39277/

SUMMARY
========

PARAMETERS
 * /lower_realsense/realsense2_camera/accel_fps: 250
 * /lower_realsense/realsense2_camera/accel_frame_id: lower_realsense_a...
 * /lower_realsense/realsense2_camera/accel_optical_frame_id: lower_realsense_a...
 * /lower_realsense/realsense2_camera/align_depth: False
 * /lower_realsense/realsense2_camera/aligned_depth_to_color_frame_id: lower_realsense_a...
 * /lower_realsense/realsense2_camera/aligned_depth_to_fisheye1_frame_id: lower_realsense_a...
 * /lower_realsense/realsense2_camera/aligned_depth_to_fisheye2_frame_id: lower_realsense_a...
 * /lower_realsense/realsense2_camera/aligned_depth_to_fisheye_frame_id: lower_realsense_a...
 * /lower_realsense/realsense2_camera/aligned_depth_to_infra1_frame_id: lower_realsense_a...
 * /lower_realsense/realsense2_camera/aligned_depth_to_infra2_frame_id: lower_realsense_a...
 * /lower_realsense/realsense2_camera/allow_no_texture_points: False
 * /lower_realsense/realsense2_camera/base_frame_id: lower_realsense_link
 * /lower_realsense/realsense2_camera/calib_odom_file: 
 * /lower_realsense/realsense2_camera/clip_distance: -2.0
 * /lower_realsense/realsense2_camera/color_fps: 30
 * /lower_realsense/realsense2_camera/color_frame_id: lower_realsense_c...
 * /lower_realsense/realsense2_camera/color_height: 480
 * /lower_realsense/realsense2_camera/color_optical_frame_id: lower_realsense_c...
 * /lower_realsense/realsense2_camera/color_width: 848
 * /lower_realsense/realsense2_camera/depth_fps: 30
 * /lower_realsense/realsense2_camera/depth_frame_id: lower_realsense_d...
 * /lower_realsense/realsense2_camera/depth_height: 720
 * /lower_realsense/realsense2_camera/depth_optical_frame_id: lower_realsense_d...
 * /lower_realsense/realsense2_camera/depth_width: 1280
 * /lower_realsense/realsense2_camera/enable_accel: False
 * /lower_realsense/realsense2_camera/enable_color: True
 * /lower_realsense/realsense2_camera/enable_depth: False
 * /lower_realsense/realsense2_camera/enable_fisheye1: True
 * /lower_realsense/realsense2_camera/enable_fisheye2: True
 * /lower_realsense/realsense2_camera/enable_fisheye: True
 * /lower_realsense/realsense2_camera/enable_gyro: False
 * /lower_realsense/realsense2_camera/enable_infra1: False
 * /lower_realsense/realsense2_camera/enable_infra2: False
 * /lower_realsense/realsense2_camera/enable_pointcloud: False
 * /lower_realsense/realsense2_camera/enable_sync: False
 * /lower_realsense/realsense2_camera/filters: 
 * /lower_realsense/realsense2_camera/fisheye1_frame_id: lower_realsense_f...
 * /lower_realsense/realsense2_camera/fisheye1_optical_frame_id: lower_realsense_f...
 * /lower_realsense/realsense2_camera/fisheye2_frame_id: lower_realsense_f...
 * /lower_realsense/realsense2_camera/fisheye2_optical_frame_id: lower_realsense_f...
 * /lower_realsense/realsense2_camera/fisheye_fps: 30
 * /lower_realsense/realsense2_camera/fisheye_frame_id: lower_realsense_f...
 * /lower_realsense/realsense2_camera/fisheye_height: 480
 * /lower_realsense/realsense2_camera/fisheye_optical_frame_id: lower_realsense_f...
 * /lower_realsense/realsense2_camera/fisheye_width: 640
 * /lower_realsense/realsense2_camera/gyro_fps: 400
 * /lower_realsense/realsense2_camera/gyro_frame_id: lower_realsense_g...
 * /lower_realsense/realsense2_camera/gyro_optical_frame_id: lower_realsense_g...
 * /lower_realsense/realsense2_camera/imu_optical_frame_id: lower_realsense_i...
 * /lower_realsense/realsense2_camera/infra1_frame_id: lower_realsense_i...
 * /lower_realsense/realsense2_camera/infra1_optical_frame_id: lower_realsense_i...
 * /lower_realsense/realsense2_camera/infra2_frame_id: lower_realsense_i...
 * /lower_realsense/realsense2_camera/infra2_optical_frame_id: lower_realsense_i...
 * /lower_realsense/realsense2_camera/infra_fps: 30
 * /lower_realsense/realsense2_camera/infra_height: 720
 * /lower_realsense/realsense2_camera/infra_width: 1280
 * /lower_realsense/realsense2_camera/initial_reset: False
 * /lower_realsense/realsense2_camera/json_file_path: 
 * /lower_realsense/realsense2_camera/linear_accel_cov: 0.01
 * /lower_realsense/realsense2_camera/odom_frame_id: lower_realsense_o...
 * /lower_realsense/realsense2_camera/pointcloud_texture_index: 0
 * /lower_realsense/realsense2_camera/pointcloud_texture_stream: RS2_STREAM_COLOR
 * /lower_realsense/realsense2_camera/pose_frame_id: lower_realsense_p...
 * /lower_realsense/realsense2_camera/pose_optical_frame_id: lower_realsense_p...
 * /lower_realsense/realsense2_camera/publish_odom_tf: True
 * /lower_realsense/realsense2_camera/rosbag_filename: 
 * /lower_realsense/realsense2_camera/serial_no: 904412060717
 * /lower_realsense/realsense2_camera/topic_odom_in: odom_in
 * /lower_realsense/realsense2_camera/unite_imu_method: 
 * /rosdistro: kinetic
 * /rosversion: 1.12.14
 * /upper_realsense/realsense2_camera/accel_fps: 250
 * /upper_realsense/realsense2_camera/accel_frame_id: upper_realsense_a...
 * /upper_realsense/realsense2_camera/accel_optical_frame_id: upper_realsense_a...
 * /upper_realsense/realsense2_camera/align_depth: False
 * /upper_realsense/realsense2_camera/aligned_depth_to_color_frame_id: upper_realsense_a...
 * /upper_realsense/realsense2_camera/aligned_depth_to_fisheye1_frame_id: upper_realsense_a...
 * /upper_realsense/realsense2_camera/aligned_depth_to_fisheye2_frame_id: upper_realsense_a...
 * /upper_realsense/realsense2_camera/aligned_depth_to_fisheye_frame_id: upper_realsense_a...
 * /upper_realsense/realsense2_camera/aligned_depth_to_infra1_frame_id: upper_realsense_a...
 * /upper_realsense/realsense2_camera/aligned_depth_to_infra2_frame_id: upper_realsense_a...
 * /upper_realsense/realsense2_camera/allow_no_texture_points: False
 * /upper_realsense/realsense2_camera/base_frame_id: upper_realsense_link
 * /upper_realsense/realsense2_camera/calib_odom_file: 
 * /upper_realsense/realsense2_camera/clip_distance: -2.0
 * /upper_realsense/realsense2_camera/color_fps: 30
 * /upper_realsense/realsense2_camera/color_frame_id: upper_realsense_c...
 * /upper_realsense/realsense2_camera/color_height: 480
 * /upper_realsense/realsense2_camera/color_optical_frame_id: upper_realsense_c...
 * /upper_realsense/realsense2_camera/color_width: 848
 * /upper_realsense/realsense2_camera/depth_fps: 30
 * /upper_realsense/realsense2_camera/depth_frame_id: upper_realsense_d...
 * /upper_realsense/realsense2_camera/depth_height: 720
 * /upper_realsense/realsense2_camera/depth_optical_frame_id: upper_realsense_d...
 * /upper_realsense/realsense2_camera/depth_width: 1280
 * /upper_realsense/realsense2_camera/enable_accel: False
 * /upper_realsense/realsense2_camera/enable_color: True
 * /upper_realsense/realsense2_camera/enable_depth: False
 * /upper_realsense/realsense2_camera/enable_fisheye1: True
 * /upper_realsense/realsense2_camera/enable_fisheye2: True
 * /upper_realsense/realsense2_camera/enable_fisheye: True
 * /upper_realsense/realsense2_camera/enable_gyro: False
 * /upper_realsense/realsense2_camera/enable_infra1: False
 * /upper_realsense/realsense2_camera/enable_infra2: False
 * /upper_realsense/realsense2_camera/enable_pointcloud: False
 * /upper_realsense/realsense2_camera/enable_sync: False
 * /upper_realsense/realsense2_camera/filters: 
 * /upper_realsense/realsense2_camera/fisheye1_frame_id: upper_realsense_f...
 * /upper_realsense/realsense2_camera/fisheye1_optical_frame_id: upper_realsense_f...
 * /upper_realsense/realsense2_camera/fisheye2_frame_id: upper_realsense_f...
 * /upper_realsense/realsense2_camera/fisheye2_optical_frame_id: upper_realsense_f...
 * /upper_realsense/realsense2_camera/fisheye_fps: 30
 * /upper_realsense/realsense2_camera/fisheye_frame_id: upper_realsense_f...
 * /upper_realsense/realsense2_camera/fisheye_height: 480
 * /upper_realsense/realsense2_camera/fisheye_optical_frame_id: upper_realsense_f...
 * /upper_realsense/realsense2_camera/fisheye_width: 640
 * /upper_realsense/realsense2_camera/gyro_fps: 400
 * /upper_realsense/realsense2_camera/gyro_frame_id: upper_realsense_g...
 * /upper_realsense/realsense2_camera/gyro_optical_frame_id: upper_realsense_g...
 * /upper_realsense/realsense2_camera/imu_optical_frame_id: upper_realsense_i...
 * /upper_realsense/realsense2_camera/infra1_frame_id: upper_realsense_i...
 * /upper_realsense/realsense2_camera/infra1_optical_frame_id: upper_realsense_i...
 * /upper_realsense/realsense2_camera/infra2_frame_id: upper_realsense_i...
 * /upper_realsense/realsense2_camera/infra2_optical_frame_id: upper_realsense_i...
 * /upper_realsense/realsense2_camera/infra_fps: 30
 * /upper_realsense/realsense2_camera/infra_height: 480
 * /upper_realsense/realsense2_camera/infra_width: 640
 * /upper_realsense/realsense2_camera/initial_reset: False
 * /upper_realsense/realsense2_camera/json_file_path: 
 * /upper_realsense/realsense2_camera/linear_accel_cov: 0.01
 * /upper_realsense/realsense2_camera/odom_frame_id: upper_realsense_o...
 * /upper_realsense/realsense2_camera/pointcloud_texture_index: 0
 * /upper_realsense/realsense2_camera/pointcloud_texture_stream: RS2_STREAM_COLOR
 * /upper_realsense/realsense2_camera/pose_frame_id: upper_realsense_p...
 * /upper_realsense/realsense2_camera/pose_optical_frame_id: upper_realsense_p...
 * /upper_realsense/realsense2_camera/publish_odom_tf: True
 * /upper_realsense/realsense2_camera/rosbag_filename: 
 * /upper_realsense/realsense2_camera/serial_no: 912322061173
 * /upper_realsense/realsense2_camera/topic_odom_in: odom_in
 * /upper_realsense/realsense2_camera/unite_imu_method: 

NODES
  /lower_realsense/
    realsense2_camera (nodelet/nodelet)
    realsense2_camera_manager (nodelet/nodelet)
  /upper_realsense/
    realsense2_camera (nodelet/nodelet)
    realsense2_camera_manager (nodelet/nodelet)

ROS_MASTER_URI=http://localhost:11311

process[lower_realsense/realsense2_camera_manager-1]: started with pid [15466]
process[lower_realsense/realsense2_camera-2]: started with pid [15467]
process[upper_realsense/realsense2_camera_manager-3]: started with pid [15468]
process[upper_realsense/realsense2_camera-4]: started with pid [15469]
[ INFO] [1588788685.468106682]: Initializing nodelet with 4 worker threads.
[ INFO] [1588788685.482146770]: Initializing nodelet with 4 worker threads.
[ INFO] [1588788685.520152048]: RealSense ROS v2.2.8
[ INFO] [1588788685.520177598]: Running with LibRealSense v2.33.1
[ INFO] [1588788685.533659376]: RealSense ROS v2.2.8
[ INFO] [1588788685.533695519]: Running with LibRealSense v2.33.1
 06/05 14:11:25,661 ERROR [139714514368256] (handle-libusb.h:95) failed to claim usb interface: 0, error: RS2_USB_STATUS_BUSY
 06/05 14:11:25,662 WARNING [139756541302528] (uvc-device.cpp:664) interrupt event received: 1, 3, 0, 7, 0, 13, 
 06/05 14:11:25,666 WARNING [139756541302528] (uvc-device.cpp:664) interrupt event received: 1, 3, 0, 7, 0, 20, 
 06/05 14:11:25,682 WARNING [139714547939072] (rs.cpp:283) null pointer passed for argument "device"
terminate called after throwing an instance of 'rs2::error'
  what():  failed to set power state
[lower_realsense/realsense2_camera_manager-1] process has died [pid 15466, exit code -6, cmd /opt/ros/kinetic/lib/nodelet/nodelet manager __name:=realsense2_camera_manager __log:=/home/nuc-admin/.ros/log/a622f598-8fb9-11ea-98e5-e470b83e61ec/lower_realsense-realsense2_camera_manager-1.log].
log file: /home/nuc-admin/.ros/log/a622f598-8fb9-11ea-98e5-e470b83e61ec/lower_realsense-realsense2_camera_manager-1*.log
[ INFO] [1588788685.919835448]: getParameters...
[ INFO] [1588788685.986562489]: setupDevice...
[ INFO] [1588788685.986592806]: JSON file is not provided
[ INFO] [1588788685.986611824]: ROS Node Namespace: upper_realsense
[ INFO] [1588788685.986633518]: Device Name: Intel RealSense D415
[ INFO] [1588788685.986645240]: Device Serial No: 912322061173
[ INFO] [1588788685.986656444]: Device FW version: 05.12.03.00
[ INFO] [1588788685.986667604]: Device Product ID: 0x0AD3
[ INFO] [1588788685.986680093]: Enable PointCloud: Off
[ INFO] [1588788685.986692181]: Align Depth: Off
[ INFO] [1588788685.986705761]: Sync Mode: Off
[ INFO] [1588788685.986744254]: Device Sensors: 
[ INFO] [1588788685.986779882]: Stereo Module was found.
[ INFO] [1588788685.986798221]: RGB Camera was found.
[ INFO] [1588788685.986825867]: (Fisheye, 0) sensor isn't supported by current device! -- Skipping...
[ INFO] [1588788685.986837817]: (Fisheye, 1) sensor isn't supported by current device! -- Skipping...
[ INFO] [1588788685.986851616]: (Fisheye, 2) sensor isn't supported by current device! -- Skipping...
[ INFO] [1588788685.986863020]: (Pose, 0) sensor isn't supported by current device! -- Skipping...
[ INFO] [1588788685.986885567]: num_filters: 0
[ INFO] [1588788685.986897894]: Setting Dynamic reconfig parameters.
[ INFO] [1588788691.143280503]: Done Setting Dynamic reconfig parameters.
[ INFO] [1588788691.171650707]: color stream is enabled - width: 848, height: 480, fps: 30
[ INFO] [1588788691.171804990]: setupPublishers...
[ INFO] [1588788691.175239860]: Expected frequency for color = 30.00000
[ INFO] [1588788691.209106475]: setupStreams...
[ INFO] [1588788691.260165542]: insert Color to RGB Camera
[ INFO] [1588788691.326572559]: SELECTED BASE:Depth, 0
[ INFO] [1588788691.368299520]: RealSense Node Is Up!
 06/05 14:11:31,800 WARNING [139756516124416] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 06/05 14:11:31,861 WARNING [139756516124416] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
[ WARN] [1588788692.198603124]: Hardware Notification:RT IC2 Config error,1.58879e+12,Error,Hardware Error
[ERROR] [1588788692.198635568]: Hardware Reset is needed. use option: initial_reset:=true
 06/05 14:11:32,725 WARNING [139756516124416] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 06/05 14:11:32,789 WARNING [139756516124416] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 06/05 14:11:33,154 WARNING [139756516124416] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 06/05 14:11:33,221 WARNING [139756516124416] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 06/05 14:11:33,224 WARNING [139756516124416] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 06/05 14:11:33,285 WARNING [139756516124416] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 06/05 14:11:33,289 WARNING [139756516124416] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 06/05 14:11:33,313 ERROR [139756021217024] (uvc-streamer.cpp:106) uvc streamer watchdog triggered on endpoint: 132
 06/05 14:11:33,314 WARNING [139756516124416] (messenger-libusb.cpp:92) usb_request_queue returned error, endpoint: 135 error: No such device, number: 19
 06/05 14:11:33,318 WARNING [139756021217024] (messenger-libusb.cpp:30) reset_endpoint returned error, index: 132, error: No such device, number: 19
 06/05 14:11:33,318 WARNING [139756021217024] (messenger-libusb.cpp:92) usb_request_queue returned error, endpoint: 132 error: No such device, number: 19
 06/05 14:11:33,318 ERROR [139756021217024] (uvc-streamer.cpp:138) failed to submit UVC request, error: -13
 06/05 14:11:33,318 WARNING [139756021217024] (messenger-libusb.cpp:92) usb_request_queue returned error, endpoint: 132 error: No such device, number: 19
 06/05 14:11:33,318 ERROR [139756021217024] (uvc-streamer.cpp:138) failed to submit UVC request, error: -13
 06/05 14:11:33,345 WARNING [139756516124416] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No such device, number: 19
 06/05 14:11:33,349 WARNING [139756516124416] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No such device, number: 19
 06/05 14:11:33,356 ERROR [139756532909824] (global_timestamp_reader.cpp:188) Error during time_diff_keeper polling: usb device disconnected
 06/05 14:11:33,359 WARNING [139756516124416] (messenger-libusb.cpp:105) usb_request_cancel returned error, endpoint: 135 error: No such device, number: 19
 06/05 14:11:33,369 ERROR [139756415477504] (error-handling.cpp:68) Error during polling error handler: usb device disconnected
 06/05 14:11:33,456 ERROR [139756516124416] (handle-libusb.h:51) failed to open usb interface: 0, error: RS2_USB_STATUS_NO_DEVICE
 06/05 14:11:33,466 ERROR [139756532909824] (global_timestamp_reader.cpp:188) Error during time_diff_keeper polling: failed to set power state
[lower_realsense/realsense2_camera-2] process has finished cleanly
log file: /home/nuc-admin/.ros/log/a622f598-8fb9-11ea-98e5-e470b83e61ec/lower_realsense-realsense2_camera-2*.log
[ERROR] [1588788695.919472157]: The device has been disconnected!
 06/05 14:11:39,582 WARNING [139756532909824] (types.cpp:49) set_xu(ctrl=1) failed! Last Error: Success
 06/05 14:11:39,582 ERROR [139756532909824] (global_timestamp_reader.cpp:188) Error during time_diff_keeper polling: set_xu(ctrl=1) failed! Last Error: Success
 06/05 14:11:39,702 WARNING [139756532909824] (types.cpp:49) hwmon command 0x1 failed. Error type: Success (0).
 06/05 14:11:39,702 ERROR [139756532909824] (global_timestamp_reader.cpp:188) Error during time_diff_keeper polling: hwmon command 0x1 failed. Error type: Success (0).
 06/05 14:11:40,385 WARNING [139756415477504] (types.cpp:49) get_xu(id=7) failed! Last Error: Success
 06/05 14:11:40,385 ERROR [139756415477504] (error-handling.cpp:68) Error during polling error handler: get_xu(id=7) failed! Last Error: Success
 06/05 14:11:45,817 WARNING [139756532909824] (types.cpp:49) set_xu(ctrl=1) failed! Last Error: Success
 06/05 14:11:45,817 ERROR [139756532909824] (global_timestamp_reader.cpp:188) Error during time_diff_keeper polling: set_xu(ctrl=1) failed! Last Error: Success
 06/05 14:11:45,831 WARNING [139756021217024] (messenger-libusb.cpp:105) usb_request_cancel returned error, endpoint: 132 error: No such device, number: 19
 06/05 14:11:45,831 WARNING [139756021217024] (messenger-libusb.cpp:105) usb_request_cancel returned error, endpoint: 132 error: No such device, number: 19
 06/05 14:11:45,831 WARNING [139756021217024] (messenger-libusb.cpp:30) reset_endpoint returned error, index: 132, error: No such device, number: 19
[ INFO] [1588788705.864227772]: Checking new devices...
 06/05 14:11:46,014 WARNING [139756390299392] (uvc-device.cpp:664) interrupt event received: 1, 3, 0, 7, 0, 0, 
 06/05 14:11:46,018 WARNING [139756390299392] (uvc-device.cpp:664) interrupt event received: 1, 3, 0, 7, 0, 0, 
[ INFO] [1588788706.273219629]: getParameters...
[ INFO] [1588788706.342231199]: setupDevice...
[ INFO] [1588788706.342261910]: JSON file is not provided
[ INFO] [1588788706.342277250]: ROS Node Namespace: upper_realsense
[ INFO] [1588788706.342293755]: Device Name: Intel RealSense D415
[ INFO] [1588788706.342307046]: Device Serial No: 912322061173
[ INFO] [1588788706.342320897]: Device FW version: 05.12.03.00
[ INFO] [1588788706.342333690]: Device Product ID: 0x0AD3
[ INFO] [1588788706.342346160]: Enable PointCloud: Off
[ INFO] [1588788706.342358541]: Align Depth: Off
[ INFO] [1588788706.342370751]: Sync Mode: Off
[ INFO] [1588788706.342388953]: Device Sensors: 
[ INFO] [1588788706.342405723]: Stereo Module was found.
[ INFO] [1588788706.342444569]: RGB Camera was found.
[ INFO] [1588788706.342463582]: (Fisheye, 0) sensor isn't supported by current device! -- Skipping...
[ INFO] [1588788706.342477531]: (Fisheye, 1) sensor isn't supported by current device! -- Skipping...
[ INFO] [1588788706.342490622]: (Fisheye, 2) sensor isn't supported by current device! -- Skipping...
[ INFO] [1588788706.342503650]: (Pose, 0) sensor isn't supported by current device! -- Skipping...
[ INFO] [1588788706.342519065]: num_filters: 0
[ INFO] [1588788706.342531574]: Setting Dynamic reconfig parameters.
[ INFO] [1588788711.496683351]: Done Setting Dynamic reconfig parameters.
[ INFO] [1588788711.539592516]: color stream is enabled - width: 848, height: 480, fps: 30
[ INFO] [1588788711.540301334]: setupPublishers...
[ INFO] [1588788711.543133283]: Expected frequency for color = 30.00000
[ INFO] [1588788711.574979698]: setupStreams...
[ INFO] [1588788711.625969888]: insert Color to RGB Camera
[ INFO] [1588788711.692674013]: SELECTED BASE:Depth, 0
[ INFO] [1588788711.738266856]: RealSense Node Is Up!
 06/05 14:11:52,166 WARNING [139756524517120] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 06/05 14:11:52,227 WARNING [139756524517120] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
^C[upper_realsense/realsense2_camera-4] killing on exit
[upper_realsense/realsense2_camera_manager-3] killing on exit
 06/05 14:33:16,176 WARNING [139757266376576] (types.cpp:49) stop_streaming() failed. UVC device is not streaming!
terminate called after throwing an instance of 'rs2::wrong_api_call_sequence_error'
  what():  stop_streaming() failed. UVC device is not streaming!
shutting down processing monitor...
... shutting down processing monitor complete
done
nuc-admin@flo-nuc:~$ alsamixer^C
@mjsobrep mjsobrep changed the title Problem with 2 D415 cameras Problem with 2 D415 cameras | RS2_USB_STATUS_BUSY | Not detected after realsense-ros in other tools May 6, 2020
@mjsobrep
Copy link
Author

mjsobrep commented May 6, 2020

I upgraded the version of realsense ros that I am using and downgraded the version of librealsense. Combined with unplugging and replugging in cables (gotta try everything), it now seems to be working as expected again.

I would love to see some guidance from intel on how to deploy their code safely. Running an apt update/upgrade should never break things. But with realsense, it happens all the time. On whole the platform seems super unstable. Is that a design problem, a user problem, both?

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented May 6, 2020

It is recommended that the RealSense ROS wrapper is matched with a particular version of the Librealsense SDK where possible. On the 'Releases' page for the wrapper, each version lists a recommended Librealsense SDK version to use it with.

https://github.com/IntelRealSense/realsense-ros/releases

The log in your first message says that you were using ROS wrapper 2.2.8 with Librealsense SDK 2.33.1. According to the Releases page, the appropriate match for wrapper version 2.2.8 would be the older Librealsense SDK version 2.25.0.

image

It is sometimes possible to 'mix and match' non-recommended combinations of wrapper and SDK, but in general the best way to make a good match between wrapper and SDK is to follow the recommended SDK for a particular wrapper version.

In regard to stability, the RealSense SDK aims to run on anything and is extremely adaptable in regard to the hardware and software that it can be used with. There are sometimes hardware and / or software configurations that require a bit more work to achieve a stable RealSense setup on though. These cases are usually mostly resolved over time as the RealSense community, the RealSense team or both develop guides and code solutions to overcome problems.

It is also worth bearing in mind that the ROS wrapper and Librealsense are not developed in sync, so there may be a delay between the release of a new Librealsense version and a ROS wrapper version that officially supports it (though the ROS wrapper for the previous Librealsense version may still work with the newer Librealsense release).

@mjsobrep
Copy link
Author

mjsobrep commented May 6, 2020

@MartyG-RealSense I agree, that mismatch was likely the source of the problem. Updating a system by looking at a webpage isn't a great solution. It is pretty easy to programatically update the realsense-ros repo and do the rebuilds, etc. Is there a way to programmatically install the recommended version of librealsense? Ideally there would just be an install-realsense-ros script and an update-realsense-ros script.

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented May 6, 2020

The set of installation instructions in the link below may provide useful insights.

https://github.com/IntelRealSense/realsense-ros/blob/development/.travis.yml

The instructions in this file are summarised by a two-step description in the link below, in which Librealsense is installed first and the ROS wrapper second:

https://github.com/IntelRealSense/realsense-ros#method-2-the-realsense-distribution

This could create a "chicken and the egg" dilemma ... if the Librealsense SDK is installed first and the ROS wrapper second, then the ROS wrapper version that was auto-selected to be installed would have to be matched to the Librealsense version installed in the previous step in order to get a recommended compatibility match. This may be why manually checking the Releases page is the most practical solution in most cases, if not the ideal one.

@mjsobrep
Copy link
Author

mjsobrep commented May 7, 2020

Yeah, I don't see how those scripts prevent one from getting a version of librealsense that is ahead of the ROS wrapper. Regardless my issue is fixed, so I am going to close this issue.

A feature request before I go though: If improving the installation experience is out of scope, then adding a check to the ros wrapper's startup that ensures that the librealsense version being used is the ideal one should be added. That check should not break anything but should make a very loud (but silenceable through parameters) noise to let the user know.

@MartyG-RealSense
Copy link
Collaborator

Could you post your feature request as a new question on its own please? Then I can label it as such and it can be left open to be tracked by the ROS wrapper developers. Thanks!

@mjsobrep
Copy link
Author

mjsobrep commented May 7, 2020

done

@MartyG-RealSense
Copy link
Collaborator

Thanks very much! I will add the label to it now.

@mjsobrep
Copy link
Author

@MartyG-RealSense, having this problem again, but I don't think it is due to a version mismatch:
image

image

Any thoughts?

@mjsobrep mjsobrep reopened this Aug 17, 2020
@mjsobrep
Copy link
Author

I'm using 2 cameras, but if I plug in only the problematic one, I get:
image

If I have nothing plugged in, I get:
image

So the camera connects to some degree, but there is this power state issue. That seems to pop up a lot in the issues. What is that and how is it solved?

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Aug 17, 2020

Hi @mjsobrep Here are some things to try:

  • Add the term initial_reset:=true. to your roslaunch statement if you are not doing so already to see if causing the camera to reset upon roslaunch makes a difference
  • If you are using a serial number in your launch statement and are testing just one camera, leave the serial number out as it is not needed when you only have one camera plugged in
  • If you are using a launch file meant for multiple cameras when using both cameras - such as rs_multiple_devices.launch - then test with one camera using a roslaunch statement suited to one camera:
    roslaunch realsense2_camera rs_camera.launch
  • Try the USB cable from your working camera with the problematic one.
  • Try the problematic camera in a different USB port if you have one available
  • Remove the micro end of the USB cable from the side of the camera, turn the connector upside down and re-insert it into the side of the camera (it is a two-way fit at the micro end)
  • If the camera is plugged into a USB hub, try booting the computer with it plugged into one of the computer's built-in USB ports if you have one available. Even a USB 2.0 port would be fine if that is all you have vacant

@mjsobrep
Copy link
Author

mjsobrep commented Aug 17, 2020

Hi @MartyG-RealSense, thanks for the suggestions:

  • Added initial reset true, doesn't seem to fix the problem, but I'm going to leave it on
  • In the above images, the problem was even showing up with rs-enumerate-devices, which is not launch file dependent.
  • For whatever reason, after my ?th reboot, the cameras which are working switched, so I am thinking it is not a cable issue, since at different times they both have worked
  • my cameras plug in directly to the computer, into USB SuperSpeed-10 ports

I'm seeing a lot of this sort of spam in ROS,
image
...
image

This is the launch file for my realsense system

I am sometimes able to open both cameras in rs-viewer, but other times not.

Both cameras have been updated to the latest firmware through rs-viewer.

@mjsobrep
Copy link
Author

Actually, it is not just restarting the computer, just re running the launch file can cause a change in which camera chooses to work.

It seems like after a while, realsense-ros stops trying to connect to the camera, sets up the publishers, and just sits there quietly, not doing anything.

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Aug 18, 2020

Apologies for the delay in responding further, as I was carefully considering your case.

Do you have more than one program running at the same time that accesses the same camera? For example, launching the RealSense ROS wrapper and then the Viewer, or launching the Viewer first and then the ROS wrapper?

If the camera is already "claimed" by a program that has an active stream when you try to access the same camera with another program then the second program may fail to access the camera.

@mjsobrep
Copy link
Author

Never at the same time. There errors persist even after a computer restart. I'm not sure if the text of the errors is the same or not whether the camera has been used previously or if coming off a restart.

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Aug 18, 2020

Further research found a past case that had similar errors to this case. The case was about launching D435 and T265 together which is not the same situation as your one with two D415s but is still an interesting reference.

https://support.intelrealsense.com/hc/en-us/community/posts/360034992254-t265-and-d435-Failed-to-claim-USB-interface-error

One RealSense ROS user with problems launching two cameras wrote some arg code for delaying the launch of one of the cameras.

#774 (comment)

I note though that you have also had problems with sometimes recognizing the cameras in the Viewer, which is not related to the ROS launch process.

@mjsobrep
Copy link
Author

Hmm, those are interesting, I will try to see if some sort of delaying helps. Is there a large power draw at startup for the cameras?

When shutting down, I have noticed that one of the realsense nodes sometimes closes messily with a sigterm
image

This is an example of the not working camera thinking it is connected:
image
It is not, looking at the color image_raw and depth image_rect images:
image

It appears that the cameras do sometimes work, but I don't yet have a feel for what might be causing it to sometimes work or the frequency of that.

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Aug 18, 2020

I'm not sure how much power the camera draws during roslaunch. If streams are being started and published then one might expect it to be more than if the camera was detected in the Viewer but not currently streaming. The recommendation that I know of when calculating the camera power needs of a project is to budget for around 2W power draw per camera.

That is why using a mains electricity powered hub is useful for multiple camera projects, as they can supply around 12W of power.

I am away from computer now for 7 hours. Good luck!

@mjsobrep
Copy link
Author

After a night of leaving the cameras and computer off, things seemed to be working more consistently. I do not know why that might be. I lowered the resolution on the color imagers to 1080x720.

Just tried changing to 1920x1080 and it didn't work. Changing back to 1080x720 did again work, although after a lot of errors and a tough time finding the cameras.

So running these sequentially

trying outcome
Reboot and run at 1920x1080 with initial reset true one camera connects (does seem to change which camera is working)
Reboot and run at 1920x1080 with initial reset false neither camera works, looks like nodes actually crashed...
run at 1080x720 with initial reset false, no reboot neither camera works, getting "No Realsense devices were found!" warning
run at 1080x720 with initial reset true, no reboot neither camera works, getting "No Realsense devices were found!" warning
Reboot and run at 1080x720 with initial reset false neither camera works, getting "No Realsense devices were found!" warning
continuing from above, unplug and plug in both cameras Both cameras now work
stop nodes, restart at 1080x720 with initial reset false, no reboot Both cameras work
reboot and run at 1080x720 with initial reset false Nodes ended up crashing, see out1 below
stop nodes, restart at 1080x720 with initial reset false, no reboot neither camera works, getting "No Realsense devices were found!" warning
stop nodes, restart at 1080x720 with initial reset true, no reboot neither camera works, getting "No Realsense devices were found!" warning
continuing from above, unplug and plug in both cameras One camera works, the other crashed with out2
reboot and run at 1080x720 with initial reset true both cameras work
reboot and run at 1080x720 with initial reset true both cameras work
reboot and run at 1080x720 with initial reset true one camera crashes (out3) and the other can't be found (out4)
flip the camera's ports (another issue somewhere mentions a bug requiring serial numbers to be ascending by port number), reboot, run at 1080x720 with initial reset true both cameras work
reboot and run at 1080x720 with initial reset true one camera works, the other crashes (out5)
try plugging into a usb hub with max 2.1A available (this limitation is why I haven't used it), reboot, run at 1080x720 with initial reset true one camera works one doesn't
try plugging one camera into the hub one into the computer, reboot, run at 1080x720, initial reset true both cameras work
continue, reboot, 1080x720, initial reset true one camera works
plug cameras back into computer, 1080x720, initial reset true, start cameras after everything else, stagger start using bash sleep (so need to move to ros2 :'( ) both cameras work
repeat last both cameras work
repeat last both cameras work

I'm not sure how to setup the timed launch system you mention above to run launch files instead of nodes, need to figure out if that can be done. Tomorrow I will investigate whether the timing delay for starting the cameras is needed or if just starting them under different terminals is what is needed. If timing is what is important then it would be good to understand from your end why that is, do you have published power draw curves for starting up and running the cameras? Is there something in software that conflicts? Tomorrow I will also try to run at full resolution to see if this staggered start works with that.

out1:

 18/08 18:54:30,771 ERROR [140491492402944] (handle-libusb.h:95) failed to claim usb interface: 0, error: RS2_USB_STATUS_BUSY
 18/08 18:54:30,794 WARNING [140491525973760] (rs.cpp:304) null pointer passed for argument "device"
[ WARN] [1597791270.794228307]: Device 1/2 failed with exception: failed to set power state
[ WARN] [1597791270.821031122]: Asked to publish at 'fps' (30) which is higher than the 'set_camera_fps' (15), we can't publish faster than the camera provides images.

.....

18/08 18:54:50,594 WARNING [139845309544192] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 18/08 18:54:50,612 WARNING [140491484010240] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 18/08 18:54:50,612 WARNING [140491525973760] (types.cpp:78) set_xu(ctrl=1) failed! Last Error: Resource temporarily unavailable
[ERROR] [1597791290.613241399]: An exception has been thrown: set_xu(ctrl=1) failed! Last Error: Resource temporarily unavailable
terminate called after throwing an instance of 'rs2::invalid_value_error'
 18/08 18:54:50,613 WARNING [139845458224896] (types.cpp:78) set_xu(ctrl=1) failed! Last Error: Resource temporarily unavailable
  what():  set_xu(ctrl=1) failed! Last Error: Resource temporarily unavailable
[ERROR] [1597791290.614003341]: An exception has been thrown: set_xu(ctrl=1) failed! Last Error: Resource temporarily unavailable
terminate called after throwing an instance of 'rs2::invalid_value_error'
  what():  set_xu(ctrl=1) failed! Last Error: Resource temporarily unavailable
[lower_realsense/realsense2_camera_manager-23] process has died [pid 2964, exit code -6, cmd /opt/ros/kinetic/lib/nodelet/nodelet manager __name:=realsense2_camera_manager __log:=/home/nuc-admin/.ros/log/
c4fa8b00-e1a5-11ea-a88a-982cbccfb098/lower_realsense-realsense2_camera_manager-23.log].
log file: /home/nuc-admin/.ros/log/c4fa8b00-e1a5-11ea-a88a-982cbccfb098/lower_realsense-realsense2_camera_manager-23*.log
[upper_realsense/realsense2_camera_manager-27] process has died [pid 3001, exit code -6, cmd /opt/ros/kinetic/lib/nodelet/nodelet manager __name:=realsense2_camera_manager __log:=/home/nuc-admin/.ros/log/
c4fa8b00-e1a5-11ea-a88a-982cbccfb098/upper_realsense-realsense2_camera_manager-27.log].
log file: /home/nuc-admin/.ros/log/c4fa8b00-e1a5-11ea-a88a-982cbccfb098/upper_realsense-realsense2_camera_manager-27*.log
[upper_realsense/realsense2_camera-28] process has finished cleanly
log file: /home/nuc-admin/.ros/log/c4fa8b00-e1a5-11ea-a88a-982cbccfb098/upper_realsense-realsense2_camera-28*.log
[lower_realsense/realsense2_camera-24] process has finished cleanly
log file: /home/nuc-admin/.ros/log/c4fa8b00-e1a5-11ea-a88a-982cbccfb098/lower_realsense-realsense2_camera-24*.l

out2:

[ INFO] [1597791807.439703133]: setupDevice...
[ INFO] [1597791807.439730552]: JSON file is not provided
[ INFO] [1597791807.439745257]: ROS Node Namespace: lower_realsense
[ INFO] [1597791807.439760414]: Device Name: Intel RealSense D415
[ INFO] [1597791807.439773509]: Device Serial No: 904412060717
[ INFO] [1597791807.439788916]: Device physical port: 2-3-21
[ INFO] [1597791807.439801651]: Device FW version: 05.12.06.00
[ INFO] [1597791807.439811870]: Device Product ID: 0x0AD3
[ INFO] [1597791807.439821559]: Enable PointCloud: Off
[ INFO] [1597791807.439833162]: Align Depth: On
[ INFO] [1597791807.439844702]: Sync Mode: On
[ INFO] [1597791807.439889529]: Device Sensors: 
 18/08 19:03:27,439 ERROR [140328753407744] (handle-libusb.h:95) failed to claim usb interface: 0, error: RS2_USB_STATUS_BUSY
 18/08 19:03:27,450 WARNING [140328900060928] (rs.cpp:360) null pointer passed for argument "list"
[ERROR] [1597791807.450248494]: An exception has been thrown: failed to set power state
terminate called after throwing an instance of 'rs2::error'
  what():  failed to set power state
[lower_realsense/realsense2_camera_manager-23] process has died [pid 10824, exit code -6, cmd /opt/ros/kinetic/lib/nodelet/nodelet manager __name:=realsense2_camera_manager __log:=/home/nuc-admin/.ros/lo$
/c4fa8b00-e1a5-11ea-a88a-982cbccfb098/lower_realsense-realsense2_camera_manager-23.log].
log file: /home/nuc-admin/.ros/log/c4fa8b00-e1a5-11ea-a88a-982cbccfb098/lower_realsense-realsense2_camera_manager-23*.log
[ INFO] [1597791807.691790266]: Device with serial number 904412060717 was found.

out3:

[ INFO] [1597792570.880249894]: Device with serial number 912322061173 was found.

[ INFO] [1597792570.880279117]: Device with physical ID 2-4-4 was found.
[ INFO] [1597792570.880287386]: Device with name Intel RealSense D415 was found.
[ INFO] [1597792570.880719944]: Device with port number 2-4 was found.
[ INFO] [1597792570.882529422]: getParameters...
[ INFO] [1597792570.943920926]: setupDevice...
[ INFO] [1597792570.943954667]: JSON file is not provided
[ INFO] [1597792570.943969764]: ROS Node Namespace: upper_realsense
[ INFO] [1597792570.943985418]: Device Name: Intel RealSense D415
[ INFO] [1597792570.943993065]: Device Serial No: 912322061173
[ INFO] [1597792570.943999906]: Device physical port: 2-4-4
[ INFO] [1597792570.944006978]: Device FW version: 05.12.06.00
[ INFO] [1597792570.944013946]: Device Product ID: 0x0AD3
[ INFO] [1597792570.944020936]: Enable PointCloud: Off
[ INFO] [1597792570.944027398]: Align Depth: On
[ INFO] [1597792570.944036343]: Sync Mode: On
[ INFO] [1597792570.944117104]: Device Sensors: 
 18/08 19:16:10,944 ERROR [140436790286080] (handle-libusb.h:95) failed to claim usb interface: 0, error: RS2_USB_STATUS_BUSY
 18/08 19:16:10,954 WARNING [140436944922368] (rs.cpp:360) null pointer passed for argument "list"
[ERROR] [1597792570.954680004]: An exception has been thrown: failed to set power state
terminate called after throwing an instance of 'rs2::error'
  what():  failed to set power state
[upper_realsense/realsense2_camera_manager-27] process has died [pid 2850, exit code -6, cmd /opt/ros/kinetic/lib/nodelet/nodelet manager __name:=realsense2_camera_manager __log:=/home/nuc-admin/.ros/log$
c41f06c2-e1a8-11ea-91ac-982cbccfb098/upper_realsense-realsense2_camera_manager-27.log].
log file: /home/nuc-admin/.ros/log/c41f06c2-e1a8-11ea-91ac-982cbccfb098/upper_realsense-realsense2_camera_manager-27*.log
[ INFO] [1597792571.130816545]: Device with serial number 912322061173 was found.

[ INFO] [1597792571.130856141]: Device with physical ID 2-4-4 was found.
[ INFO] [1597792571.130887521]: Device with name Intel RealSense D415 was found.
[ INFO] [1597792571.131294457]: Device with port number 2-4 was found.
[ERROR] [1597792571.152285131]: The requested device with serial number 904412060717 is NOT found. Will Try again.
[upper_realsense/realsense2_camera-28] process has finished cleanly
log file: /home/nuc-admin/.ros/log/c41f06c2-e1a8-11ea-91ac-982cbccfb098/upper_realsense-realsense2_camera-28*.log
[ INFO] [1597792577.198455498]:  
 18/08 19:16:17,210 ERROR [140436698031872] (handle-libusb.h:95) failed to claim usb interface: 0, error: RS2_USB_STATUS_BUSY
 18/08 19:16:17,231 WARNING [140436827858688] (rs.cpp:304) null pointer passed for argument "device"

out4

This just keeps repeating:

[ INFO] [1597792727.138558932]: Device with physical ID 2-4-4 was found.
[ INFO] [1597792727.138579174]: Device with name Intel RealSense D415 was found.
[ INFO] [1597792727.139113313]: Device with port number 2-4 was found.
[ERROR] [1597792727.160540464]: The requested device with serial number 904412060717 is NOT found. Will Try again.
[ INFO] [1597792733.209254157]:  
 18/08 19:18:53,222 ERROR [140436698031872] (handle-libusb.h:95) failed to claim usb interface: 0, error: RS2_USB_STATUS_BUSY
 18/08 19:18:53,243 WARNING [140436827858688] (rs.cpp:304) null pointer passed for argument "device"
[ WARN] [1597792733.243276919]: Device 1/2 failed with exception: failed to set power state
[ INFO] [1597792733.620774728]: Device with serial number 912322061173 was found.

[ INFO] [1597792733.620800564]: Device with physical ID 2-4-4 was found.
[ INFO] [1597792733.620808152]: Device with name Intel RealSense D415 was found.
[ INFO] [1597792733.621159204]: Device with port number 2-4 was found.
[ERROR] [1597792733.642218041]: The requested device with serial number 904412060717 is NOT found. Will Try again.
[ERROR] [1597792737.586395839]: Kobuki : malformed sub-payload detected. [25][170][19 AA 55 4D 01 0F ]
[ INFO] [1597792739.690636658]:  
 18/08 19:18:59,703 ERROR [140436706424576] (handle-libusb.h:95) failed to claim usb interface: 0, error: RS2_USB_STATUS_BUSY
 18/08 19:18:59,724 WARNING [140436827858688] (rs.cpp:304) null pointer passed for argument "device"
[ WARN] [1597792739.724495557]: Device 1/2 failed with exception: failed to set power state
[ INFO] [1597792740.120868031]: Device with serial number 912322061173 was found.

[ INFO] [1597792740.120903760]: Device with physical ID 2-4-4 was found.
[ INFO] [1597792740.120914623]: Device with name Intel RealSense D415 was found.
[ INFO] [1597792740.121461940]: Device with port number 2-4 was found.
[ERROR] [1597792740.142978243]: The requested device with serial number 904412060717 is NOT found. Will Try again.
[ INFO] [1597792746.192939340]:  
 18/08 19:19:06,206 ERROR [140436689639168] (handle-libusb.h:95) failed to claim usb interface: 0, error: RS2_USB_STATUS_BUSY
 18/08 19:19:06,226 WARNING [140436827858688] (rs.cpp:304) null pointer passed for argument "device"
[ WARN] [1597792746.226949295]: Device 1/2 failed with exception: failed to set power state
[ INFO] [1597792746.618079961]: Device with serial number 912322061173 was found.

out5

 18/08 19:39:03,730 WARNING [139913949128448] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Device or resource busy, number: 16
 18/08 19:39:03,790 WARNING [139913949128448] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Device or resource busy, number: 16
 18/08 19:39:03,851 WARNING [139913982699264] (types.cpp:78) set_xu(ctrl=1) failed! Last Error: Resource temporarily unavailable
 18/08 19:39:03,874 WARNING [139913982699264] (rs.cpp:304) null pointer passed for argument "device"
[ WARN] [1597793943.874385919]: Device 1/2 failed with exception: set_xu(ctrl=1) failed! Last Error: Resource temporarily unavailable
[upper_realsense/realsense2_camera_manager-27] process has died [pid 2995, exit code -11, cmd /opt/ros/kinetic/lib/nodelet/nodelet manager __name:=realsense2_camera_manager __log:=/home/nuc-admin/.ros/lo$
/fa9fb2de-e1ab-11ea-ac81-982cbccfb098/upper_realsense-realsense2_camera_manager-27.log].
log file: /home/nuc-admin/.ros/log/fa9fb2de-e1ab-11ea-ac81-982cbccfb098/upper_realsense-realsense2_camera_manager-27*.log
[ INFO] [1597793944.362110477]:  
 18/08 19:39:04,374 ERROR [140144354014976] (handle-libusb.h:95) failed to claim usb interface: 0, error: RS2_USB_STATUS_BUSY
 18/08 19:39:04,395 WARNING [140144395978496] (rs.cpp:304) null pointer passed for argument "device"
[ WARN] [1597793944.395339391]: Device 1/2 failed with exception: failed to set power state
──────────────────────────────────────────────────────────────────────────────────┬─────────────

@MartyG-RealSense
Copy link
Collaborator

Thanks very much for the very detailed testing report!

With ROS, the multicam launch timing issue particularly affected a pairing of D435 and T265 cameras. If D435 launched first then T265 could not be recognized properly. #774 that linked to earlier sums this phenomenon up well.

Would you say that you get the most frequent success when both cameras are plugged into the computer (instead of both in the hub or one in hub and one in computer)?

There was a recent case where a RealSense user had no problems when both cameras were plugged into the full size USB ports on their computer but had problems if using the micro USB-C port on the computer. They found that the ports that worked all used an Intel USB controller with xHCI support. The USB-C port that didn't work used an Asmedia USB controller without xHCI support.

IntelRealSense/librealsense#6899 (comment)

The workaround that they eventually found was to create a script that rebinds the USB driver.

IntelRealSense/librealsense#6899 (comment)

@mjsobrep
Copy link
Author

You're welcome, hopefully it can help in tracking this all down.

I read through that report. Since both of my cameras are D415s, I'm not sure how to interpret that. There are reports elsewhere in the issues about there being a sensitivity to starting the D415s in order of their serial number or plugging them in the order of their serial number.

The hub plugs into a thunderbolt port and is powered by a battery pack (the computer, an intel nuc is also powered by the same battery pack). There are a lot of other devices plugged into the hub (described in this paper) so I an concerned about stability of a camera being plugged into it. I have not noticed an improvement by plugging into the hub. We know that the realsense pushes USB pretty hard, that is why we need nice cables and the like. I have no confidence that a commodity hub off Amazon has the same quality of connection as the high end cables we all use. I would hope that whatever is breaking out USB inside of a computer (or maybe the nuc has ports with dedicated controllers?) is higher quality than an off the shelf hub. So for those reasons, I think I am going to avoid using the hub for now.

That is interesting. Wouldn't restarting the computer prevent that sort of rebinding from being needed? Once I get it all working after a reboot then perhaps I will take a look at starting after stopping.

I think there is some underlying engineering work that could be done on this to characterize the sensors and how to properly start them up. For a long period of time, I have been using an older NUC with this launch file (with the custom record param off). So I have been running with just color at a moderate resolution. I recently upgraded to a newer nuc with much better compute capabilities. Now using depth with higher resolution on the color than before.

@MartyG-RealSense
Copy link
Collaborator

When Intel wrote their white paper on multiple cameras for the 400 Series cameras, they tested with the AmazonBasics 4-port mains powered hub from Amazon. I bought the 7-port version of it based on that recommendation and have had no problems at all with it.

I have seen a couple of reports in the past of expensive USB 3.2 industrial grade hubs having problems with RealSense, whilst a cheap USB 3.1 hub worked fine. So industrial-grade may not be a guaranteed solution if there is a conflict with another aspect of the product (such as the USB 3.2 standard).

I believe that the issue of needing to insert cameras in a certain order mostly applies to Nvidia Jetson boards rather than NUCs.

In regard to an earlier question about power draw: the data sheet document for the 400 Series has extreme levels of detail about power delivery. If you open the PDF and do a search operation for the word power then that will help you to browse quickly through all the references.

https://dev.intelrealsense.com/docs/intel-realsense-d400-series-product-family-datasheet

image

Regarding the rebinding: my knowledge is hazy on that aspect, but I believe that there can be sections in the USB section of the computer's BIOS boot-up settings that specify whether a port has xHCI support. BIOSes tend to vary widely between different computers though depending on what chipset the computer is using.

I have a NUC myself as my main personal workstation, a NUC8i5INH with Radeon graphics. It has been excellent for me.

https://www.intel.co.uk/content/www/uk/en/products/boards-kits/nuc/mini-pcs/nuc8i5inh.html

@mjsobrep
Copy link
Author

Hmm, so do you know if the realsense cameras have more trouble with USB 3.2 ports compared to USB 3.1? The new nuc that I am using is all USB 3.2 ports.

Looking through the spec sheet. I see a number of places where power sequencing is discussed, as well as steady state power consumption. No discussion of any sort of higher turn on current, maybe because it doesn't exist. It would be interesting to see some real world plots of power consumption during startup of the cameras, but I am more and more thinking that is not the problem.

Do you know what the ideal settings are for the USB controllers in the BIOS (of course every bios is different, but maybe for the intel nuc bios since those are widely used and we both have them)?

@MartyG-RealSense
Copy link
Collaborator

I re-reviewed the case. control_transfer returned errors are very hard to diagnose a specific cause for though. I recall a case where the errors were solved simply by updating OS and kernel. I note that your problems at the start of this case began when you updated the computer. So there may indeed by a causal relationship between OS / kernel and problems.

@mjsobrep
Copy link
Author

hmm, that is helpful to know. The OS is Ubuntu 16 with the latest kernel. Perhaps a simple re-install of the OS would fix the problem.

Could secure boot cause these sorts of problems?

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Aug 25, 2020

There is not a great deal of concrete evidence for or against having secure boot turned off. Having Secure boot disabled was a recommendation for the previous RealSense ROS wrapper (realsense_camera) for the earlier generation of cameras though as it made DKMS packages fail. And having it enabled may affect patching of the video driver in Linux, in which case a member of the RealSense team recommended disabling it in 2019.

IntelRealSense/librealsense#3354 (comment)

If it is not a policy of your company that Secure Boot has to be enabled on its machines then it should be okay to turn it off.

@MartyG-RealSense
Copy link
Collaborator

Hi @mjsobrep Do you require further assistance with this case, please? Thanks!

@mjsobrep
Copy link
Author

mjsobrep commented Sep 5, 2020

Hey @MartyG-RealSense I think I am good for right now. I have some stuff that I want to check out, but have the information I think I need. Thanks!

@mjsobrep mjsobrep closed this as completed Sep 5, 2020
@MartyG-RealSense
Copy link
Collaborator

Hi @mjsobrep Thanks very much for the update!

@jcgarciaca
Copy link

I am facing a similar issue. I have a robot with three D435i cameras and sometimes I got RS2_USB_STATUS_BUSY with any of them (it is not always the same camera which fails). Usually, I just stop and re-launch them again and everything works well. I added launch delays, but the error still appears. Actually, it appears the first time I try to run the cameras (when turning on the PC), although it appears any time. Cameras are directly connected to three different USB3 ports.

How to avoid this to happen?

Just for reference, I use RealSense ROS v2.2.20 and LibRealSense v2.40.0.

Thank you

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Dec 23, 2020

Hi @jcgarciaca Which roslaunch method are you using to launch your multiple cameras please - rs_camera.launch or rs_multiple_devices.launch?

https://github.com/IntelRealSense/realsense-ros#work-with-multiple-cameras

@jcgarciaca
Copy link

Hi @MartyG-RealSense, actually I use rs_rgbd.launch with corresponding camera and serial_no arguments for every camera.

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Dec 23, 2020

Are you putting each camera on a separate terminal on the same computer, with a separate roslaunch for each camera, as Step Two of Intel's multiple camera ROS tutorial suggests?

https://www.intelrealsense.com/how-to-multiple-camera-setup-with-ros/

@jcgarciaca
Copy link

jcgarciaca commented Dec 23, 2020

Yes, you're right. But I use rs_rgbd

<terminal 1>
roslaunch realsense2_camera rs_rgbd.launch camera:=cam_1 serial_no:=<serial_number_1>
<2 sec delay>

<terminal 2>
roslaunch realsense2_camera rs_rgbd.launch camera:=cam_2 serial_no:=<serial_number_2>
<2 sec delay>

<terminal 3>
roslaunch realsense2_camera rs_rgbd.launch camera:=cam_3 serial_no:=<serial_number_3>

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Dec 23, 2020

Do you experience any improvement in performance if you add a reset instruction to the end of each of the three roslaunch instructions so that the camera undergoes a hardware reset at launch (similar to a USB unplug-replug of the camera).

initial_reset:=true

@jcgarciaca
Copy link

I just tested with the initial_reset set to true, but I got the same error

@MartyG-RealSense
Copy link
Collaborator

@jcgarciaca You said in your original comment "Usually, I just stop and re-launch them again and everything works well". Could you tell me the method that you use to re-launch the cameras please?

@jcgarciaca
Copy link

jcgarciaca commented Dec 23, 2020

In my application I have some sh scripts to execute roslaunch.
Inside every sh file I have roslaunch realsense2_camera rs_rgbd.launch camera:=cam_X serial_no:=<serial_number_X>
Then, when I need to stop the execution I do rosnode kill -a, killall -9 roscore and killall -9 rosmaster and I re-run the corresponding sh files and the cameras start properly.

@MartyG-RealSense
Copy link
Collaborator

Okay, thank you very much for the information. I will continue to investigate on Thursday. Thanks for your patience!

@MartyG-RealSense
Copy link
Collaborator

If the cameras work correctly after a stop and restart, I wonder if you could incorporate a start-stop-start into your sh script.

Start
roslaunch realsense2_camera rs_rgbd.launch camera:=cam_X serial_no:=<serial_number_X>
Stop
rosnode kill -a, killall -9 roscore and killall -9 rosmaster
Start
roslaunch realsense2_camera rs_rgbd.launch camera:=cam_X serial_no:=<serial_number_X>

@jcgarciaca
Copy link

jcgarciaca commented Dec 29, 2020

Well, with the roslaunch API I can start and stop the cameras when errors are detected. Such that it is a way to solve this situation.

@MartyG-RealSense
Copy link
Collaborator

Thanks very much @jcgarciaca

@doronhi
Copy link
Contributor

doronhi commented Dec 31, 2020

Right now I don't have a better solution for the occasional RS2_USB_STATUS_BUSY error.
#1504 introduces a "reset" service. I wonder if you would find it useful for now.

@doronhi
Copy link
Contributor

doronhi commented Jan 13, 2021

RS2_USB_STATUS_BUSY is a message of the RSUSB backend which is built into the ros-librealsense2 package.
For most platforms, especially when using multiple cameras, it's best to use the build that uses the v4l backend.
That build exist in the normal installation of librealsense2 as described in librealsense2 installation guide.
In order to use it you'll have to remove the ros-librealsense2 and realsense2_camera installation:
That is:
sudo apt purge ros-melodic-realsense2_camera

And rebuild realsense2_camera from code

Does the reliability seems better with that librealsense2 build?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants