Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The issue of "No stream match for pointcloud chosen texture Process" when playing back recording #1826

Closed
victorll998 opened this issue Apr 27, 2021 · 22 comments
Labels

Comments

@victorll998
Copy link

Hi everyone,

I have been testing with rs_from_file.launch. Now I can replay depth and color stream from the bag files that I recorded either through rs_record or realsense viewer. But when I tried to reproduce point cloud from the d455 bag file using rs_from_file.launch , I got nothing. To be more exact, I got a warning " No stream match for pointcloud chosen texture Process - Color "

SUMMARY
========

PARAMETERS
 * /d455_left/realsense2_camera/accel_fps: 0
 * /d455_left/realsense2_camera/accel_frame_id: d455_left_accel_f...
 * /d455_left/realsense2_camera/accel_optical_frame_id: d455_left_accel_o...
 * /d455_left/realsense2_camera/align_depth: True
 * /d455_left/realsense2_camera/aligned_depth_to_color_frame_id: d455_left_aligned...
 * /d455_left/realsense2_camera/aligned_depth_to_fisheye1_frame_id: d455_left_aligned...
 * /d455_left/realsense2_camera/aligned_depth_to_fisheye2_frame_id: d455_left_aligned...
 * /d455_left/realsense2_camera/aligned_depth_to_fisheye_frame_id: d455_left_aligned...
 * /d455_left/realsense2_camera/aligned_depth_to_infra1_frame_id: d455_left_aligned...
 * /d455_left/realsense2_camera/aligned_depth_to_infra2_frame_id: d455_left_aligned...
 * /d455_left/realsense2_camera/allow_no_texture_points: False
 * /d455_left/realsense2_camera/base_frame_id: d455_left_link
 * /d455_left/realsense2_camera/calib_odom_file: 
 * /d455_left/realsense2_camera/clip_distance: -1.0
 * /d455_left/realsense2_camera/color_fps: 30
 * /d455_left/realsense2_camera/color_frame_id: d455_left_color_f...
 * /d455_left/realsense2_camera/color_height: 480
 * /d455_left/realsense2_camera/color_optical_frame_id: d455_left_color_o...
 * /d455_left/realsense2_camera/color_width: 640
 * /d455_left/realsense2_camera/confidence_fps: 30
 * /d455_left/realsense2_camera/confidence_height: 480
 * /d455_left/realsense2_camera/confidence_width: 640
 * /d455_left/realsense2_camera/depth_fps: 30
 * /d455_left/realsense2_camera/depth_frame_id: d455_left_depth_f...
 * /d455_left/realsense2_camera/depth_height: 480
 * /d455_left/realsense2_camera/depth_optical_frame_id: d455_left_depth_o...
 * /d455_left/realsense2_camera/depth_width: 640
 * /d455_left/realsense2_camera/device_type: 
 * /d455_left/realsense2_camera/enable_accel: False
 * /d455_left/realsense2_camera/enable_color: True
 * /d455_left/realsense2_camera/enable_confidence: True
 * /d455_left/realsense2_camera/enable_depth: True
 * /d455_left/realsense2_camera/enable_fisheye1: False
 * /d455_left/realsense2_camera/enable_fisheye2: False
 * /d455_left/realsense2_camera/enable_fisheye: False
 * /d455_left/realsense2_camera/enable_gyro: False
 * /d455_left/realsense2_camera/enable_infra1: False
 * /d455_left/realsense2_camera/enable_infra2: False
 * /d455_left/realsense2_camera/enable_infra: False
 * /d455_left/realsense2_camera/enable_pointcloud: True
 * /d455_left/realsense2_camera/enable_pose: False
 * /d455_left/realsense2_camera/enable_sync: True
 * /d455_left/realsense2_camera/filters: 
 * /d455_left/realsense2_camera/fisheye1_frame_id: d455_left_fisheye...
 * /d455_left/realsense2_camera/fisheye1_optical_frame_id: d455_left_fisheye...
 * /d455_left/realsense2_camera/fisheye2_frame_id: d455_left_fisheye...
 * /d455_left/realsense2_camera/fisheye2_optical_frame_id: d455_left_fisheye...
 * /d455_left/realsense2_camera/fisheye_fps: 30
 * /d455_left/realsense2_camera/fisheye_frame_id: d455_left_fisheye...
 * /d455_left/realsense2_camera/fisheye_height: 0
 * /d455_left/realsense2_camera/fisheye_optical_frame_id: d455_left_fisheye...
 * /d455_left/realsense2_camera/fisheye_width: 0
 * /d455_left/realsense2_camera/gyro_fps: 0
 * /d455_left/realsense2_camera/gyro_frame_id: d455_left_gyro_frame
 * /d455_left/realsense2_camera/gyro_optical_frame_id: d455_left_gyro_op...
 * /d455_left/realsense2_camera/imu_optical_frame_id: d455_left_imu_opt...
 * /d455_left/realsense2_camera/infra1_frame_id: d455_left_infra1_...
 * /d455_left/realsense2_camera/infra1_optical_frame_id: d455_left_infra1_...
 * /d455_left/realsense2_camera/infra2_frame_id: d455_left_infra2_...
 * /d455_left/realsense2_camera/infra2_optical_frame_id: d455_left_infra2_...
 * /d455_left/realsense2_camera/infra_fps: 30
 * /d455_left/realsense2_camera/infra_height: 480
 * /d455_left/realsense2_camera/infra_rgb: False
 * /d455_left/realsense2_camera/infra_width: 640
 * /d455_left/realsense2_camera/initial_reset: False
 * /d455_left/realsense2_camera/json_file_path: 
 * /d455_left/realsense2_camera/linear_accel_cov: 0.01
 * /d455_left/realsense2_camera/odom_frame_id: d455_left_odom_frame
 * /d455_left/realsense2_camera/ordered_pc: False
 * /d455_left/realsense2_camera/pointcloud_texture_index: 0
 * /d455_left/realsense2_camera/pointcloud_texture_stream: RS2_STREAM_COLOR
 * /d455_left/realsense2_camera/pose_frame_id: d455_left_pose_frame
 * /d455_left/realsense2_camera/pose_optical_frame_id: d455_left_pose_op...
 * /d455_left/realsense2_camera/publish_odom_tf: True
 * /d455_left/realsense2_camera/publish_tf: True
 * /d455_left/realsense2_camera/rosbag_filename: /home/parallels/D...
 * /d455_left/realsense2_camera/serial_no: 
 * /d455_left/realsense2_camera/stereo_module/exposure/1: 7500
 * /d455_left/realsense2_camera/stereo_module/exposure/2: 1
 * /d455_left/realsense2_camera/stereo_module/gain/1: 16
 * /d455_left/realsense2_camera/stereo_module/gain/2: 16
 * /d455_left/realsense2_camera/tf_publish_rate: 0.0
 * /d455_left/realsense2_camera/topic_odom_in: d455_left/odom_in
 * /d455_left/realsense2_camera/unite_imu_method: none
 * /d455_left/realsense2_camera/usb_port_id: 
 * /rosdistro: melodic
 * /rosversion: 1.14.10

NODES
  /d455_left/
    realsense2_camera (nodelet/nodelet)
    realsense2_camera_manager (nodelet/nodelet)

auto-starting new master
process[master]: started with pid [16377]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to c5cfea14-a71d-11eb-9e08-001c42b717af
process[rosout-1]: started with pid [16391]
started core service [/rosout]
process[d455_left/realsense2_camera_manager-2]: started with pid [16398]
process[d455_left/realsense2_camera-3]: started with pid [16399]
[ INFO] [1619503188.789944166]: Initializing nodelet with 8 worker threads.
[ INFO] [1619503188.957092140]: RealSense ROS v2.2.23
[ INFO] [1619503188.957134965]: Built with LibRealSense v2.44.0
[ INFO] [1619503188.957155517]: Running with LibRealSense v2.44.0
[ INFO] [1619503188.969869258]: publish topics from rosbag file: /home/parallels/Documents/20210427_150421.bag
[ INFO] [1619503189.006703823]: getParameters...
[ INFO] [1619503189.050615456]: setupDevice...
[ INFO] [1619503189.050652789]: JSON file is not provided
[ INFO] [1619503189.050671083]: ROS Node Namespace: d455_left
[ INFO] [1619503189.050688392]: Device Name: Intel RealSense D455
[ INFO] [1619503189.050710862]: Device Serial No: 043422252203
[ INFO] [1619503189.050726808]: Device physical port: /sys/devices/pci0000:00/0000:00:1d.6/usb4/4-9/4-9:1.0/video4linux/video3
[ INFO] [1619503189.050765890]: Device FW version: 05.12.12.100
[ INFO] [1619503189.050793871]: Device Product ID: 0x0B5C
[ INFO] [1619503189.050819998]: Enable PointCloud: On
[ INFO] [1619503189.050838882]: Align Depth: On
[ INFO] [1619503189.050856494]: Sync Mode: On
[ INFO] [1619503189.050897358]: Device Sensors: 
[ INFO] [1619503189.050998008]: Stereo Module was found.
[ INFO] [1619503189.051066980]: RGB Camera was found.
[ INFO] [1619503189.051086243]: Motion Module was found.
[ INFO] [1619503189.051114878]: (Confidence, 0) sensor isn't supported by current device! -- Skipping...
[ INFO] [1619503189.051162240]: Add Filter: pointcloud
[ INFO] [1619503189.051425231]: num_filters: 1
[ INFO] [1619503189.051453191]: Setting Dynamic reconfig parameters.
[ INFO] [1619503189.059235488]: Done Setting Dynamic reconfig parameters.
[ WARN] [1619503189.059376911]: Given stream configuration is not supported by the device!  Stream: Depth, Stream Index: 0, Width: 640, Height: 480, FPS: 30, Format: Z16
[ WARN] [1619503189.059447383]: Using default profile instead.
[ INFO] [1619503189.059855952]: depth stream is enabled - width: 640, height: 480, fps: 5, Format: Z16
[ WARN] [1619503189.059890554]: Given stream configuration is not supported by the device!  Stream: Color, Stream Index: 0, Width: 640, Height: 480, FPS: 30, Format: None
[ WARN] [1619503189.059912491]: Using default profile instead.
[ INFO] [1619503189.060321779]: color stream is enabled - width: 640, height: 480, fps: 5, Format: RGB8
[ INFO] [1619503189.061302098]: setupPublishers...
[ INFO] [1619503189.063261523]: Expected frequency for depth = 5.00000
[ INFO] [1619503189.084820095]: Expected frequency for color = 5.00000
[ INFO] [1619503189.097839129]: Expected frequency for aligned_depth_to_color = 5.00000
[ INFO] [1619503189.111305855]: setupStreams...
[ INFO] [1619503189.111407815]: insert Depth to Stereo Module
[ INFO] [1619503189.111447757]: insert Color to RGB Camera
[ INFO] [1619503189.111620735]: SELECTED BASE:Depth, 0
[ INFO] [1619503189.111860884]: RealSense Node Is Up!
[ WARN] [1619503189.113693673]: 
 27/04 15:59:49,113 WARNING [140200667764480] (playback_device.cpp:198) Playback device does not provide a matcher
 27/04 15:59:49,151 WARNING [140200676157184] (playback_device.cpp:198) Playback device does not provide a matcher
[ INFO] [1619503192.151847262]: publishing unordered pointcloud.
[ WARN] [1619503192.951852379]: No stream match for pointcloud chosen texture Process - Color

I am not sure if I made a mistake or the base node doesn't support point cloud reproduction. If so, Is there any way to reproduce pointcloud from the recording? Thank you in advance.

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Apr 27, 2021

Hi @victorll998 It was stated by Doronhi the RealSense ROS developer in a past case (in the link below) that the warning No stream match for pointcloud chosen texture Process - Color can occur due to a frame drop.

#588 (comment)

I note in your launch log that the color stream FPS is initially set at 30 FPS but the camera configuration was decided by the launch process to be invalid and color was set to 5 FPS instead (the minimum FPS of the D455 camera model).

I also note that your log indicates that you have a mismatch between your librealsense SDK version and ROS wrapper version. For SDK 2.44.0, the wrapper version 2.2.24 should be used instead of 2.2.23. This is because it is important to match the wrapper version as closely as possible with the specific librealsense version recommended in a wrapper's release notes.

https://github.com/IntelRealSense/realsense-ros/releases/tag/2.2.24

image

@doronhi
Copy link
Contributor

doronhi commented Apr 29, 2021

I think it is related to #1778
Waiting for a fix in librealsense2

@victorll998 victorll998 changed the title How to reproduce pointcloud from recording? The issue of "No stream match for pointcloud chosen texture Process" when playing back recording May 3, 2021
@victorll998
Copy link
Author

@MartyG-RealSense thank you for your responses. How careless I am to not notice the mismatch of version and now I have fixed that.

Now I also notice the fps has been reset by the launch process. Does that mean I need to know the fps and resolution too before I load the recording using rs_from_file? The default values in '''rs_from_file``` are all set to 0 which is very confusing. Shouldn't it make more sense when getting the resolution and fps from the existing video_stream_ profile?

@MartyG-RealSense
Copy link
Collaborator

What is the full roslaunch instruction that you are using when attempting to produce a point cloud from the bag data, please? Is it this:

roslaunch realsense2_camera rs_from_file.launch filters:=pointcloud

@victorll998
Copy link
Author

yes, I am using " rs_from_file.launch" filters:=pointcloud rosbag_filename:=globalfilepath

@MartyG-RealSense
Copy link
Collaborator

Hi @victorll998 I apologize for the delay in responding further. Do you still have the No stream match for pointcloud chosen texture Process - Color please?

@victorll998
Copy link
Author

@MartyG-RealSense

I have done some more tests on Nano Tx2 and unfortunately this issue is still there. Now I have switched to intel NUC i7 to eliminate any possible frame drops. I observed that other than frame drop if the first frame in depth and color stream doesn't match in timestamp it would cause this issue too. But I only have this issue when running it on 2.2.44. Now I am not sure if this issue comes along with the version. What are your thoughts?

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented May 12, 2021

Now that the RealSense ROS wrapper 2.3.0 for use with librealsense SDK version 2.45.0 is available, is it possible for you to test with these versions to see whether No stream match for pointcloud chosen texture Process - Color continues to be present, given that you only experience the problem on the previous ROS wrapper 2.2.24

https://github.com/IntelRealSense/realsense-ros/releases/tag/2.3.0

@doronhi
Copy link
Contributor

doronhi commented May 12, 2021

As far as I know, the causing issue that first appeared in librealsense2 v2.43.0, reported in #1778, is not fixed yet.

@MartyG-RealSense
Copy link
Collaborator

Hi @victorll998 Bearing in mind the comment of Doronhi above, do you require further assistance with this case please? Thanks!

@victorll998
Copy link
Author

Thank you @MartyG-RealSense

I hope this issue would be fixed soon. Meanwhile, I would like to if there is any way that I could change time tag of the recording when I playback recording using sim_time.

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented May 27, 2021

I researched your use_sim_time question. As far as I can tell, modifying the timing of use_sim_time from its default '0' value would require use of the /clock topic.

http://wiki.ros.org/Clock#Using_Simulation_Time_from_the_.2Fclock_Topic

https://answers.ros.org/question/361531/publish-rosbag-in-synch-with-clock-by-roscpp/

@MartyG-RealSense
Copy link
Collaborator

Hi @victorll998 Do you require further assistance with this case, please? Thanks!

@victorll998
Copy link
Author

victorll998 commented Jun 3, 2021

@MartyG-RealSense

Sorry for the late update. use_sim_time is be useful to sync topics when reading from one rosbag file. In my multi-cameras node case, I create a sudo /clock signal using frame.get_timestamp() from one of rosbag files and it allows all ros nodes to keep a common time tag that I want.

But there are two follow-up questions which I am eager to know the answers:

  1. Because point cloud reproduction issue merges since v2.43.0, I decided to trace back/downgrade to v2.42.0 from v2.45.0. I had a refresh install of Ubuntu 20.04 to avoid any chance of realsense packages not cleaning up. Long story short, I can replay pointcloud from rosbag files but t265 is not recognized/detected anymore on my system. Check my latest comment in this thread more details

  2. I wonder if there is any possibility that I can stop/resume the playback when using rs_from_file

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Jun 3, 2021

Thanks very much @victorll998 for the update regarding your success!

In regard to your follow-up questions:

  1. I am not involved in T265 support, so I cannot provide advice on that particular subject unfortunately.

  2. You could maybe use rosnode kill to exit from the launched RealSense ROS node.

#1187 (comment)

@MartyG-RealSense
Copy link
Collaborator

Hi @victorll998 Regarding your second question "I wonder if there is any possibility that I can stop/resume the playback when using rs_from_file": do you require further assistance with this please? Thanks!

@victorll998
Copy link
Author

Hi @MartyG-RealSense Thank you for your consistent and useful help. I really appreciate that. I walked around a big circle to find out that rosnode kill is the most effective method to stop ROS nodes. Before that I tried roslaunch python api but it won't let me kill ROS nodes completely.

Currently, I'm working on building a controller of realsense. I want to develop via ROS and C++ because most of my system's processing is on ROS. What I found is that realsense ROS has only little freedom to manipulate from the lower level. For example, it is difficult to detect and extract device/sensor states. If I develop via SDK, I could miss some useful ROS functionalities. I'm seeking somewhere between the two. I wonder if you have any suggestions on that.

@MartyG-RealSense
Copy link
Collaborator

You can obtain some information from the /diagnostics topic that the RealSense ROS wrapper publishes, including ASIC, projector and motion module temperature readings.

If you are able to use ROS2, the 3.1.6 version of the ROS2 wrapper in May 2021 also added an enable service to toggle sensors on / off.

@victorll998
Copy link
Author

@MartyG-RealSense actually, I'm thinking if the realsense ros wrapper has any heartbeat-like function/topics

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Jun 14, 2021

I suppose that the current FPS speed of a particular stream would be like a heartbeat, with a relatively fixed and consistent rate when healthy and demonstrating significant variations when unhealthy. rostopic hz could display the average publishing rate of a particular stream topic.

http://wiki.ros.org/rostopic#rostopic_hz

The RealSense-using robot_devkit project has also suggested a couple of performance monitoring tools for ROS2 (rtmonitor and rdkperf).

https://intel.github.io/robot_devkit_doc/pages/overview.html#id6

@MartyG-RealSense
Copy link
Collaborator

Hi @victorll998 Do you require further assistance with this case, please? Thanks!

@MartyG-RealSense
Copy link
Collaborator

Case closed due to no further comments received.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants