Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to save an Intel RealSense D435 capture using ROS2 humble in a .bag file in mp4 format? #2918

Closed
monajalal opened this issue Oct 30, 2023 · 10 comments
Labels

Comments

@monajalal
Copy link

monajalal commented Oct 30, 2023

Description

I want to save the capture in .mp4 format for use in upstream app. I am able to view the feed in RVIZ and record the capture in bag files. I don't care about the method or package used as long as I can get a depth align RGB feed along with depth from RealSense D435 camera that I can save to a video file (or a series of consecutive frames).

Expected Behavior

** clear and concise description of what you expected to happen **

Actual Behavior

** clear and concise description of what actually happened. include copied logs whenever possible **

To Reproduce

** Steps to reproduce the behavior, e.g.

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. Logs print error '...' **

System (please complete the following information)

  • OS: Ubuntu 22.04 Jammy
  • ROS 2 Distro: Humble
  • Install Method: APT
  • Version: [release, branch, commit hash, patch release number]

Additional context

** Add any other context about the problem here **

(base) mona@mona-ThinkStation-P7:~/realsense-ros$ ros2 bag record -a
2023-10-24 15:31:27.067 [RTPS_TRANSPORT_SHM Error] Failed init_port fastrtps_port7417: open_and_lock_file failed -> Function open_port_internal
[INFO] [1698175887.078891824] [rosbag2_recorder]: Press SPACE for pausing/resuming
[INFO] [1698175887.080581927] [rosbag2_storage]: Opened database 'rosbag2_2023_10_24-15_31_27/rosbag2_2023_10_24-15_31_27_0.db3' for READ_WRITE.
[INFO] [1698175887.081582715] [rosbag2_recorder]: Listening for topics...
[INFO] [1698175887.081620349] [rosbag2_recorder]: Event publisher thread: Starting
[INFO] [1698175887.083172945] [rosbag2_recorder]: Subscribed to topic '/rosout'
[INFO] [1698175887.083883528] [rosbag2_recorder]: Subscribed to topic '/parameter_events'
[INFO] [1698175887.085310809] [rosbag2_recorder]: Subscribed to topic '/initialpose'
[INFO] [1698175887.086000091] [rosbag2_recorder]: Subscribed to topic '/goal_pose'
[INFO] [1698175887.086516674] [rosbag2_recorder]: Subscribed to topic '/events/write_split'
[INFO] [1698175887.087021693] [rosbag2_recorder]: Subscribed to topic '/clicked_point'
[INFO] [1698175887.087135670] [rosbag2_recorder]: Recording...
[INFO] [1698175888.098073631] [rosbag2_recorder]: Subscribed to topic '/camera/camera/infra1/camera_info'
[INFO] [1698175888.098993595] [rosbag2_recorder]: Subscribed to topic '/camera/camera/color/camera_info'
[INFO] [1698175888.100210654] [rosbag2_recorder]: Subscribed to topic '/camera/camera/extrinsics/depth_to_infra1'
[INFO] [1698175888.101016168] [rosbag2_recorder]: Subscribed to topic '/camera/camera/extrinsics/depth_to_depth'
[INFO] [1698175888.102269815] [rosbag2_recorder]: Subscribed to topic '/camera/camera/infra2/image_rect_raw'
[INFO] [1698175888.103011224] [rosbag2_recorder]: Subscribed to topic '/camera/camera/color/image_raw'
[INFO] [1698175888.103726223] [rosbag2_recorder]: Subscribed to topic '/camera/camera/extrinsics/depth_to_infra2'
[INFO] [1698175888.105131418] [rosbag2_recorder]: Subscribed to topic '/tf_static'
[INFO] [1698175888.105814924] [rosbag2_recorder]: Subscribed to topic '/camera/camera/color/metadata'
[INFO] [1698175888.106507153] [rosbag2_recorder]: Subscribed to topic '/camera/camera/depth/image_rect_raw'
[INFO] [1698175888.107098512] [rosbag2_recorder]: Subscribed to topic '/camera/camera/infra2/metadata'
[INFO] [1698175888.107652084] [rosbag2_recorder]: Subscribed to topic '/camera/camera/infra2/camera_info'
[INFO] [1698175888.108310735] [rosbag2_recorder]: Subscribed to topic '/camera/camera/depth/camera_info'
[INFO] [1698175888.108822687] [rosbag2_recorder]: Subscribed to topic '/camera/camera/infra1/metadata'
[INFO] [1698175888.109590832] [rosbag2_recorder]: Subscribed to topic '/camera/camera/depth/metadata'
[INFO] [1698175888.110120543] [rosbag2_recorder]: Subscribed to topic '/camera/camera/infra1/image_rect_raw'
[INFO] [1698175888.110656426] [rosbag2_recorder]: Subscribed to topic '/camera/camera/extrinsics/depth_to_color'
[INFO] [1698176112.322939531] [rosbag2_cpp]: Writing remaining messages from cache to the bag. It may take a while
[INFO] [1698176112.325282159] [rosbag2_recorder]: Event publisher thread: Exiting
[INFO] [1698176112.325686133] [rosbag2_recorder]: Recording stopped
[INFO] [1698176112.333185738] [rosbag2_recorder]: Recording stopped

and

(base) mona@mona-ThinkStation-P7:~/realsense-ros/rosbag2_2023_10_24-15_31_27$ ls
total 6.8G
drwxrwxr-x 12 mona mona 4.0K Oct 24 15:31 ..
-rw-r--r--  1 mona mona 6.8G Oct 24 15:35 rosbag2_2023_10_24-15_31_27_0.db3
drwxrwxr-x  2 mona mona 4.0K Oct 24 15:35 .
-rw-rw-r--  1 mona mona  15K Oct 24 15:35 metadata.yaml

and

(base) mona@mona-ThinkStation-P7:~/realsense-ros$ ros2 bag play  rosbag2_2023_10_24-15_31_27/rosbag2_2023_10_24-15_31_27_0.db3  --topics /camera/camera/color/image_raw
2023-10-24 15:37:13.061 [RTPS_TRANSPORT_SHM Error] Failed init_port fastrtps_port7417: open_and_lock_file failed -> Function open_port_internal

closing.

closing.
[INFO] [1698176233.069976756] [rosbag2_storage]: Opened database 'rosbag2_2023_10_24-15_31_27/rosbag2_2023_10_24-15_31_27_0.db3' for READ_ONLY.
[INFO] [1698176233.120592540] [rosbag2_player]: Set rate to 1
[INFO] [1698176233.123598009] [rosbag2_player]: Adding keyboard callbacks.
[INFO] [1698176233.123629086] [rosbag2_player]: Press SPACE for Pause/Resume
[INFO] [1698176233.123636591] [rosbag2_player]: Press CURSOR_RIGHT for Play Next Message
[INFO] [1698176233.123642857] [rosbag2_player]: Press CURSOR_UP for Increase Rate 10%
[INFO] [1698176233.123648335] [rosbag2_player]: Press CURSOR_DOWN for Decrease Rate 10%

closing.

closing.
[INFO] [1698176233.124033920] [rosbag2_storage]: Opened database 'rosbag2_2023_10_24-15_31_27/rosbag2_2023_10_24-15_31_27_0.db3' for READ_ONLY.

Here, I read that rosbag2video package/git repo only works for foxy. So what is the solution for humble?

@MartyG-RealSense
Copy link
Collaborator

Hi @monajalal The master branch of the rosbag2video repository now states support for Humble.

https://github.com/mlaiacker/rosbag2video

@monajalal
Copy link
Author

@MartyG-RealSense
Thank you so much for pointing this. I think I may have been confused since I noticed this on their README. I will give it a try today and update here accordingly.
image

@monajalal
Copy link
Author

monajalal commented Nov 1, 2023

@MartyG-RealSense have it recorded it wrongly? I only have a db3 file not a bag file.

(base) mona@ada:~/rosbag2video$ ls ~/realsense-ros/rosbag2_2023_10_24-15_31_27/
total 6.8G
-rw-r--r--  1 mona mona 6.8G Oct 24 15:35 rosbag2_2023_10_24-15_31_27_0.db3
drwxrwxr-x  2 mona mona 4.0K Oct 24 15:35 .
-rw-rw-r--  1 mona mona  15K Oct 24 15:35 metadata.yaml
drwxrwxr-x 12 mona mona 4.0K Nov  1 08:59 ..

These were my steps (please correct me if wrong -- I am new to ros2)

1. In one terminal type: (base) mona@ada:~/realsense-ros$ ros2 run realsense2_camera realsense2_camera_node > run.txt
2. In another terminal type: (base) mona@ada:~/realsense-ros$ ros2 launch realsense2_camera rs_launch.py
3. Now in a third terminal type: (base) mona@ada:~/realsense-ros$ rviz2 rviz2

You should do these steps after successfully installing realsense-viewer and being able to run it to make sure you have an RGB feed from your camera.

After step 3, you should see something like the following in RVIZ.
rviz_output

(base) mona@mona-ThinkStation-P7:~/realsense-ros$ ros2 bag record -a (records all data)

^^^ actually I am only interested in depth align depth along with RGB bags (no other bags) similar to NVlabs/BundleSDF#82 (comment)

Please let me know if further info may be needed.

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Nov 1, 2023

You do not need to launch with ros2 run and ros2 launch in separate terminals. One or the other methods should be used but not both at the same time.

I would recommend testing with the ros2 launch command in the first terminal with align_depth enabled and run rviz2 in a second terminal.

ros2 launch realsense2_camera rs_launch.py align_depth.enable:=true

Then after launch of the camera has completed, record all currently published ROS topics to a bag file by using ros2 bag record -a, which I see that you have already done.

Once you have the bag file then you can use it with rosbag2video to convert the bag file to a video.

@monajalal
Copy link
Author

@MartyG-RealSense thanks a lot for your response. I followed your instructions and

  1. first window:
    ros2 launch realsense2_camera rs_launch.py align_depth.enable:=true
  2. second window:
    rviz2 rviz2

and
in first window I see these messages:

(base) mona@ada:~$ ros2 launch realsense2_camera rs_launch.py align_depth.enable:=true
[INFO] [launch]: All log files can be found below /home/mona/.ros/log/2023-11-01-15-24-33-551883-ada-1924496
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [realsense2_camera_node-1]: process started with pid [1924497]
[realsense2_camera_node-1] [INFO] [1698866673.949945011] [camera.camera]: RealSense ROS v4.54.1
[realsense2_camera_node-1] [INFO] [1698866673.950102844] [camera.camera]: Built with LibRealSense v2.54.2
[realsense2_camera_node-1] [INFO] [1698866673.950121641] [camera.camera]: Running with LibRealSense v2.54.1
[realsense2_camera_node-1] [WARN] [1698866673.950132948] [camera.camera]: ***************************************************
[realsense2_camera_node-1] [WARN] [1698866673.950138443] [camera.camera]: ** running with a different librealsense version **
[realsense2_camera_node-1] [WARN] [1698866673.950146472] [camera.camera]: ** than the one the wrapper was compiled with!   **
[realsense2_camera_node-1] [WARN] [1698866673.950152112] [camera.camera]: ***************************************************
[realsense2_camera_node-1] [INFO] [1698866673.965290836] [camera.camera]: Device with serial number 819112070232 was found.
[realsense2_camera_node-1] 
[realsense2_camera_node-1] [INFO] [1698866673.965382115] [camera.camera]: Device with physical ID /sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video0 was found.
[realsense2_camera_node-1] [INFO] [1698866673.965392536] [camera.camera]: Device with name Intel RealSense D435 was found.
[realsense2_camera_node-1] [INFO] [1698866673.965554808] [camera.camera]: Device with port number 1-8.4 was found.
[realsense2_camera_node-1] [INFO] [1698866673.965565958] [camera.camera]: Device USB type: 2.1
[realsense2_camera_node-1] [WARN] [1698866673.965572848] [camera.camera]: Device 819112070232 is connected using a 2.1 port. Reduced performance is expected.
[realsense2_camera_node-1] [INFO] [1698866673.966459784] [camera.camera]: getParameters...
[realsense2_camera_node-1] [INFO] [1698866673.966638731] [camera.camera]: JSON file is not provided
[realsense2_camera_node-1] [INFO] [1698866673.966652492] [camera.camera]: Device Name: Intel RealSense D435
[realsense2_camera_node-1] [INFO] [1698866673.966661684] [camera.camera]: Device Serial No: 819112070232
[realsense2_camera_node-1] [INFO] [1698866673.966669633] [camera.camera]: Device physical port: /sys/devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8.4/1-8.4:1.0/video4linux/video0
[realsense2_camera_node-1] [INFO] [1698866673.966678443] [camera.camera]: Device FW version: 5.15.1
[realsense2_camera_node-1] [INFO] [1698866673.966687773] [camera.camera]: Device Product ID: 0x0B07
[realsense2_camera_node-1] [INFO] [1698866673.966693332] [camera.camera]: Sync Mode: Off
[realsense2_camera_node-1] [INFO] [1698866673.982458639] [camera.camera]: Set ROS param depth_module.profile to default: 640x480x15
[realsense2_camera_node-1] [INFO] [1698866673.986495372] [camera.camera]: Set ROS param rgb_camera.profile to default: 640x480x15
[realsense2_camera_node-1] [INFO] [1698866673.993765463] [camera.camera]: Stopping Sensor: Depth Module
[realsense2_camera_node-1] [INFO] [1698866674.006475223] [camera.camera]: Starting Sensor: Depth Module
[realsense2_camera_node-1] [INFO] [1698866674.032475486] [camera.camera]: Open profile: stream_type: Depth(0), Format: Z16, Width: 640, Height: 480, FPS: 15
[realsense2_camera_node-1] [INFO] [1698866674.032877896] [camera.camera]: Stopping Sensor: RGB Camera
[realsense2_camera_node-1] [INFO] [1698866674.045941175] [camera.camera]: Starting Sensor: RGB Camera
[realsense2_camera_node-1] [INFO] [1698866674.050354105] [camera.camera]: Open profile: stream_type: Color(0), Format: RGB8, Width: 640, Height: 480, FPS: 15
[realsense2_camera_node-1] [INFO] [1698866674.052112680] [camera.camera]: RealSense Node Is Up!
[realsense2_camera_node-1] [WARN] [1698866674.257211843] [camera.camera]: 

and in second window I see these messages:

(base) mona@ada:~$ rviz2 rviz2
[INFO] [1698866712.071019974] [rviz2]: Stereo is NOT SUPPORTED
[INFO] [1698866712.071174406] [rviz2]: OpenGl version: 4.6 (GLSL 4.6)
[INFO] [1698866712.104732644] [rviz2]: Stereo is NOT SUPPORTED
[INFO] [1698866765.915047184] [rviz2]: Stereo is NOT SUPPORTED
[INFO] [1698866788.661620579] [rviz2]: Stereo is NOT SUPPORTED
[INFO] [1698866797.036233767] [rviz2]: Stereo is NOT SUPPORTED

and finally, I get an error in rviz and cannot visualize the image:
Screenshot from 2023-11-01 15-26-52
Screenshot from 2023-11-01 15-27-07

as you see below, the camera is connected:

(base) mona@ada:~$ lsusb
Bus 002 Device 003: ID 2109:0813 VIA Labs, Inc. VL813 Hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 036: ID 8086:0b07 Intel Corp. RealSense D435
Bus 001 Device 091: ID 2109:2813 VIA Labs, Inc. VL813 Hub
Bus 001 Device 007: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 004: ID 8087:0032 Intel Corp. AX210 Bluetooth
Bus 001 Device 003: ID 17aa:1056 Generic USB Audio
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

@monajalal
Copy link
Author

actually please scratch that I am also not able to get a feed from the realsense-viewer it has happened a few times before that we transiently don't get a feed from realsense-viewer

Screenshot from 2023-11-01 15-31-56

I'll spend some time trying to figure what's going on and then I'll update.

@monajalal
Copy link
Author

monajalal commented Nov 1, 2023

@MartyG-RealSense
so I restarted the machine to free resources. was able to confirm that realsense camera works using realsense-viewer and then when I ran the command rziv2 rviz2 I first see this window which shows the error for Global Status: Error --> Fixed Frame: Frame [map] does not exist. Also, when I add the image using the add button nothing is shown.

Do you know what's happening?
Screenshot from 2023-11-01 15-41-55
Screenshot from 2023-11-01 15-42-05
Screenshot from 2023-11-01 15-43-59

Here's how I ran them:
Screenshot from 2023-11-01 15-46-29

^^^ update, I was choosing the image not from the By topic now it works. Sorry about confusion.
Not sure why I get the Global Status: Error. Should I ignore it?

Screenshot from 2023-11-01 15-48-51

here's the image:
Screenshot from 2023-11-01 15-50-30

@monajalal
Copy link
Author

monajalal commented Nov 1, 2023

@MartyG-RealSense
I am not sure how to use ros2bag2video.py file for a ros2 bag created using ros 2 humble. If I don't use -t flag it tells me I have to use topic name and when I select one of the topics from the metadata.yaml file of the recorded bag file, I get this error. What's your opinion?

(base) mona@ada:~/rosbag2video$ /usr/bin/python3.10 ros2bag2video.py -t /camera/camera/extrinsics/depth_to_color ~/rosbag2_2023_11_01-15_52_42/
FPS (int) =  25
Rate (float) =  1.0
Topic (str) =  /camera/camera/extrinsics/depth_to_color
Output File (str) =  output.mp4
Verbose (bool) =  False
Reading info from bag file: /home/mona/rosbag2_2023_11_01-15_52_42/
ROS Message name =  realsense2_camera_msgs/msg/Extrinsics
Image count =  1
msgtype =  None
AJB: subscribing to msg:  None on topic:  /camera/camera/extrinsics/depth_to_color
Traceback (most recent call last):
  File "/home/mona/rosbag2video/ros2bag2video.py", line 430, in <module>
    main(sys.argv)
  File "/home/mona/rosbag2video/ros2bag2video.py", line 420, in main
    videowriter = RosVideoWriter(args)
  File "/home/mona/rosbag2video/ros2bag2video.py", line 142, in __init__
    self.subscription = self.create_subscription(
  File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/node.py", line 1367, in create_subscription
    check_is_valid_msg_type(msg_type)
  File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/type_support.py", line 35, in check_is_valid_msg_type
    check_for_type_support(msg_type)
  File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/type_support.py", line 20, in check_for_type_support
    ts = msg_or_srv_type.__class__._TYPE_SUPPORT
AttributeError: type object 'NoneType' has no attribute '_TYPE_SUPPORT' This might be a ROS 1 message type but it should be a ROS 2 message type. Make sure to source your ROS 2 workspace after your ROS 1 workspace.

and with no topic, I get this error:

(base) mona@ada:~/rosbag2video$ /usr/bin/python3.10 ros2bag2video.py ~/rosbag2_2023_11_01-15_52_42/
FPS (int) =  25
Rate (float) =  1.0
Topic (str) =  
Output File (str) =  output.mp4
Verbose (bool) =  False
Reading info from bag file: /home/mona/rosbag2_2023_11_01-15_52_42/
ROS Message name =  geometry_msgs/msg/PointStamped
Image count =  0
msgtype =  None
AJB: subscribing to msg:  None on topic:  
Traceback (most recent call last):
  File "/home/mona/rosbag2video/ros2bag2video.py", line 430, in <module>
    main(sys.argv)
  File "/home/mona/rosbag2video/ros2bag2video.py", line 420, in main
    videowriter = RosVideoWriter(args)
  File "/home/mona/rosbag2video/ros2bag2video.py", line 142, in __init__
    self.subscription = self.create_subscription(
  File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/node.py", line 1356, in create_subscription
    raise ex from None
  File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/node.py", line 1354, in create_subscription
    self._validate_topic_or_service_name(topic)
  File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/node.py", line 1183, in _validate_topic_or_service_name
    validate_topic_name(topic_or_service_name, is_service=is_service)
  File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/validate_topic_name.py", line 43, in validate_topic_name
    raise InvalidTopicNameException(name, error_msg, invalid_index)
rclpy.exceptions.InvalidTopicNameException: Invalid topic name: topic name must not be empty string:
  ''
   ^

I also have:

$ ls ~/rosbag2_2023_11_01-15_52_42/
total 2.7G
-rw-r--r--  1 mona mona 2.7G Nov  1 15:54 rosbag2_2023_11_01-15_52_42_0.db3
-rw-rw-r--  1 mona mona  10K Nov  1 15:54 metadata.yaml
drwxr-x--- 56 mona mona 4.0K Nov  1 16:00 ..
drwxrwxr-x  2 mona mona 4.0K Nov  1 16:00 .

and here's the metadata.yaml file content:

rosbag2_bagfile_information:
  version: 5
  storage_identifier: sqlite3
  duration:
    nanoseconds: 86343663847
  starting_time:
    nanoseconds_since_epoch: 1698868362240195604
  message_count: 10373
  topics_with_message_count:
    - topic_metadata:
        name: /goal_pose
        type: geometry_msgs/msg/PoseStamped
        serialization_format: cdr
        offered_qos_profiles: "- history: 3\n  depth: 0\n  reliability: 1\n  durability: 2\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 9223372036\n    nsec: 854775807\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false"
      message_count: 0
    - topic_metadata:
        name: /camera/camera/extrinsics/depth_to_color
        type: realsense2_camera_msgs/msg/Extrinsics
        serialization_format: cdr
        offered_qos_profiles: "- history: 3\n  depth: 0\n  reliability: 1\n  durability: 1\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 9223372036\n    nsec: 854775807\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false"
      message_count: 1
    - topic_metadata:
        name: /events/write_split
        type: rosbag2_interfaces/msg/WriteSplitEvent
        serialization_format: cdr
        offered_qos_profiles: "- history: 3\n  depth: 0\n  reliability: 1\n  durability: 2\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 9223372036\n    nsec: 854775807\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false"
      message_count: 0
    - topic_metadata:
        name: /camera/camera/color/camera_info
        type: sensor_msgs/msg/CameraInfo
        serialization_format: cdr
        offered_qos_profiles: "- history: 3\n  depth: 0\n  reliability: 1\n  durability: 2\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 9223372036\n    nsec: 854775807\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false"
      message_count: 1295
    - topic_metadata:
        name: /tf_static
        type: tf2_msgs/msg/TFMessage
        serialization_format: cdr
        offered_qos_profiles: "- history: 3\n  depth: 0\n  reliability: 1\n  durability: 1\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 9223372036\n    nsec: 854775807\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false"
      message_count: 1
    - topic_metadata:
        name: /camera/camera/aligned_depth_to_color/camera_info
        type: sensor_msgs/msg/CameraInfo
        serialization_format: cdr
        offered_qos_profiles: "- history: 3\n  depth: 0\n  reliability: 1\n  durability: 2\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 9223372036\n    nsec: 854775807\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false"
      message_count: 1295
    - topic_metadata:
        name: /camera/camera/color/metadata
        type: realsense2_camera_msgs/msg/Metadata
        serialization_format: cdr
        offered_qos_profiles: "- history: 3\n  depth: 0\n  reliability: 1\n  durability: 2\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 9223372036\n    nsec: 854775807\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false"
      message_count: 1295
    - topic_metadata:
        name: /camera/camera/depth/image_rect_raw
        type: sensor_msgs/msg/Image
        serialization_format: cdr
        offered_qos_profiles: "- history: 3\n  depth: 0\n  reliability: 1\n  durability: 1\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 9223372036\n    nsec: 854775807\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false"
      message_count: 1295
    - topic_metadata:
        name: /rosout
        type: rcl_interfaces/msg/Log
        serialization_format: cdr
        offered_qos_profiles: "- history: 3\n  depth: 0\n  reliability: 1\n  durability: 1\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 10\n    nsec: 0\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false\n- history: 3\n  depth: 0\n  reliability: 1\n  durability: 1\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 10\n    nsec: 0\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false\n- history: 3\n  depth: 0\n  reliability: 1\n  durability: 1\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 10\n    nsec: 0\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false\n- history: 3\n  depth: 0\n  reliability: 1\n  durability: 1\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 10\n    nsec: 0\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false"
      message_count: 10
    - topic_metadata:
        name: /parameter_events
        type: rcl_interfaces/msg/ParameterEvent
        serialization_format: cdr
        offered_qos_profiles: "- history: 3\n  depth: 0\n  reliability: 1\n  durability: 2\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 9223372036\n    nsec: 854775807\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false\n- history: 3\n  depth: 0\n  reliability: 1\n  durability: 2\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 9223372036\n    nsec: 854775807\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false"
      message_count: 0
    - topic_metadata:
        name: /camera/camera/depth/camera_info
        type: sensor_msgs/msg/CameraInfo
        serialization_format: cdr
        offered_qos_profiles: "- history: 3\n  depth: 0\n  reliability: 1\n  durability: 2\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 9223372036\n    nsec: 854775807\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false"
      message_count: 1295
    - topic_metadata:
        name: /initialpose
        type: geometry_msgs/msg/PoseWithCovarianceStamped
        serialization_format: cdr
        offered_qos_profiles: "- history: 3\n  depth: 0\n  reliability: 1\n  durability: 2\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 9223372036\n    nsec: 854775807\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false"
      message_count: 0
    - topic_metadata:
        name: /camera/camera/color/image_raw
        type: sensor_msgs/msg/Image
        serialization_format: cdr
        offered_qos_profiles: "- history: 3\n  depth: 0\n  reliability: 1\n  durability: 1\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 9223372036\n    nsec: 854775807\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false"
      message_count: 1295
    - topic_metadata:
        name: /camera/camera/aligned_depth_to_color/image_raw
        type: sensor_msgs/msg/Image
        serialization_format: cdr
        offered_qos_profiles: "- history: 3\n  depth: 0\n  reliability: 1\n  durability: 1\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 9223372036\n    nsec: 854775807\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false"
      message_count: 1295
    - topic_metadata:
        name: /camera/camera/depth/metadata
        type: realsense2_camera_msgs/msg/Metadata
        serialization_format: cdr
        offered_qos_profiles: "- history: 3\n  depth: 0\n  reliability: 1\n  durability: 2\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 9223372036\n    nsec: 854775807\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false"
      message_count: 1295
    - topic_metadata:
        name: /camera/camera/extrinsics/depth_to_depth
        type: realsense2_camera_msgs/msg/Extrinsics
        serialization_format: cdr
        offered_qos_profiles: "- history: 3\n  depth: 0\n  reliability: 1\n  durability: 1\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 9223372036\n    nsec: 854775807\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false"
      message_count: 1
    - topic_metadata:
        name: /clicked_point
        type: geometry_msgs/msg/PointStamped
        serialization_format: cdr
        offered_qos_profiles: "- history: 3\n  depth: 0\n  reliability: 1\n  durability: 2\n  deadline:\n    sec: 9223372036\n    nsec: 854775807\n  lifespan:\n    sec: 9223372036\n    nsec: 854775807\n  liveliness: 1\n  liveliness_lease_duration:\n    sec: 9223372036\n    nsec: 854775807\n  avoid_ros_namespace_conventions: false"
      message_count: 0
  compression_format: ""
  compression_mode: ""
  relative_file_paths:
    - rosbag2_2023_11_01-15_52_42_0.db3
  files:
    - path: rosbag2_2023_11_01-15_52_42_0.db3
      starting_time:
        nanoseconds_since_epoch: 1698868362240195604
      duration:
        nanoseconds: 86343663847
      message_count: 10373

@monajalal
Copy link
Author

monajalal commented Nov 1, 2023

@MartyG-RealSense I think I was selecting the wrong topic, I may still be. I selected a different topic from the metadata.yaml but it seems the source is empty. I did capture for a few seconds. Is there a way to check sanity of recorded before converting or why do think I am getting this error?

(base) mona@ada:~/rosbag2video$ ./ros2bag2video.py -t /camera/camera/depth/image_rect_raw ~/rosbag2_2023_11_01-15_52_42/
FPS (int) =  25
Rate (float) =  1.0
Topic (str) =  /camera/camera/depth/image_rect_raw
Output File (str) =  output.mp4
Verbose (bool) =  False
Reading info from bag file: /home/mona/rosbag2_2023_11_01-15_52_42/
ROS Message name =  sensor_msgs/msg/Image
Image count =  1295
msgtype =  <class 'sensor_msgs.msg._image.Image'>
AJB: subscribing to msg:  <class 'sensor_msgs.msg._image.Image'> on topic:  /camera/camera/depth/image_rect_raw
Starting ROS bag playback...
[INFO] [1698869926.735708955] [ros2bag2videos]: Image Received [1/1295]
Traceback (most recent call last):
  File "/opt/ros/humble/local/lib/python3.10/dist-packages/cv_bridge/core.py", line 141, in compressed_imgmsg_to_cv2
    res = cvtColor2(im, 'bgr8', desired_encoding)
RuntimeError: OpenCV(4.5.4) ./modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mona/rosbag2video/./ros2bag2video.py", line 430, in <module>
    main(sys.argv)
  File "/home/mona/rosbag2video/./ros2bag2video.py", line 422, in main
    rclpy.spin(videowriter)
  File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/__init__.py", line 222, in spin
    executor.spin_once()
  File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 713, in spin_once
    raise handler.exception()
  File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/task.py", line 239, in __call__
    self._handler.send(None)
  File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 418, in handler
    await call_coroutine(entity, arg)
  File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 343, in _execute_subscription
    await await_or_execute(sub.callback, msg)
  File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 107, in await_or_execute
    return callback(*args)
  File "/home/mona/rosbag2video/./ros2bag2video.py", line 355, in listener_callback
    img = self.bridge.compressed_imgmsg_to_cv2(msg, self.msg_fmt)
  File "/opt/ros/humble/local/lib/python3.10/dist-packages/cv_bridge/core.py", line 143, in compressed_imgmsg_to_cv2
    raise CvBridgeError(e)
cv_bridge.core.CvBridgeError: OpenCV(4.5.4) ./modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

[INFO] [1698869926.878085902] [rosbag2_storage]: Opened database '/home/mona/rosbag2_2023_11_01-15_52_42/rosbag2_2023_11_01-15_52_42_0.db3' for READ_ONLY.
[INFO] [1698869926.878163020] [rosbag2_player]: Set rate to 1
[INFO] [1698869926.880363152] [rosbag2_player]: Adding keyboard callbacks.
[INFO] [1698869926.880403684] [rosbag2_player]: Press SPACE for Pause/Resume
[INFO] [1698869926.880415358] [rosbag2_player]: Press CURSOR_RIGHT for Play Next Message
[INFO] [1698869926.880424862] [rosbag2_player]: Press CURSOR_UP for Increase Rate 10%
[INFO] [1698869926.880434734] [rosbag2_player]: Press CURSOR_DOWN for Decrease Rate 10%
[INFO] [1698869926.880854251] [rosbag2_storage]: Opened database '/home/mona/rosbag2_2023_11_01-15_52_42/rosbag2_2023_11_01-15_52_42_0.db3' for READ_ONLY.
(base) mona@ada:~/rosbag2video$ Caught exception: "Error in tcsetattr old_term_settings. errno = 5"

I have:

(base) mona@ada:~$ ros2 bag info rosbag2_2023_11_01-15_52_42/

Files:             rosbag2_2023_11_01-15_52_42_0.db3
Bag size:          2.6 GiB
Storage id:        sqlite3
Duration:          86.343s
Start:             Nov  1 2023 15:52:42.240 (1698868362.240)
End:               Nov  1 2023 15:54:08.583 (1698868448.583)
Messages:          10373
Topic information: Topic: /goal_pose | Type: geometry_msgs/msg/PoseStamped | Count: 0 | Serialization Format: cdr
                   Topic: /camera/camera/extrinsics/depth_to_color | Type: realsense2_camera_msgs/msg/Extrinsics | Count: 1 | Serialization Format: cdr
                   Topic: /events/write_split | Type: rosbag2_interfaces/msg/WriteSplitEvent | Count: 0 | Serialization Format: cdr
                   Topic: /camera/camera/color/camera_info | Type: sensor_msgs/msg/CameraInfo | Count: 1295 | Serialization Format: cdr
                   Topic: /tf_static | Type: tf2_msgs/msg/TFMessage | Count: 1 | Serialization Format: cdr
                   Topic: /camera/camera/aligned_depth_to_color/camera_info | Type: sensor_msgs/msg/CameraInfo | Count: 1295 | Serialization Format: cdr
                   Topic: /camera/camera/color/metadata | Type: realsense2_camera_msgs/msg/Metadata | Count: 1295 | Serialization Format: cdr
                   Topic: /camera/camera/depth/image_rect_raw | Type: sensor_msgs/msg/Image | Count: 1295 | Serialization Format: cdr
                   Topic: /rosout | Type: rcl_interfaces/msg/Log | Count: 10 | Serialization Format: cdr
                   Topic: /parameter_events | Type: rcl_interfaces/msg/ParameterEvent | Count: 0 | Serialization Format: cdr
                   Topic: /camera/camera/depth/camera_info | Type: sensor_msgs/msg/CameraInfo | Count: 1295 | Serialization Format: cdr
                   Topic: /initialpose | Type: geometry_msgs/msg/PoseWithCovarianceStamped | Count: 0 | Serialization Format: cdr
                   Topic: /camera/camera/color/image_raw | Type: sensor_msgs/msg/Image | Count: 1295 | Serialization Format: cdr
                   Topic: /camera/camera/aligned_depth_to_color/image_raw | Type: sensor_msgs/msg/Image | Count: 1295 | Serialization Format: cdr
                   Topic: /camera/camera/depth/metadata | Type: realsense2_camera_msgs/msg/Metadata | Count: 1295 | Serialization Format: cdr
                   Topic: /camera/camera/extrinsics/depth_to_depth | Type: realsense2_camera_msgs/msg/Extrinsics | Count: 1 | Serialization Format: cdr
                   Topic: /clicked_point | Type: geometry_msgs/msg/PointStamped | Count: 0 | Serialization Format: cdr

@MartyG-RealSense
Copy link
Collaborator

In regard to the global status error, what happens if you change Fixed Frame from 'map' to camera_link please?

In regard to your other questions, I do not have information about how a rosbag recording could be sanity checked and are not familiar with how rosbag2video works. I do apologize.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants