Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot play MCAP files with different topics having different serialization format #1245

Closed
brian386 opened this issue Feb 13, 2023 · 7 comments
Closed

Cannot play MCAP files with different topics having different serialization format #1245

brian386 opened this issue Feb 13, 2023 · 7 comments
Labels
wad Work As Designed wontfix This will not be worked on

Comments

@brian386
Copy link

brian386 commented Feb 13, 2023 •
edited
Loading

Description

** clear and concise description of the bug is **

When I run ros2 bag play bag_name.mcap, I get this error message: Topics with different rwm serialization format have been found. All topics must have the same serialization format.
According to this issue, different topics having different serialization methods is supposed to be supported.

Edit: I also tried playing an mcap file with only protobuf serialization, and I get this error [ERROR] [1676265641.870980365] [rosbag2_cpp]: Could not initialize RMWImplementedConverter: No RMW implementation found supporting serialization format protobuf. Is there any way around this?

Additional context

** Add any other context about the problem here **
I am trying to play the NuScenes dataset as a ROS2 bag. I used the nuscenes2mcap repository to convert NuScenes data to MCAP files. Below is the info about the file. Most of the topics use serialization format protobuf, but /odom and /imu use JSON.

Files:             NuScenes-v1.0-mini-scene-0655.mcap
Bag size:          475.1 MiB
Storage id:        mcap
Duration:          19.844s
Start:             Aug 27 2018 15:51:32.104 (1535385092.104)
End:               Aug 27 2018 15:51:51.949 (1535385111.949)
Messages:          35471
Topic information: Topic: /diagnostics | Type: diagnostic_msgs/DiagnosticArray | Count: 22922 | Serialization Format: ros1
                   Topic: /markers/car | Type: foxglove.SceneUpdate | Count: 41 | Serialization Format: protobuf
                   Topic: /markers/annotations | Type: foxglove.SceneUpdate | Count: 41 | Serialization Format: protobuf
                   Topic: /gps | Type: foxglove.LocationFix | Count: 41 | Serialization Format: protobuf
                   Topic: /pose | Type: foxglove.PoseInFrame | Count: 41 | Serialization Format: protobuf
                   Topic: /CAM_FRONT_LEFT/annotations | Type: foxglove.ImageAnnotations | Count: 41 | Serialization Format: protobuf
                   Topic: /CAM_FRONT_LEFT/lidar | Type: foxglove.ImageAnnotations | Count: 237 | Serialization Format: protobuf
                   Topic: /CAM_FRONT_LEFT/camera_info | Type: foxglove.CameraCalibration | Count: 237 | Serialization Format: protobuf
                   Topic: /CAM_FRONT_LEFT/image_rect_compressed | Type: foxglove.CompressedImage | Count: 237 | Serialization Format: protobuf
                   Topic: /CAM_BACK/annotations | Type: foxglove.ImageAnnotations | Count: 41 | Serialization Format: protobuf
                   Topic: /CAM_BACK/lidar | Type: foxglove.ImageAnnotations | Count: 230 | Serialization Format: protobuf
                   Topic: /CAM_BACK/camera_info | Type: foxglove.CameraCalibration | Count: 230 | Serialization Format: protobuf
                   Topic: /CAM_FRONT/image_rect_compressed | Type: foxglove.CompressedImage | Count: 237 | Serialization Format: protobuf
                   Topic: /LIDAR_TOP | Type: foxglove.PointCloud | Count: 396 | Serialization Format: protobuf
                   Topic: /RADAR_BACK_RIGHT | Type: foxglove.PointCloud | Count: 262 | Serialization Format: protobuf
                   Topic: /RADAR_BACK_LEFT | Type: foxglove.PointCloud | Count: 269 | Serialization Format: protobuf
                   Topic: /RADAR_FRONT_RIGHT | Type: foxglove.PointCloud | Count: 262 | Serialization Format: protobuf
                   Topic: /RADAR_FRONT_LEFT | Type: foxglove.PointCloud | Count: 265 | Serialization Format: protobuf
                   Topic: /RADAR_FRONT | Type: foxglove.PointCloud | Count: 270 | Serialization Format: protobuf
                   Topic: /drivable_area | Type: foxglove.Grid | Count: 41 | Serialization Format: protobuf
                   Topic: /tf | Type: foxglove.FrameTransform | Count: 3177 | Serialization Format: protobuf
                   Topic: /semantic_map | Type: foxglove.SceneUpdate | Count: 1 | Serialization Format: protobuf
                   Topic: /map | Type: foxglove.Grid | Count: 1 | Serialization Format: protobuf
                   Topic: /odom | Type: Pose | Count: 984 | Serialization Format: json
                   Topic: /imu | Type: IMU | Count: 1975 | Serialization Format: json
                   Topic: /CAM_FRONT/camera_info | Type: foxglove.CameraCalibration | Count: 237 | Serialization Format: protobuf
                   Topic: /CAM_FRONT/lidar | Type: foxglove.ImageAnnotations | Count: 237 | Serialization Format: protobuf
                   Topic: /CAM_FRONT/annotations | Type: foxglove.ImageAnnotations | Count: 41 | Serialization Format: protobuf
                   Topic: /CAM_FRONT_RIGHT/image_rect_compressed | Type: foxglove.CompressedImage | Count: 236 | Serialization Format: protobuf
                   Topic: /CAM_FRONT_RIGHT/camera_info | Type: foxglove.CameraCalibration | Count: 236 | Serialization Format: protobuf
                   Topic: /CAM_FRONT_RIGHT/lidar | Type: foxglove.ImageAnnotations | Count: 236 | Serialization Format: protobuf
                   Topic: /CAM_FRONT_RIGHT/annotations | Type: foxglove.ImageAnnotations | Count: 41 | Serialization Format: protobuf
                   Topic: /CAM_BACK_RIGHT/image_rect_compressed | Type: foxglove.CompressedImage | Count: 237 | Serialization Format: protobuf
                   Topic: /CAM_BACK_RIGHT/camera_info | Type: foxglove.CameraCalibration | Count: 237 | Serialization Format: protobuf
                   Topic: /CAM_BACK_RIGHT/lidar | Type: foxglove.ImageAnnotations | Count: 237 | Serialization Format: protobuf
                   Topic: /CAM_BACK_RIGHT/annotations | Type: foxglove.ImageAnnotations | Count: 41 | Serialization Format: protobuf
                   Topic: /CAM_BACK_LEFT/image_rect_compressed | Type: foxglove.CompressedImage | Count: 235 | Serialization Format: protobuf
                   Topic: /CAM_BACK_LEFT/camera_info | Type: foxglove.CameraCalibration | Count: 235 | Serialization Format: protobuf
                   Topic: /CAM_BACK_LEFT/lidar | Type: foxglove.ImageAnnotations | Count: 235 | Serialization Format: protobuf
                   Topic: /CAM_BACK_LEFT/annotations | Type: foxglove.ImageAnnotations | Count: 41 | Serialization Format: protobuf
                   Topic: /CAM_BACK/image_rect_compressed | Type: foxglove.CompressedImage | Count: 230 | Serialization Format: protobuf

When I run ros2 bag play NuScenes-v1.0-mini-scene-0655.mcap, I get Topics with different rwm serialization format have been found. All topics must have the same serialization format.

To Reproduce

** Steps to reproduce the behavior, e.g.

  1. Download the NuScenes dataset here. We need Full dataset (v1.0) Mini, Map expansion and CAN bus expansion
  2. Follow the instructions on to convert the dataset to MCAP. The data directory should be as follows.
.
├── can_bus
├── maps
│   ├── basemap
│   ├── expansion
│   └── prediction
├── samples
│   ├── CAM_BACK
│   ├── CAM_BACK_LEFT
│   ├── CAM_BACK_RIGHT
│   ├── CAM_FRONT
│   ├── CAM_FRONT_LEFT
│   ├── CAM_FRONT_RIGHT
│   ├── LIDAR_TOP
│   ├── RADAR_BACK_LEFT
│   ├── RADAR_BACK_RIGHT
│   ├── RADAR_FRONT
│   ├── RADAR_FRONT_LEFT
│   └── RADAR_FRONT_RIGHT
├── sweeps
│   ├── CAM_BACK
│   ├── CAM_BACK_LEFT
│   ├── CAM_BACK_RIGHT
│   ├── CAM_FRONT
│   ├── CAM_FRONT_LEFT
│   ├── CAM_FRONT_RIGHT
│   ├── LIDAR_TOP
│   ├── RADAR_BACK_LEFT
│   ├── RADAR_BACK_RIGHT
│   ├── RADAR_FRONT
│   ├── RADAR_FRONT_LEFT
│   └── RADAR_FRONT_RIGHT
└── v1.0-mini

The resulting MCAP files should be in output/ after the script has been run
3. Copy the MCAP files to a ROS2 environment and install rosbag2 dependencies with sudo apt-get install ros-rolling-ros2bag ros-rolling-rosbag2*
4. Run ros2 bag play <filename>.mcap

System (please complete the following information)

  • OS: Ubuntu 20.04.4 LTS
  • ROS 2 Distro: rolling
  • Version: release
@brian386 brian386 added the bug Something isn't working label Feb 13, 2023
@emersonknapp
Copy link
Collaborator

emersonknapp commented Feb 14, 2023 •
edited
Loading

The problem here is that rosbag2 has no idea how to understand your data in order to play it back - it doesn't have a deserializer for protobuf data. I'm also noticing that the "type" field is foxglove types, like foxglove.ImageAnnotations, which isn't a ROS 2 message type.

This is kind of a big can of worms, actually!

Beyond expecting that there isn't an error printed - what is the goal you're trying to achieve by running ros2 bag play nuscenes.mcap? Other ros2 nodes subscribing to the data, viewing in RViz, piping through the foxglove bridge to foxglove studio? Something else? The more we understand about what is trying to be accomplished the better we can do to determine where/what functionality is missing.

EDIT: Just want to clarify a point that maybe isn't immediately clear. rosbag2 recording can create a rosbag2-compatible MCAP file that can be played back, but not all MCAP files are data that rosbag2 can use. MCAP is a generic data file format, that happens to be used (now) as a backing store for ros2 recording data.

@brian386
Copy link
Author

Thanks for your response!

My goal is that I have other ros2 nodes that need to subscribe to the data. We are trying to test our autonomous vehicle stack on the nuscenes dataset.

@emersonknapp
Copy link
Collaborator

emersonknapp commented Feb 15, 2023 •
edited
Loading

Yeah, that seems like a useful case. Just packing the nuscenes data into an MCAP file is not going to be sufficient to achieve it, unfortunately. There will need to be some sort of conversion from protobuf data to equivalent ROS 2 message types (like sensor_msgs/msg/PointCloud etc). Either at playback runtime (potentially costly for big bandwidth), or offline in a conversion step.

I'm not sure at the moment what exactly that process would entail, without more research.

There is the rosbag2 SerializationFormatConverter plugin framework, for this purpose, but we haven't actually built any such plugins to date!

Given that it's a limited set of data types, if you have e.g. python code to parse the protobuf message fields, you could write a script that reads them in, constructs ROS 2 messages, and writes them to a bag using rosbag2_py API. That wouldn't be fully general but may meet your needs in the short term.

@Erically
Copy link

Erically commented May 16, 2023 •
edited
Loading

@brian386 I face the same problem,have you resolved it?

@brian386
Copy link
Author

@Erically I made a fork of the nuscenes2mcap repo to convert nuscenes data into ros2bag-playable MCAP files here. Not sure if you had the same use case, but hope this helps!

@Erically
Copy link

@brian386 nice job

@MichaelOrlov MichaelOrlov added wontfix This will not be worked on wad Work As Designed and removed bug Something isn't working labels May 5, 2024
@MichaelOrlov
Copy link
Contributor

  • Closing this issue as wouldn't be fixed. Rosbag2 works as designed. There are general ROS 2 limitations that it is support only one rmw layer during runtime.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
wad Work As Designed wontfix This will not be worked on
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@emersonknapp @MichaelOrlov @Erically @brian386