ros2 bag record cmd - Invalid choice mcap in --storage flag

[sagocz]

Description

I want to use .mcap format in ros2 bag record. I'm using the newest humble version.

Expected Behavior

ros2 bag record cmd with .mcap format writing.

Actual Behavior

ros2 bag record: error: argument -s/--storage: invalid choice: 'mcap' (choose from 'my_test_plugin', 'sqlite3')

System (please complete the following information)

• OS: Ubuntu Jammy
• ROS 2 Distro: Humble
• Version: latest

[MichaelOrlov]

It looks like the rosbag2_storage_mcap package which is mcap storage plugin hasn't been built or is absent or hasn't been sourced.

[sagocz]

@MichaelOrlov
It's ok. But I also tried to pull the newest docker image from osrf. -> humble image
And when I run it, I sourced to . /opt/ros/humble/setup.bash and next type cmd ros2 bag record -h.
I think there may be missing installation of specific packages from lack of dependencies in the sources. Going to the /opt/ros/humble/share directory, packages regarding .mcap are missing. On the other hand, downloading the iron version everything works as expected and you can record bags directly from ros2 bag record. What I find strange about this behavior is that when I go to branch humble in the ros2/rosbag2 repo, I see that in version 0.15.8 there are packages that support .mcap recording, while they cannot be used from the cmd command ros2 bag record -s mcap.

root@1f08929c461d:/# ros2 pkg list
action_msgs
action_tutorials_cpp
action_tutorials_interfaces
action_tutorials_py
actionlib_msgs
actuator_msgs
ament_cmake
ament_cmake_auto
ament_cmake_copyright
ament_cmake_core
ament_cmake_cppcheck
ament_cmake_cpplint
ament_cmake_export_definitions
ament_cmake_export_dependencies
ament_cmake_export_include_directories
ament_cmake_export_interfaces
ament_cmake_export_libraries
ament_cmake_export_link_flags
ament_cmake_export_targets
ament_cmake_flake8
ament_cmake_gen_version_h
ament_cmake_gmock
ament_cmake_gtest
ament_cmake_include_directories
ament_cmake_libraries
ament_cmake_lint_cmake
ament_cmake_pep257
ament_cmake_pytest
ament_cmake_python
ament_cmake_ros
ament_cmake_target_dependencies
ament_cmake_test
ament_cmake_uncrustify
ament_cmake_version
ament_cmake_xmllint
ament_copyright
ament_cppcheck
ament_cpplint
ament_flake8
ament_index_cpp
ament_index_python
ament_lint
ament_lint_auto
ament_lint_cmake
ament_lint_common
ament_package
ament_pep257
ament_uncrustify
ament_xmllint
angles
builtin_interfaces
camera_calibration
camera_calibration_parsers
camera_info_manager
class_loader
common_interfaces
composition
composition_interfaces
compressed_depth_image_transport
compressed_image_transport
console_bridge_vendor
cv_bridge
demo_nodes_cpp
demo_nodes_cpp_native
demo_nodes_py
depth_image_proc
depthimage_to_laserscan
desktop
desktop_full
diagnostic_msgs
domain_coordinator
dummy_map_server
dummy_robot_bringup
dummy_sensors
eigen3_cmake_module
example_interfaces
examples_rclcpp_minimal_action_client
examples_rclcpp_minimal_action_server
examples_rclcpp_minimal_client
examples_rclcpp_minimal_composition
examples_rclcpp_minimal_publisher
examples_rclcpp_minimal_service
examples_rclcpp_minimal_subscriber
examples_rclcpp_minimal_timer
examples_rclcpp_multithreaded_executor
examples_rclpy_executors
examples_rclpy_minimal_action_client
examples_rclpy_minimal_action_server
examples_rclpy_minimal_client
examples_rclpy_minimal_publisher
examples_rclpy_minimal_service
examples_rclpy_minimal_subscriber
fastrtps_cmake_module
filters
geometry2
geometry_msgs
image_common
image_geometry
image_pipeline
image_proc
image_publisher
image_rotate
image_tools
image_transport
image_transport_plugins
image_view
interactive_markers
intra_process_demo
joy
kdl_parser
keyboard_handler
laser_filters
laser_geometry
launch
launch_ros
launch_testing
launch_testing_ament_cmake
launch_testing_ros
launch_xml
launch_yaml
libcurl_vendor
libstatistics_collector
libyaml_vendor
lifecycle
lifecycle_msgs
logging_demo
map_msgs
message_filters
nav_msgs
orocos_kdl_vendor
osrf_pycommon
pcl_conversions
pcl_msgs
pcl_ros
pendulum_control
pendulum_msgs
perception
perception_pcl
pluginlib
pybind11_vendor
python_cmake_module
python_qt_binding
qt_dotgraph
qt_gui
qt_gui_cpp
qt_gui_py_common
quality_of_service_demo_cpp
quality_of_service_demo_py
rcl
rcl_action
rcl_interfaces
rcl_lifecycle
rcl_logging_interface
rcl_logging_spdlog
rcl_yaml_param_parser
rclcpp
rclcpp_action
rclcpp_components
rclcpp_lifecycle
rclpy
rcpputils
rcutils
resource_retriever
rmw
rmw_dds_common
rmw_fastrtps_cpp
rmw_fastrtps_shared_cpp
rmw_implementation
rmw_implementation_cmake
robot_state_publisher
ros2action
ros2bag
ros2cli
ros2cli_common_extensions
ros2component
ros2doctor
ros2interface
ros2launch
ros2lifecycle
ros2multicast
ros2node
ros2param
ros2pkg
ros2run
ros2service
ros2topic
ros_base
ros_core
ros_environment
ros_gz_bridge
ros_gz_image
ros_gz_interfaces
ros_gz_sim
ros_gz_sim_demos
ros_ign_bridge
ros_ign_gazebo
ros_ign_gazebo_demos
ros_ign_image
ros_ign_interfaces
ros_workspace
rosbag2
rosbag2_compression
rosbag2_compression_zstd
rosbag2_cpp
rosbag2_interfaces
rosbag2_py
rosbag2_storage
rosbag2_storage_default_plugins
rosbag2_transport
rosgraph_msgs
rosidl_adapter
rosidl_cli
rosidl_cmake
rosidl_default_generators
rosidl_default_runtime
rosidl_generator_c
rosidl_generator_cpp
rosidl_generator_py
rosidl_parser
rosidl_runtime_c
rosidl_runtime_cpp
rosidl_runtime_py
rosidl_typesupport_c
rosidl_typesupport_cpp
rosidl_typesupport_fastrtps_c
rosidl_typesupport_fastrtps_cpp
rosidl_typesupport_interface
rosidl_typesupport_introspection_c
rosidl_typesupport_introspection_cpp
rpyutils
rqt_action
rqt_bag
rqt_bag_plugins
rqt_common_plugins
rqt_console
rqt_graph
rqt_gui
rqt_gui_cpp
rqt_gui_py
rqt_image_view
rqt_msg
rqt_plot
rqt_publisher
rqt_py_common
rqt_py_console
rqt_reconfigure
rqt_service_caller
rqt_shell
rqt_srv
rqt_topic
rttest
rviz2
rviz_assimp_vendor
rviz_common
rviz_default_plugins
rviz_ogre_vendor
rviz_rendering
sdformat_urdf
sdl2_vendor
sensor_msgs
sensor_msgs_py
shape_msgs
shared_queues_vendor
simulation
spdlog_vendor
sqlite3_vendor
sros2
sros2_cmake
statistics_msgs
std_msgs
std_srvs
stereo_image_proc
stereo_msgs
tango_icons_vendor
teleop_twist_joy
teleop_twist_keyboard
tf2
tf2_bullet
tf2_eigen
tf2_eigen_kdl
tf2_geometry_msgs
tf2_kdl
tf2_msgs
tf2_py
tf2_ros
tf2_ros_py
tf2_sensor_msgs
tf2_tools
theora_image_transport
tinyxml2_vendor
tinyxml_vendor
tlsf
tlsf_cpp
topic_monitor
tracetools
tracetools_image_pipeline
trajectory_msgs
turtlesim
uncrustify_vendor
unique_identifier_msgs
urdf
urdf_parser_plugin
vision_opencv
visualization_msgs
xacro
yaml_cpp_vendor
zstd_vendor

[clalancette]

I think there may be missing installation of specific packages from lack of dependencies in the sources. Going to the /opt/ros/humble/share directory, packages regarding .mcap are missing. On the other hand, downloading the iron version everything works as expected and you can record bags directly from ros2 bag record.

This is more-or-less working as designed. In Humble, the default backend for rosbag2 is sqlite3, which is installed above (in Humble, it is part of rosbag2_storage_default_plugins). rosbag2_storage_mcap is not part of the default install. At the end of the day, the variants (core, base, desktop, desktop-full, etc) are a tradeoff between keeping the installation small, and providing functionality.

With all of that said, I see 3 ways forward here:

1. Just accept that mcap is not part of the default install in Humble, and leave it at that. This may require a documentation update specifically for Humble.
2. Add in rosbag2_storage_mcap as a dependency of one of the packages in https://github.com/ros2/rosbag2/tree/humble . That will pull it in for all variants. We'll want to carefully consider what the size impact on each of the variants is before we do this.
3. Add in rosbag2_storage_mcap as a variant dependency in https://github.com/ros2/variants . This has the benefit of being more targeted (we can make it a dependency only for certain variants), but it means that humble is "different" from all of the rest.

Personally, I think we should just stick with 1, with a possible documentation update. MCAP is not the default in Humble.

[sagocz]

@clalancette
Option #2 seems better to me, because of the advantages of the mcap format over sqlite3 (less CPU load when recording, less chance of losing messages when writing). It's a pity you didn't add it for humble vs iron already.
I think that in the next version of ros the mcap format will already be the default or at least an optional form of recording.
At this moment, as I understand using the humble version, I have no possibility in the near future (next month) to record in this format?

[clalancette]

It's a pity you didn't add it for humble vs iron already.

It is added in both of them. It is just not the default for humble. You can always install it with apt-get install ros-humble-rosbag2-storage-mcap.

I think that in the next version of ros the mcap format will already be the default or at least an optional form of recording.

Correct. In both Iron and Rolling, mcap is the default.

At this moment, as I understand using the humble version, I have no possibility in the near future (next month) to record in this format?

No, it works totally fine in Humble. You just need to install it as I mentioned above.