rosbag2_py reader and writer

[mabelzhang]

Initial reader functionalities for rosbag2_py.

Dependencies:

sudo apt-get install python-pybind11
git clone https://github.com/ros2/pybind11_vendor.git

Example usage is here (minus the lines about StorageFilter which need #302 and Python binding) in the test.

[piraka9011]

Thanks! Not too familiar with pybind but is it possible to have this binding for the general reader as opposed to a specific one? That way, we can specify whatever kind of reader we want in the Python API.

[mabelzhang]

@piraka9011 Thanks for the review! f8c4391 wraps around Reader. The type e.g. "SequentialReader" is passed in as string. Usage example is added in pytest file.

[mabelzhang]

Rerun CI on all platforms:

• Linux
• Linux-aarch64
• macOS
• Windows

[mabelzhang]

I think this is ready if I can get another approval.

[Karsten1987]

I haven't had a change to test it yet, but this looks good to me so far.
But I guess it would be good to have somebody else look over it as well.

[mabelzhang]

@emersonknapp Could you take a last pass on this PR? We need one more approval before merging. Thanks!

[emersonknapp]

This looks really good, thanks for going through all the revisions on it

[nuclearsandwich]

I think this change has introduced a build failure in the nightly clang job: https://ci.ros2.org/view/nightly/job/nightly_linux_clang_libcxx/513

00:47:40.956 In file included from /home/jenkins-agent/workspace/nightly_linux_clang_libcxx/ws/src/ros2/rosbag2/rosbag2_py/src/rosbag2_py/_writer.cpp:15:
00:47:40.956 /home/jenkins-agent/workspace/nightly_linux_clang_libcxx/ws/install/pybind11_vendor/include/pybind11/pybind11.h:1093:9: error: expression result unused [-Werror,-Wunused-value]
00:47:40.956         PYBIND11_EXPAND_SIDE_EFFECTS(add_base<options>(record));
00:47:40.956         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
00:47:40.956 /home/jenkins-agent/workspace/nightly_linux_clang_libcxx/ws/install/pybind11_vendor/include/pybind11/detail/common.h:662:47: note: expanded from macro 'PYBIND11_EXPAND_SIDE_EFFECTS'
00:47:40.956 #define PYBIND11_EXPAND_SIDE_EFFECTS(PATTERN) pybind11::detail::expand_side_effects{ ((PATTERN), void(), false)..., false }
00:47:40.956                                               ^                                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
00:47:40.956 /home/jenkins-agent/workspace/nightly_linux_clang_libcxx/ws/src/ros2/rosbag2/rosbag2_py/src/rosbag2_py/_writer.cpp:84:3: note: in instantiation of function template specialization 'pybind11::class_<rosbag2_py::Writer<rosbag2_cpp::writers::SequentialWriter>>::class_<>' requested here
00:47:40.956   pybind11::class_<rosbag2_py::Writer<rosbag2_cpp::writers::SequentialWriter>>(
00:47:40.956   ^
00:47:40.956 In file included from /home/jenkins-agent/workspace/nightly_linux_clang_libcxx/ws/src/ros2/rosbag2/rosbag2_py/src/rosbag2_py/_writer.cpp:15:
00:47:40.956 /home/jenkins-agent/workspace/nightly_linux_clang_libcxx/ws/install/pybind11_vendor/include/pybind11/pybind11.h:1093:9: error: expression result unused [-Werror,-Wunused-value]
00:47:40.956         PYBIND11_EXPAND_SIDE_EFFECTS(add_base<options>(record));
00:47:40.956         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
00:47:40.956 /home/jenkins-agent/workspace/nightly_linux_clang_libcxx/ws/install/pybind11_vendor/include/pybind11/detail/common.h:662:47: note: expanded from macro 'PYBIND11_EXPAND_SIDE_EFFECTS'
00:47:40.956 #define PYBIND11_EXPAND_SIDE_EFFECTS(PATTERN) pybind11::detail::expand_side_effects{ ((PATTERN), void(), false)..., false }
00:47:40.956                                               ^                                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
00:47:40.956 /home/jenkins-agent/workspace/nightly_linux_clang_libcxx/ws/src/ros2/rosbag2/rosbag2_py/src/rosbag2_py/_writer.cpp:92:3: note: in instantiation of function template specialization 'pybind11::class_<rosbag2_py::Writer<rosbag2_compression::SequentialCompressionWriter>>::class_<>' requested here
00:47:40.956   pybind11::class_<rosbag2_py::Writer<rosbag2_compression::SequentialCompressionWriter>>(
00:47:40.956   ^
00:47:40.956 2 errors generated.
00:47:40.956 make[2]: *** [CMakeFiles/_writer.dir/build.make:63: CMakeFiles/_writer.dir/src/rosbag2_py/_writer.cpp.o] Error 1
00:47:40.956 make[1]: *** [CMakeFiles/Makefile2:82: CMakeFiles/_writer.dir/all] Error 2
00:47:40.956 make: *** [Makefile:141: all] Error 2

[nuclearsandwich]

In addition to the compilation failure mentioned above there also appear to be issues on Windows Debug which I've opened #504 to track.

[evclark]

Noob question, but how do I actually use rosbag2_py? I have installed it as per the instructions on the rosbag2 README,

$ export CHOOSE_ROS_DISTRO=crystal # rosbag2 is available starting from crystal
$ sudo apt-get install ros-$CHOOSE_ROS_DISTRO-ros2bag ros-$CHOOSE_ROS_DISTRO-rosbag2-transport

but I do not see it in ros2 pkg list and when I try to import it in python I get an error ModuleNotFoundError: No module named 'rosbag2_py'

Running on Ubuntu 20.04 on Foxy.

Thanks in advance!

[mabelzhang]

Hi @evclark, for future reference, please post questions as a new Issue or on ROS Answers, as opposed to commenting on a merged PR.

First, you are using Foxy, but you installed the packages for crystal? That does not look right. Your distro should be foxy.
Second, you should install ros-$CHOOSE_ROS_DISTRO-rosbag2, which is the meta package. The two packages that you did install are only two of the packages, not the full stack.

Since this PR was merged very recently, and we had just fixed some issues and re-enabled rosbag2_py 21 days ago, I don't know if it's included in ros-foxy-rosbag2 already. Your best bet is probably to build this repository from source.

[evclark]

Hi @evclark, for future reference, please post questions as a new Issue or on ROS Answers, as opposed to commenting on a merged PR.

Noted, will do, thanks

First, you are using Foxy, but you installed the packages for crystal? That does not look right. Your distro should be foxy.

Ah yes, sorry, copy/paste error from the README, I did install the foxy, not crystal, packages.

Since this PR was merged very recently, and we had just fixed some issues and re-enabled rosbag2_py 21 days ago, I don't know if it's included in ros-foxy-rosbag2 already. Your best bet is probably to build this repository from source.

It appears not to be. I'll try building from source, thanks!

[jacobperron]

FWIW, I don't expect this feature to land in Foxy. While we wait for Galactic to be released, you can build from source. Once a new release is made into Rolling, you should be able to install the binary.

[agutenkunst]

FWIW, I don't expect this feature to land in Foxy. While we wait for Galactic to be released, you can build from source. Once a new release is made into Rolling, you should be able to install the binary.

Building from source including rosbag2_py on foxy and not against Rolling worked for me with

git clone git@github.com:ros2/rosbag2.git -b 0.3.5 && cd rosbag2 && git merge 038bae248e2f97e03010da7765bfbe9da0d8e0b9 -X theirs --no-edit

Only small unresolvable conflict is inside a workflow file.

Is there a better solution? I think lot of people would be happy having this feature in foxy!

Disclaimer: I did not try to read a bagfile yet.

[jacobperron]

In retrospect, it may be easy to backport this new package to Foxy (assuming it's not relying on anything new in Galactic).

Pinging maintainers to see if there is any opposition, @mjeronimo @ros2/tooling-approvers

[emersonknapp]

It may be easy, but the APIs that it binds to may have changed. I'm all for someone trying it out, no opposition from me - I'm just not 100% confident it will be easy