Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update documentation about recorder and player creation via composition #1510

Merged
merged 10 commits into from
Apr 10, 2024
Merged
119 changes: 119 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@ These verbs are available for `ros2 bag`:

For up-to-date information on the available options for each, use `ros2 bag <verb> --help`.

Moreover, `rosbag2_transport::Player` and `rosbag2_transport::Recorder` components can be instantiated in `rclcpp` component containers, which makes possible to use intra-process communication for greater efficiency.
See [composition](#using-with-composition) section for details.

### Recording data

In order to record all topics currently available in the system:
Expand Down Expand Up @@ -343,6 +346,122 @@ For example, if we named the above XML launch script, `record_all.launch.xml`:
$ ros2 launch record_all.launch.xml
```

You can also invoke the `play` and `record` functionalities provided by `rosbag2_transport` package as nodes.
The advantage to use this invocation strategy is that the Python layer handling the `ros2 bag` CLI is completely skipped.

```python
import launch

def generate_launch_description():
return launch.LaunchDescription([
launch.actions.Node(
package='rosbag2_transport',
executable='player',
name='player',
output="screen",
parameters=["/path/to/params.yaml"],
)
])
```
MichaelOrlov marked this conversation as resolved.
Show resolved Hide resolved

## Using with composition

Play and record are fundamental tasks of `rosbag2`. However, playing or recording data at high rates may have limitations (e.g. spurious packet drops) due to one of the following:
- low network bandwith
- high CPU load
- slow mass memory
- ROS 2 middleware serialization/deserialization delays & overhead

ROS 2 C++ nodes can benefit from intra-process communication to partially or completely bypass network transport of messages between two nodes.

Multiple _components_ can be _composed_, either [statically](https://docs.ros.org/en/rolling/Tutorials/Intermediate/Composition.html#compile-time-composition-using-ros-services) or [dynamically](https://docs.ros.org/en/rolling/Tutorials/Intermediate/Composition.html#run-time-composition-using-ros-services-with-a-publisher-and-subscriber): all the composed component will share the same address space because they will be loaded in a single process.

A prerequirement is for each C++ node to be [_composable_](https://docs.ros.org/en/rolling/Concepts/Intermediate/About-Composition.html?highlight=composition) and to follow the [guidelines](https://docs.ros.org/en/rolling/Tutorials/Demos/Intra-Process-Communication.html?highlight=intra) for efficient publishing & subscription.

With the above requirements met, the user can:
- compose multiple nodes together
- explicitly enable intra-process communication

Whenever a publisher and a subscriber on the same topic belong to the same _composed_ process, and intra-process is enabled for both, `rclcpp` completely bypasses RMW layer and below transport layer (i.e. DDS). Instead, messages are shared via process memory and *potentially* never copied. Some exception hold, so please have a look to the [IPC guidelines](https://docs.ros.org/en/rolling/Tutorials/Demos/Intra-Process-Communication.html?highlight=intra).
Comment on lines +369 to +385
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if we want to re-explain the benefits from Composition in general, which is described in https://docs.ros.org/en/rolling/Concepts/Intermediate/About-Composition.html#ros-2-unified-api good.
I would refer to this link, and keep this doc to how to use player and recorder as components only in rosbag2 package.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw, i just tried with the following. this is not expected result, i may be mistaken?

root@tomoyafujita:~/ros2_ws/colcon_ws# ros2 component load /ComponentManager composition rosbag2_transport::Player
Failed to load component: Failed to find class with the requested plugin name.
root@tomoyafujita:~/ros2_ws/colcon_ws# ros2 component load /ComponentManager composition rosbag2_transport::Recorder
Failed to load component: Failed to find class with the requested plugin name.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fujitatomoya It works with

ros2 component load /ComponentManager rosbag2_transport rosbag2_transport::Player

need to use rosbag2_transport instead of composition.
It seems to be able to use it as you expected with ros2 component load /ComponentManager composition rosbag2_transport::Player will need to use composition instead of the ${PROJECT_NAME} here

rclcpp_components_register_node(
${PROJECT_NAME} PLUGIN "rosbag2_transport::Player" EXECUTABLE player)
rclcpp_components_register_node(
${PROJECT_NAME} PLUGIN "rosbag2_transport::Recorder" EXECUTABLE recorder)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As regards to the

I am not sure if we want to re-explain the benefits from Composition in general, which is described in https://docs.ros.org/en/rolling/Concepts/Intermediate/About-Composition.html#ros-2-unified-api good.
I would refer to this link, and keep this doc to how to use player and recorder as components only in rosbag2 package.

I have an opposite opinion and think it would be very useful to have a brief introduction with an explanation of why it is needed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As regards composition name. Me and @fujitatomoya think it would be better to use rosbag2 name instead of rosbag2_transport or composition.
i.e. to be

ros2 component load /ComponentManager rosbag2 rosbag2_transport::Player

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately not easy, if you remember we already discussed during implementation. Current CMake registration macro doesn't allow as far as I understand to change package name when registering components in ament.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

@roncapat roncapat Feb 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The most trivial way to achieve so would be to register the components in rosbag2 package, adding something like (draft code here):

#include "rosbag2_transport/player.hpp"

namespace rosbag2 {

class Player : rosbag2_transport::Player {
  using rosbag2_transport::Player::Player;
};

}

#include "rclcpp_components/register_node_macro.hpp"

// Register the component with class_loader.
// This acts as a sort of entry point, allowing the component to be
// discoverable when its library is being loaded into a running process.
RCLCPP_COMPONENTS_REGISTER_NODE(rosbag2::Player)

Then build and register it inside rosbag2 package CMakeLists.txt. Still unclear then if some component load tests would need to be moved out from rosbag2_transport too then.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@roncapat Thanks for the proposal with a workaround in the `rosbag2 package.
I would appreciate it if you could try to make prototype changes for that.

I am sorry, that I don't have the capacity for such an experiment by myself for a while.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possible approach here #1603


Here is an example of Python launchfile composition. Notice that composable container components do not expect YAML files to be directly passed to them: parameters have to be "dumped" out from the YAML file (if you have one). A suggestion of possible implementation is offered as a starting point.

```python
import launch
import launch_ros
import yaml

'''
Used to load parameters for composable nodes from a standard param file
'''
def dump_params(param_file_path, node_name):
with open(param_file_path, 'r') as file:
return [yaml.safe_load(file)[node_name]['ros__parameters']]

def generate_launch_description():
return launch.LaunchDescription([
launch.actions.ComposableNodeContainer(
name='composable_container',
package='rclcpp_components',
executable='component_container',
composable_node_descriptions=[
launch_ros.descriptions.ComposableNode(
package='rosbag2_transport',
plugin='rosbag2_transport::Player',
name='player',
parameters=dump_params("/path/to/params.yaml", "player"),
extra_arguments=[{'use_intra_process_comms': True}]
),
# your other components here
]
)
])
}
```

Here's an example YAML configuration for both composable player and recorder:
```yaml
recorder:
ros__parameters:
use_sim_time: false
record:
all: true
is_discovery_disabled: false
topic_polling_interval:
sec: 0
nsec: 10000000
include_hidden_topics: true
ignore_leaf_topics: false
start_paused: false

storage:
uri: "/path/to/destination/folder"
storage_id: "sqlite3"
max_cache_size: 20000000
```
and
```yaml
player:
ros__parameters:
play:
read_ahead_queue_size: 1000
node_prefix: ""
rate: 1.0
loop: false
# Negative timestamps will make the playback to not stop.
playback_duration:
sec: -1
nsec: 00000000
start_paused: false

storage:
uri: "path/to/rosbag/file"
storage_id: "mcap"
storage_config_uri: ""
```

MichaelOrlov marked this conversation as resolved.
Show resolved Hide resolved
For a full list of available parameters, you can refer to [`player`](rosbag2_transport/test/resources/player_node_params.yaml) and [`recorder`](rosbag2_transport/test/resources/recorder_node_params.yaml) configurations from the `test` folder of `rosbag2_transport`.

## Storage format plugin architecture

Looking at the output of the `ros2 bag info` command, we can see a field `Storage id:`.
Expand Down
Loading