Improve intraprocess communication behavior

[mjcarroll]

There have been some performance issues uncovered in the behavior and assumptions made in the implementation of intraprocess communications. This is a meta-ticket to track improvements to intraprocess communications planned for D-turtle.

Tasks:

• Document talk between @mjcarroll and @wjwwood that describes current behavior and enumerates properties of various publisher/subscription types.
• Use rmw_count_matched_* to avoid publishing interprocess when a publication only has intraprocess subscriptions.
  • In the current system, a copy of the message must be made in order for both interprocess and intraprocess communications to have a copy that is owned by the respective system. Given the newly exposed subscription/publication count, this copy could be avoided.
  • This should be doable by checking the total subscription count vs the intraprocess subscription count.
  • Done in Added count matching api and intra-process subscriber count rclcpp#628
• Store unique or shared message in intraprocess manager based on publisher/subscription type.
  • In the current system, it is assumed that unique_ptr is used on both the publisher and subscription side. If the intraprocess manager was capable of storing shared_ptr, and subscriptions received shared_ptr (read only), then copies could be avoided in the case of multiple intraprocess subscriptions as well as mixed intra/interprocess subscriptions
  • Done in: New IntraProcessManager capable of storing shared_ptr<const T> rclcpp#690
• (stretch) offer "direct dispatch" (C++ only) - This would be similar to how ROS1 nodelets worked in that subscriptions could be directly invoked from the publication-side publish method. This is particularly useful in processing pipeline environments

Based on:

• Fix a bug when publishing unique_ptr rclcpp#504
• Intra-process communication latency demos#289

References:

• https://index.ros.org/doc/ros2/Tutorials/Intra-Process-Communication/

[dirk-thomas]

Use rmw_count_matched_* to avoid publishing interprocess when a publication only has intraprocess subscriptions.

I am not sure this is feasible. The publisher could use QoS settings to deliver previously published messages to late joining subscribers. In that case skipping the inter process publish just because no subscribers are present at the moment would be wrong.

[wjwwood]

I am not sure this is feasible. The publisher could use QoS settings to deliver previously published messages to late joining subscribers. In that case skipping the inter process publish just because no subscribers are present at the moment would be wrong.

Obviously for non volatile durability you have to publish anyways, but yes it's worth noting that this only works if there are no subscriptions and durability is volatile.

[hidmic]

@mjcarroll did put down your offline discussion w/ @wjwwood in a document? Maybe I can start pushing one of these bullets. Which one's the smallest? :)

[mjcarroll]

Probably not ready for a PR, but includes more notes from our discussion: https://github.com/ros2/design/blob/mjcarroll/intraprocess/articles/160_intraprocess_communications.md

[dirk-thomas]

Latest PR on this topic: ros2/rclcpp#690.

[alsora]

Hi, I was having a look at the current intra process manager.
Are there plans to remove the do_intra_process_publish function that still relies on the DDS?
https://github.com/ros2/rclcpp/blob/98f610c114b8cc7bda2ba3a3d1d486767ee293e5/rclcpp/include/rclcpp/publisher.hpp#L91

If no, could I ask which are the reasons why it's required?