Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Design Document for Zero-Copy via Loaned Messages #256

Open
wants to merge 9 commits into
base: gh-pages
from

Conversation

@mjcarroll
Copy link
Contributor

commented Sep 17, 2019

Design document for using loaned messages to provide support for zero-copy middleware implementations.

This is from notes that @Karsten1987 and @wjwwood composed, reformatted and expanded.

mjcarroll added 3 commits Sep 17, 2019
Signed-off-by: Michael Carroll <michael@openrobotics.org>
Signed-off-by: Michael Carroll <michael@openrobotics.org>
Signed-off-by: Michael Carroll <michael@openrobotics.org>
Copy link
Contributor

left a comment

I just added some typo fixes. I haven't reviewed the whole thing yet, particularly the proposed API.

articles/zero_copy.md Outdated Show resolved Hide resolved
articles/zero_copy.md Outdated Show resolved Hide resolved
articles/zero_copy.md Outdated Show resolved Hide resolved
articles/zero_copy.md Outdated Show resolved Hide resolved
articles/zero_copy.md Outdated Show resolved Hide resolved
Co-Authored-By: Chris Lalancette <clalancette@openrobotics.org>
@Karsten1987

This comment has been minimized.

Copy link

commented Sep 17, 2019

articles/zero_copy.md Outdated Show resolved Hide resolved
mjcarroll added 2 commits Sep 17, 2019
Signed-off-by: Michael Carroll <michael@openrobotics.org>
Signed-off-by: Michael Carroll <michael@openrobotics.org>
* The user must be able to get a loaned message from the middleware when calling take.
* The user must be able to get a sequence of loaned messages from the middleware when calling take.
* The loaned message or sequence must be returned by the user.
* When taking a loan, the middleware should not do anything "special", that is that the user must be able to influence the allocation.

This comment has been minimized.

Copy link
@Karsten1987

Karsten1987 Sep 17, 2019

Should that be when not taking a loan? To my understanding the middleware should be completely in charge of allocations when taking a loan. However, if the middleware isn’t capable of zero copy, the user should have means to pass in custom allocators and thus control the allocation.

This comment has been minimized.

Copy link
@mjcarroll

mjcarroll Sep 17, 2019

Author Contributor

I was a bit unclear on this as I was going through the previous notes.

Is the idea that users should always take a loan, regardless of if the middleware supports it? That would put the burden on the middleware for managing that memory. Or should it be the case that there is no allocation when loans aren't supported?

* The user must be able to have the middleware fill data into messages allocated in the user's stack or heap.
* The user must be able to get a loaned message from the middleware when calling take.
* The user must be able to get a sequence of loaned messages from the middleware when calling take.
* The loaned message or sequence must be returned by the user.

This comment has been minimized.

Copy link
@mjcarroll

mjcarroll Sep 17, 2019

Author Contributor

Note from @Karsten1987:

We should specify how we can make sure that the user doesn't hold too many of these loaned messages when not taking them via the executor/callback but rather via polling the waitset.

AUTOSAR specifies this as not possible to gain access to the sequence without specifying an according callback where the sequence eventually gets passed in via std::unique_ptr.

This comment has been minimized.

Copy link
@deeplearningrobotics

deeplearningrobotics Oct 2, 2019

@mjcarroll: How does the AUTOSAR approach help? I still can just keep the unique_ptr in my container forever.


* The user must be able to have the middleware fill data into messages allocated in the user's stack or heap.
* The user must be able to get a loaned message from the middleware when calling take.
* The user must be able to get a sequence of loaned messages from the middleware when calling take.

This comment has been minimized.

Copy link
@mjcarroll

mjcarroll Sep 17, 2019

Author Contributor

Note from @Karsten1987 :

There should be a parameter for specifying how many items such a sequence can hold. That would help to use fixed size containers in its implementation. This could be defaulted to the subscription history depth though.

Copy link
Contributor

left a comment

I think that the idea of loaned messages is the best solution for zero-copy intraprocess communication.
Currently, most of the middleware vendors we're supporting doesn't have a loaned message API.
So, it may be a good idea to also push development to add this in some of them.

Fixes #251.
That issue have some comments in line with this proposal.
Also, there is a similar previous discussion in #239.

rclcpp::Publisher::can_loan_messages()
```

### RCLCPP Subscription

This comment has been minimized.

Copy link
@ivanpauno

ivanpauno Sep 18, 2019

Contributor

I guess, that no modification in RCLCPP subscription would be needed.
Maybe we would need to add a new subscription signature more convenient for loaned messages, but not more than that. Probably, similar changes would be needed in rcl/rmw.

```

### RCLCPP Subscription

This comment has been minimized.

Copy link
@ivanpauno

ivanpauno Sep 18, 2019

Contributor

We should also add to the document an RCLPY API

@Karsten1987

This comment has been minimized.

Copy link

commented Sep 19, 2019

I think we talked about this, but I think we didn't come to an conclusion.

When we talk about taking sequences for messages which can be loaned, this implies that there has to be some sort of adjustment/enhancement on for a rmw_take_loaned_message_sequence function.
Right now I envision something like:

rmw_ret_t
rmw_take_loaned_message_sequence(rmw_message_sequence_t * message_sequence, size_t n);

Where n is the amount of messages being fetched and stored in the message sequence.

If I bring this all the way up to a possible implementation in the rclcpp::Executor::execute_subscription function, this would like as such:

void
Executor::execute_subscription(
  rclcpp::SubscriptionBase::SharedPtr subscription)
{
  rmw_message_info_t message_info;
  message_info.from_intra_process = false;

  // handle serialized messages
  if (subscription->is_serialized()) {
    auto serialized_msg = subscription->create_serialized_message();
    auto ret = rcl_take_serialized_message(
      subscription->get_subscription_handle().get(),
      serialized_msg.get(), &message_info, nullptr);
    if (RCL_RET_OK == ret) {
      auto void_serialized_msg = std::static_pointer_cast<void>(serialized_msg);
      subscription->handle_message(void_serialized_msg, message_info);
    } else if (RCL_RET_SUBSCRIPTION_TAKE_FAILED != ret) {
      RCUTILS_LOG_ERROR_NAMED(
        "rclcpp",
        "take_serialized failed for subscription on topic '%s': %s",
        subscription->get_topic_name(), rcl_get_error_string().str);
      rcl_reset_error();
    }
    subscription->return_serialized_message(serialized_msg);
    return;
  }

  // if a middleware can provide loaned messages, we always take them as such
  if (subscription->can_loan_messages()) {
    size_t n = 1;  // how many messages are we fetch at a time
    // Could be part of the subscription API in case of polling
    // Maybe take a parameter n for reserving space in the loaned message sequence
    auto LoanedMessageSequence lms = subscription->create_loaned_sequence(n);
    auto ret = rcl_take_loaned_message_sequence(lms.get_sequence_handle(), n);
    if (RCL_RET_OK == ret) {
      subscription->handle_message(lms.msg_at(0), lms.msg_info_at(0));
    } else if (RCL_RET_TAKE_FAILED != ret) {
      RCUTILS_LOG_ERROR_NAMED(
        "rclcpp",
        "could not take loaned message sequence on topic '%s': %s",
        subscription->get_topic_name(), rcl_get_error_string().str);
      rcl_reset_error();
    }
    subscription->return_loaned_sequence(lms);
    return;
  }

  // if the middleware can't loan message, we create our own instances and take them
  std::shared_ptr<void> message = subscription->create_message();
  auto ret = rcl_take(
    subscription->get_subscription_handle().get(),
    message.get(), &message_info, nullptr);
  if (RCL_RET_OK == ret) {
    subscription->handle_message(message, message_info);
  } else if (RCL_RET_SUBSCRIPTION_TAKE_FAILED != ret) {
    RCUTILS_LOG_ERROR_NAMED(
      "rclcpp",
      "could not deserialize serialized message on topic '%s': %s",
      subscription->get_topic_name(), rcl_get_error_string().str);
    rcl_reset_error();
  }
  subscription->return_message(message);
}

Looking at the scribble above, you can see an inconsistency between serialized/non-loaned messages and loaned messages. The first two types are always fetch one-by-one, whereas loaned messages are always fetched in a sequence - main reason for this being a strong requirements for RTI Connext Micro in order to enable zero-copy.

The question which arises now is whether any rmw_take_* should be modified in a way that it always takes n instances or if we restrict that to loaned messages exclusively.
If we did change rmw_take to always take multiple, then we have to push this consequently to rmw_serialized_messages as well as ROS messages allocated by the rclcpp::MessageMemoryStrategy and implement C-based container structures for it.

My thinking here is that in a context of an executor, we probably would get around implementing the message sequences for serialized and non-loaned messages as we handle one callback at a time.
But thinking this further and having the ability to use the API outside of the scope of an executor (by polling the waitset or having different implementations of an executor), I personally believe it would be the right thing to do and extend the sequencing also to these types. As the rmw_take_multiple functions can be implemented next to the existing rmw_take, we can still use the latter inside the executor.

Opinions?

In the second case, the memory that is used for the loaned message should be optionally provided by the middleware.
For example, the middleware may use this opportunity to use a preallocated pool of message instances, or it may return instances of messages allocated in shared memory.
However, if the middleware does not have special memory handling or preallocations, it may refuse to loan memory to the client library, and in that case, a user provided allocator should be used.
This allows the user to have control over the allocation of the message when the middleware might otherwise use the standard allocator (new/malloc).

This comment has been minimized.

Copy link
@allenh1

allenh1 Sep 19, 2019

it may refuse to loan memory to the client library, and in that case, a user provided allocator should be used.

this makes a lot of sense.

This allows the user to have control over the allocation of the message when the middleware might otherwise use the standard allocator (new/malloc)

This seems to contradict that prior statement, though. Maybe I'm not reading this quite right?

I think this should read something more along the lines of "If the middleware does not have specific protocols for handling preallocations of messages, it should use the rcutils allocator."

I could be way off base in my interpretation, though.

rmw_ret_t
rmw_publish(
const rmw_publisher_t * publisher
const void * ros_message

This comment has been minimized.

Copy link
@allenh1

allenh1 Sep 19, 2019

Missing some ','s on these lines.


```
void *
rcl_allocate_loaned_message(

This comment has been minimized.

Copy link
@allenh1

allenh1 Sep 19, 2019

Has the message already been loaned? If it is planned to be loaned, maybe this should be called rcl_allocate_loaner_message?

@dejanpan dejanpan referenced this pull request Sep 19, 2019
27 of 29 tasks complete
@Karsten1987

This comment has been minimized.

Copy link

commented Sep 21, 2019

connected to ros2/ros2#785

articles/zero_copy.md Outdated Show resolved Hide resolved
Signed-off-by: Michael Carroll <michael@openrobotics.org>
mjcarroll added 2 commits Oct 15, 2019
Signed-off-by: Michael Carroll <michael@openrobotics.org>
Signed-off-by: Michael Carroll <michael@openrobotics.org>
@mjcarroll mjcarroll marked this pull request as ready for review Oct 15, 2019
@wuffle-ros wuffle-ros bot added the in review label Oct 15, 2019

### RCLCPP LoanedMessage

In order to support loaned messages in `rclcpp`, we tntroduce the concept of a `LoanedMessage`.

This comment has been minimized.

Copy link
@alsora

alsora Oct 16, 2019

Suggested change
In order to support loaned messages in `rclcpp`, we tntroduce the concept of a `LoanedMessage`.
In order to support loaned messages in `rclcpp`, we introduce the concept of a `LoanedMessage`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.