RCL Executor and convenience functions

[JanStaschulat]

Real Time Executor and convenience functions based on RCL layer.

rcl_executor package

Overview

This real-time executor is targeted as C API based on the RCL layer for ROS2 applications running on a micro-controller[micro-ROS]. It allows the user to specify in which order rcl-handles (subscriptions, timers) are executed after they become ready in DDS. Additionally it implements the logical-execution-time (LET) paradigm. Well known in automotive, this paradigm describes the sequence of processing tasks: First new input data is read for all tasks, then all tasks are executed, and finally the output data of all tasks is written. The benefits are very low synchronization efforts because there is no interference between input data [EK2018] [BP2017]. See also a more detailed motivation of this Real Time Executor on the micro-ROS website.

API

The API of the RCL-Executor provides functions, like add_subscription and add_timer, to add rcl-subscriptions and rcl-timers to the Executor. The order, in which the user adds these handles to the Executor, determines later on the execution order when new data is available for these handles. The executor is activated by calling the spin_some, spin_period() or spin() method.

LET semantics

The LET-semantic is implemented by,

1. waiting for new data from DDS (rcl_wait()), then
2. requesting all new data for all ready handles from DDS (rcl_take()) and then
3. executing all ready handles in the order specified by the user.

Waiting until all callbacks are processed and then publishing them, has been omitted in this implementation of the LET semantics, because this would require potentially unbounded buffers for publishing messages. However, this sequence already guarantees no interference between input data for the handles.

Memory allocation

Special care has been taken about memory allocation. Memory is only allocated in the configuration phase, i.e. by specifying how many handles shall be executed, while at running phase, no memory is allocated by the RCL Executor.

rcl_ext package

The package rcl_ext provides extensions to rcl to create a node, a subscription, a timer as a one-liner, just like rclc but without creating new data structures for these objects. This is especially valuable for ease-of-use for micro-controller use-cases.

Example:

An example, how to use the RCL Executor and the rcl_ext package can be found in the repository micro-ROS-demos on branch feature/rcl_executor_examples

Building and running the example:
After having built the rcl package with rcl_executor and rcl_ext), do the following:

$>colcon build --packages-select rcl_executor_examples
$>ros2 run rcl_executor_examples executor_with_rcl_ext

Limitations:

• support for subscriptions and timers (services, clients, guard conditions are not supported yet)

References

[micro-ROS] micro-ROS project

[EK2018] R. Ernst, S. Kuntz, S. Quinton, M. Simons: The Logical Execution Time Paradigm: New Perspectives for Multicore Systems, February 25-28 2018 (Dagstuhl Seminar 18092). Paper

[BP2017] A. Biondi, P. Pazzaglia, A. Balsini, M. D. Natale: Logical Execution Time Implementation and Memory Optimization Issues in AUTOSAR Applications for Multicores, International Worshop on Analysis Tools and Methodologies for Embedded and Real-Time Systems (WATERS2017), Dubrovnik, Croatia.Paper

[ralph-lange]

@wjwwood, we (in particular @JanStaschulat) will also present and discuss the LET Executor and this PR in the next Real-Time WG. A review until this meeting is nevertheless highly appreciated. Looking forward to your feedback!

[jacobperron]

Thanks for the contribution! I've done an initial review of rcl_executor (I haven't looked at the tests). It seems like a reasonable addition to me, though I'd like to hear from others about its inclusion.

Do you imagine the LET executor implementation being used by the higher-level client libraries (e.g. rclcpp and rclpy)? And if so, are you willing to contribute these changes?

I haven't look at rcl_ext yet, but it sounds like something that might be merged into rclc. In any case, I think it makes more sense to open a separate PR for rcl_ext to help with the review process.

[fujitatomoya]

rcl_executor is planned to be used on top of rclcpp and rclpy? is that why trying to merge this into rcl?
or if rcl_executor is expected to be in C API only, rclc is suitable place to be merged into?

sorry i did not the history very much, maybe this is alredy dicussed and done.

[JanStaschulat]

the rcl_executor is intended as a deterministic real-time executor for C programming language. It is going to be used in the micro-ros project https://micro-ros.github.io/docs/concepts/client_library/real-time_executor/ to provide ROS2 functionality for micro-controllers.

It differs to the standard rclcpp executor regarding: static order of execution of all handles (whereas in the static executor this has not a clear semantics as described here http://drops.dagstuhl.de/opus/volltexte/2019/10743/

as well as LET (logical execution time) semantics. That is before executing any callback in the round of spin_some, ALL input data is requested from DDS with rcl_take(). This has the benefit, that synchronization between input dat is not requirered any more.

Indeed, as the rcl_executor is not just an C API, but also changes the semantics of the executor, oe might want to expose this to rclcpp rclpy to many other ROS2 users.

[fujitatomoya]

@jacobperron

i do agree on this, it seems to be different language frontend for me.

[jacobperron]

@jacobperron Is it actually necessary to call get_zero_initialized_wait_set again?

I guess maybe not, but I'm not sure. It's probably safer to call it again, I don't think it's adding much overhead.

[jacobperron]

I've spoke with @wjwwood about these changes, and we have some concerns I've mentioned below.

Additionally, I think until we see PRs where the rcl_executor implementation is used in rclcpp and rclpy, it doesn't make sense to merge any implementation here. The reason being that the purpose of rcl packages is to contain shared implementation for the ROS client libraries. If we don't plan to use the executor functions in multiple client libraries, then it is probably best to move the functionality to rclc.

[JanStaschulat]

As discussed in Real-Time WG, I already started refactoring this PR for rclc in https://github.com/micro-ROS/rclc/tree/feature/new_api_and_LET_executor. Will pose a PR to rclc in January and close this PR here.

[JanStaschulat]

@wjwwood @dejanpan @fujitatomoya @jacobperron @Karsten1987.
I am currently updating the documentation and working on the branch https://github.com/micro-ROS/rclc/tree/feature/new_api_and_LET_executor_trigger
which includes also some more features.

[jacobperron]

@JanStaschulat Since it is preferred to put this type of change in rclc, I'm going to close ticket. Feel free to reference a PR to rclc here when it is opened.