Shared memory transport design documents <master> [7932]

[MiguelCompany]

Starting with the proposal information from Sony about shared memory transport and shared memory data sharing (#810), we will discuss and create a final design document for the shared memory transport.

[IkerLuengo]

I reworked the ms1_rtps_shared_memory_transport.md, but there are some comments pending

[fujitatomoya]

what is the definition by machine? including container which actually in the same machine?

this is also related to how it can detect if inter-process or not, but i guess that better to mention that also.

[adolfomarver]

It's true "machine" is not a precise definition, The idea is that shared-memory will improve communications between processes capable of sharing local memory, that running in the same operating system instance in a machine, but of course processes running in the same container or virtual machine but not intercontainer or virtual machines.
How could we well define this idea in a phrase?

[fujitatomoya]

i think there would be some steps to take,

• basically lock-free is better than lock-required for any reason. but eventually we do need the mutex locks between processes.
• exception must be caught by handler, and it must release the lock before the process gone. (actually process crash means that there is a bug.)
• and we can also consider read/write locks instead of mutex, and also Compare And Swap instruction to get the lock.

one question by here, what about the multiple writer case? it does not require the lock? expecting this is because writer creates local memory space to be shared so that writer does not need a lock with other writers. am i understanding correct?

[adolfomarver]

In a shared-memory port's segment resides a RingBuffer, an interprocess-mutex, and an interprocess-condition variable. Operations over the ring buffer are lock-free atomic (compare and swap) so writers and readers may push or pop data without locking any mutex.

The mutex and condition variable are used to wait on when the ring buffer is empty. So if a reader detects the ring buffer is empty it will wait on the condition. When writers push elements, lock the empy condition before pushing and after pushing notify on the condition to wake-up listeners.

So if no bugs or crashes everything is OK, but if a process crash holding an interprocess-lock it will block all readers and writers using that port. This is bug situation, so involved processes would need to be restarted or a fail-tolerance mecanishm is needed.

[fujitatomoya]

is there already plan how it provide the interface to configure the memory space? API or configuration file to load?

[adolfomarver]

The memory space can be configured through the transport descriptor, that can be done in the application's C++ code or with an external XML file using the fastrtps XML profiles feature.

[adolfomarver]

It's true "machine" is not a precise definition, The idea is that shared-memory will improve communications between processes capable of sharing local memory, that running in the same operating system instance in a machine, but of course processes running in the same container or virtual machine but not intercontainer or virtual machines.
How could we well define this idea in a phrase?

[adolfomarver]

In a shared-memory port's segment resides a RingBuffer, an interprocess-mutex, and an interprocess-condition variable. Operations over the ring buffer are lock-free atomic (compare and swap) so writers and readers may push or pop data without locking any mutex.

The mutex and condition variable are used to wait on when the ring buffer is empty. So if a reader detects the ring buffer is empty it will wait on the condition. When writers push elements, lock the empy condition before pushing and after pushing notify on the condition to wake-up listeners.

So if no bugs or crashes everything is OK, but if a process crash holding an interprocess-lock it will block all readers and writers using that port. This is bug situation, so involved processes would need to be restarted or a fail-tolerance mecanishm is needed.

[adolfomarver]

The memory space can be configured through the transport descriptor, that can be done in the application's C++ code or with an external XML file using the fastrtps XML profiles feature.

[IkerLuengo]

Move the documentation of the current implementation (subfolder eprosima) to doc/design. Keep the documentation of the proposal by SONY on proposals.

[IkerLuengo]

👍