-
Notifications
You must be signed in to change notification settings - Fork 875
Client Concurrency Model
When clients interact with Aeron it is worth being aware of the concurrency model to know what is safe and what is not safe to be used across threads or processes.
Aeron clients communicate with media driver via the command and control (C'n'C) file which is memory mapped. The media driver can run in or out of process as required. An out of process driver is isolated and can be shared by many clients. Each media driver can have many clients which can concurrently interact. Clients send commands to the driver and the driver will broadcast back events. A bi-directional heart-beating protocol is used between the driver and clients to track each others presence.
Within a process the Aeron client is thread safe by default and can be shared between threads to manage the life-cycles of publications and subscriptions. It is good practice to have one Aeron client per process but this is not a hard requirement. It is possible to use the NoOpLock in the client when used with the Invoker for greater efficiency when used from only one thread.
Aeron Publications are thread safe within and across processes. A Publication object can be used concurrently from many threads. When separate processes add the same Publication (channel and stream id) then they will map to the same underlying memory-mapped file and can be safely used concurrently. Messages offered to the same publication will be serialised. Publications with a different channel and/or stream id are completely independent from each other and can be used concurrently.
ExclusivePublications can be created for exclusive use by a single thread, they are not thread safe. These exclusive publications can co-exist with other publications of the same channel and stream identifiers but get another session id. Exclusive publications enjoy the benefits of the single writer principle and can benefit from greater throughput, especially in burst scenarios for small messages.
Exclusive publications can additionally provide control of the header when used with the BufferClaim. It is also possible to set the channel URI params for init-term-id, term-id, and term-offset which can be useful in replay scenarios. These params combined with mtu and term-length provide for replay control. Note: If implementing your own replay then careful consideration needs to be given to happen padding is applied in term buffers when unblocking or padding to the end.
Subscriptions are not thread safe. A Subscription is a aggregate over a set of Images from different sources for the same channel and stream identifier. Each subscribing thread must create its own Subscription and poll for updates. Received messages will be delivered to all subscribers in a similar fashion to how multicast packets are received. Subscriptions to the same channel and stream id in different processes will all receive the same messages from the media driver they are connected to. Subscribers all receive the messages in parallel without additional copies.
When subscriptions join a stream they will receive messages starting from the position reached by an existing stream when they joined, each stream appears as an Image of the source Publication. They do not receive messages which have been received by other subscribers previous to the position at which they joined. If a Subscription is closed and then reopen, it will be rejoining the stream and will miss the messages contain in the stream between the old subscription closing and the new one being added. When a new Image becomes available it will be added to all matching active Subscriptions so that they all get the same join position.
When an IPC, or spy, Subscription is added then it joins existing Publication on that driver before the add operation returns so that there is no gap in the stream being the Subscription being added and the image becoming available. If a non-spy UDP Subscription is added and there is an existing matching Publication then there is a gap between the Subscription being added and the Image becoming available as this has to go via the network.
When multiple Subscriptions are present on the same driver for the same channel and stream id then they operate independently with flow control applied to apply back pressure based on the slowest.
Implementations of functional interfaces can be registered as handlers for events in the client for the notification of images and countered becoming available and unavailable. The handlers are called on the client conductor thread. The client conductor thread expects handlers to do safe publication of any state to other threads and not be long running or re-entrant with the client. If the client AgentInvoker is used then the callbacks occur on the same thread which called invoke.