A common problem with streaming platforms is whether peers and tasks can exhibit backpressure to upstream tasks.
Onyx provides natural backpressure via Aeron’s bounded term buffers. Whenever a
message is offered to a peer that has not consumed enough of its term buffer,
the offer will not succeed and a backpressure signal will be signaled back to
the peer. As Onyx’s messaging is written in a non blocking way, the peer will then back-off for between
idle-min-sleep-ns and
idle-max-sleep-ns.
Onyx’s checkpointing mechanism is also subject to backpressure as only one checkpoint will be running for a job at any given time.