Scaling-out

[JohnGreenan]

At the moment the whole platform runs on one box. This is not viable for production use. The APIs between the components would need to be reworked inside the implementation, such that

the number of FIX engines can be increased and distributed to multiple boxes
the number of OrderBooks can be increased and distributed to multiple boxes
market data distribution can be performed from a specified set of UDP/Multicast ports

[JohnGreenan]

This is something that can be done relatively easily as a proof-of-concept.

Allow the Matching engine to interact with an OrderBook with a message oriented middleware queue in the middle rather than a Java ConcurrentLinkedinQueue.

In an enterprise implementation this would involve passing an SBE message around, for now this can use some naive Java serialisation/deserialisation model

[JohnGreenan]

Decided to implement using SBE rather than a naive Java serialisation/deserialisation model.

[JohnGreenan]

FIX>POJO>Binary to matching engine completed.
OrderBook matches the order
POJO event emitted by OrderBook.
The POJO needs to be converted to Binary, published to Kafka and then collected by a process running alongside the FIX engine instance.
Pattern suggestion - one topic per workflow item (execution report, unsolicited amendment, reject and so on)
One thread per topic.
Each topic listener converts binary to FIX and then queues up for the FIX engine to send to the counterparty.

[JohnGreenan]

resolved in #22