How, and why we use Redis

This page briefly describes the problems that lead to me picking Redis, as well as how I implemented solutions.

These features are listed in the order that they were implemented.

Maintaining a Consistent View of an Account's Trades

Trade data is different from most data on the exchange, as it remains static after it's created. We don't need it for future exchange, or even account operations, aside from a rare occurrence like allowing a user to view their past trades, or keeping track of ownership.

For these reasons, trade data really doesn't need to be in RAM, and it especially doesn't need to be taking up our application's address space. The ideal place to store trade data is on disk, but there are scenarios where storing it in RAM, at least temporarily, is essential.

Suppose we are flushing Trade buffer to disk. This process takes some time, maybe seconds, maybe minutes, possibly even hours (factors include disk speed, number of inserts, etc). Then, for an unknown period of time, we cannot be sure that the database is fully consistent and will have the most recent trades of the user. As stated earlier, we really don't want to waste our limited RAM on storing trades, but we need to have a fast way to store and access this data during the period of uncertainty.

This is a great opportunity for a remote key-value store like Redis. We can just associate a list of trades to a user, and over time we can grow the list as more trades are executed, or shrink the list as trades make their way into our Postgresql server (TODO still).

Implementation

During program execution, we cache users in our app when they submit/cancel orders, or have orders filled by someone else. While a user is in cache, we keep track of any new trades that occur, and once the user is evicted (either normal eviction or on controlled shutdown of the exchange) we write all new trades to Redis.

An order is converted into a trade in one of 2 ways, either the order is the filled, i.e it was pending and a new order came along and filled it, or the order is the filler--it is the new order that filled an old one.

We differentiate trades in this way on an account basis. In any trade, there is a filler order and a filled order, so one account will store the trade in their filled list, and the other in their filler list. We use the following keys: filled:user_id and filler:user_id.

User Accounts

User Accounts

• user:username

User ID map to Username

• id:user_id

Fetching Pending Orders

Pending Order Ref Count

• active_markets:id