Skip to content
grantr edited this page Oct 12, 2012 · 2 revisions

Bootstrap servers provide a durable storage api for streams. The durable storage itself is delegated to external k/v storage such as cassandra or dynamodb. This storage should support efficient range scans.

Clients that request transactions that have been evicted from the relays' buffers are redirected to the bootstrap server. The bootstrap server keeps a window of older transactions for streaming to clients and periodically rolls up transactions into snapshots. Snapshots are uploaded to S3. Clients starting from sequence zero are redirected to a snapshot url.

Snapshots are best suited for streams that represent a database with records having ids. Strictly event-based streams can still use snapshots to reduce load on transaction storage, but they won't get the efficiency benefit of rolled up snapshots.

TODO api, interaction with relays

Clone this wiki locally