Unorthodox Key Value store (and less).
- Shroud is a key-value store with an emphasis on asynchronous writes.
- Shroud seperates interface and implementation to enable modularity.
- Shroud is designed for applications, that insert, but don't update records.
Features:
- Read-After-Write consistent: As soon as
SetEntry()returns, the inserted Entry is readable withGetValue()andHasValue(). - Eventual Durability:
SetEntry()returns as early as possible, whilst the Entry is persisted in the Background. - Low write latency: Because the data is persisted in background,
SetEntry()usually takes 10-20ms1, depending on write concurrency to return.
1Currently the backend uses db.Batch(...) instead of db.Update(...) on the transaction bypass cache. This hurts latency at single-threaded writes,
but improves performance at multi-threaded workloads.
How it works:
The backend is based on badger and go-unstable/bbolt.
Badger serves as the persistent datastore, and Bolt serves as the transaction bypass cache. Bolt is initialized with NoSync and the DB_WriteSharedMmap-flag,
this ensures low commit latency and high write throughput, at the expense of Durability. Writing Entries into a badger-database will cause a certain latency
until the data is visible to other readers 1. Shroud is designed to work around this issue 2. When a new Entry is inserted with
SetEntry(), the entry is written into the transaction bypass cache, rendering it instantly observable (and thus, readable). Then it is submitted to a worker
thread writing the Entry to the persistent datastore. Once the Entry is committed, it is evicted from the transaction bypass cache, as it can now be retrieved
from the persistent datastore.
1As Badger is a transactional datastore, updates and inserts become visible, only after the transaction is fully committed and durable.
2This issue is use-case specific. For most applications, transactional semantics are desired, which is: New Data is observable after it is durable.