Roadmap

[ndragazis]

Task

Implement a simple persistent key/value store using the seastar framework

Requirements

1. Dataset:
   1. Keys: UTF-8 strings, up to 255 bytes
   2. Values: UTF-8 strings, unlimited length
   3. Data sharded among core based on keys
2. API:
   1. Insert / update key-value
   2. Query a key
   3. Query sorted list of keys
   4. Delete a key
3. LRU cache

[ndragazis]

Action Items

• [api] Design the REST API #2
• [http-server] Create an HTTP server with Seastar #3
• [storage] Store data on disk #4
• [seastar] Use the Seastar framework for async IO #5
• [seastar-sharding] Shard data among cores #6
• [cache] Add a caching layer #7

[ndragazis]

Backlog

Adding here some random things that I thought of while writing code (these should be organized in existing or new issues):

• Do not flush an empty memtable into an SSTable upon termination.
• The list of active/immutable memtables is broken right now. First of all, it needs to become asynchronous (the KVStore should be able to serve requests while flushing them in the background) and the flushes should happen in chronological order (mutable memtable included).
• Update the memtable size on key updates.
• Improve the file format for WALs and SSTables. The values can be any kind of string. Currently, my code assumes that they do not contain newlines.
• Can we combine async with RAII? For example, create async constructors/destructors? Alternatively, can we call async code from synchronous code (i.e., block until the async operation resolves)?
• Proper exception handling.
• Do we need any synchronization for the memtable and sstable lists?