Skip to content

Architecture :: Data Stores

Jump to bottom
Angel Sanadinov edited this page Jun 9, 2020 · 1 revision

Maybe Consistent

The approach taken to maintaining the state of the entity database is one of unenforced consistency. Relationships between entities exist (a device belongs to a user), but they may not be always valid. For example, removing a user does not cause the removal of its associated devices (as part of the same operation). Eventually, a maintenance process should run and clean up orphan entries.

This is done primarily to keep the original request relatively simple (a single entry from a single table/database is removed or marked as deleted). As a nice side effect, this approach keeps child entries around for a while after the parent is discarded; a malicious or accidental deletion will not cause data loss right away.

Core vs Server

The core submodule defines a set of entities used for managing the data exchange. The server submodule also defines its own set of entities, used for dealing with users, devices, backups and scheduling. Since the server is the actual, runnable service, it includes one or more core endpoints and is therefore responsible for handling the core persistence as well (deciding which backends are to be used, how they are initialized, etc).

Backends

Various data stores are used throughout the codebase for keeping track of entities such as nodes, users and devices. They do not depend on any storage implementation or database; the expectation is that a storage backend of the required type will be provided on initialization.

Backend Types:
  • EventLogBackend - receives a stream of events and updates its state; state updates can be streamed to subscribers
  • KeyValueBackend - data is stored with its associated key and can be retrieved back with that key
  • StreamingBackend - same as KeyValueBackend but for storing binary data that is streamed back to the caller; meant for storing crates as they are not expected to fit entirely in memory
Backends:
Clone this wiki locally