Skip to content

Keep a note snapshots on disk with deduplication #270

@vitonsky

Description

@vitonsky

In #263 we moved to an SQLite DB.

We still keep note versions in DB with no deduplication.
The DB is single threaded, so the more data will be moved out of DB to load in parallel the better. Because we could query DB while await the content from the disk.

We could move a note versions into the storage with de-duplication based on sha3 hashes, and keep in DB only meta info.

Benefits of such refactoring is

  • Async loading of a note snapshots
  • Faster DB response. Less the DB lock time. Less data round trips between WASM - Worker - Main thread
  • Less RAM consumption, since our InMemory DB would offload the snapshots data on disk forever
  • Deduplication would decrease the disk usage

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions