Skip to content

jiangxinmeng1/logstore

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

68 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Log-structured data store

Design Doc

Requirements

  • Rotatable
  • Revertable
  • Commit pipeline
  • High concurrency
  • Crash-safe
  • Friendly interface, easy to integrate

RFC

TODO

Example

The full outline of wal commit pipeline operation is as follows:

                           |--------------- Guarded ------------|   |------------- Concurrent ------------|
  Insert                   |<------------- Preparing ---------->|   |<------------ Committing ------------|
 ---------> OnRequest ---> Alloc LSN ---> Prepare insert buffer ---> Fill insert buffer --> Wait sync done ----->  Return response
                                      |                         |                                            |
                                      |                         |                                            |
                                       -> Prepare log to WAL ==>                                             |(Sync done)
                                                                |                                            |
                                                                |                                            |
WalCommitter: --- Commit ---> Commit ===> Commit ---> Commit ===> Commit logging to WAL ---> Commit ---> ... |
                                        |                       |                               |            |
                                        |                       |                               |            |
WalSyncer   :                    ----- Sync -----------------> Sync -------------------------> Sync ------------------> ...


Used as a metadata-like storage engine

TODO

Used as a columnar storage database storage engine

TODO

About

A log-structured data store

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Go 100.0%