Evolution of KeyValue System

This project is currently for personal study and research purposes only, so please use caution for commercial use.

What is TemplateKV

TemplateKV is an open-source, future-oriented, key-value (KV) infrastructure for databases. It offers dynamic data structure adjustment, elasticity to adapt to various levels of distribution, and the ability to perceive and operate on multiple dimensions, such as system, hardware, cluster, and load, in an asynchronous manner. Its focus is on providing high abstraction and high-performance solutions to meet the diverse underlying storage needs of different databases.

• High performance

  TemplateKV uses the latest concurrency technology and the latest popular hardware to make reading and writing silky smooth.

• Ultra Elasticity

  Use dynamic combination of cell-level operators to allow the runtime system to scale at a finer granularity.

• Multi-dimensional perception

  Dynamically senses multiple dimensions of operating system, hardware, workload, and user configuration to squeeze every machine dry.

• Asynchronous Arithmetic Manipulation

  Design different arithmetic logic for different layers, request, read/write, plan generation in the whole life cycle.

• Multiple protocol support

  Support mainstream data read/write protocols for community integration.

Architecture(pre)

Try TemplateKV

# Install Rust toolchain
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# Clone the repo
git clone https://github.com/devillove084/TemplateKV.git && cd TemplateKV
# Compile
cargo build --release
# Test
cargo install nextest # (Optional)Use next test framework
cargo test # (Optional)cargo nextest test

Getting Started

Deploying TemplateKV

• Understanding Deployment Modes

• Deploying a Standalone TemplateKV

• Deploying a Distribute TemplateKV

Connecting to TemplateKV

• How to Connect TemplateKV with MySQL Client
• How to Connect TemplateKV with ClickHouse HTTP Handler
• How to Execute Queries in Python

Loading Data into TemplateKV

• How to Load Data from Local File System
• How to Load Data from Remote Files
• How to Load Data from Amazon S3
• How to Unload Data from TemplateKV

Learning TemplateKV

• Design of Dynamic Data Structure

• Design of Stream Operators

• Design of Unified Consensus Protocol Abstraction Layer

Performance

• How to Benchmark TemplateKV using TPC-H

Contributors

@devillove084

@Sisphyus

Contribution

Welcome! Post Issues or submit a Pull Request, refer to theCONTRIBUTING.md.

TemplateKV follow the Contributor Covenant code of conduct.

Roadmap

• Roadmap v0.2

License

TemplateKV is released under the Apache License 2.0

When contributing to Databend, you can find the relevant license header in each code file.