Skip to content
Decentralized, GDPR-compliant, Trusted, SQL Database with Blockchain features
Branch: develop
Clone or download
Latest commit b6ebe17 May 24, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github Update stale.yml Apr 29, 2019
api Remove imports blank lines Feb 25, 2019
bin Rename docker observer to explorer. May 24, 2019
blockproducer Make tx log informative May 15, 2019
chainbus Add cloudflare key Dec 5, 2018
client Fix repeating line ending. May 21, 2019
cmd Add SQLChainPeriod while generate config. Get ListenAddr from -miner … May 24, 2019
conf Add SQLChainPeriod while generate config. Get ListenAddr from -miner … May 24, 2019
consistent
crypto Add AES-128-CBC PKCS#7 end to end encryption and decryption May 17, 2019
docker Add android release feature May 14, 2019
kayak Separate storage commit error in follower node May 7, 2019
keys add cpu_count, loadavg metric. normarlize crucial metrics to "mem_ava… Aug 1, 2018
logo Add bench chart in README.md (#318) May 15, 2019
merkle Format godoc Feb 25, 2019
metric Merge remote-tracking branch 'origin/develop' into feature/rpc_improve Apr 1, 2019
naconn Rename noconn to naconn which is "node aware connection" Apr 8, 2019
pow/cpuminer Minor fix Apr 1, 2019
proto Add config field and node info filed for direct RPC server Apr 4, 2019
route Update vendor and ipv6seed May 21, 2019
rpc Fix reconnect test Apr 8, 2019
sqlchain Rename -no-password to -with-password and reverse it's meaning. May 14, 2019
storage Format godoc Feb 25, 2019
test Rename docker observer to explorer. May 24, 2019
twopc Fix ETLS permission test Apr 2, 2019
types Fix account type IsValid bug. May 23, 2019
utils Upgrade msgpack dep to avoid broken pipe panic May 7, 2019
vendor Merge branch 'develop' into feature/parser_double_quote_ident May 21, 2019
worker Fix race and return error via channel May 13, 2019
xenomint Add test case for double quoted table/column name ident May 16, 2019
.dockerignore Add keystore in conf to docker ignore list Mar 20, 2019
.gitignore Update gitignore for keystore temp files including sqlite wal/shm Mar 21, 2019
.gitlab-ci.yml Update ci previous version to v0.6.0. May 15, 2019
.gitmodules use readonly GNTE submodule Aug 2, 2018
.travis.yml Remove bin/cql-utils in release pkg Mar 28, 2019
CHANGELOG.md Add v0.6.0 changelog May 15, 2019
Gopkg.lock Merge branch 'develop' into feature/parser_double_quote_ident May 21, 2019
Gopkg.toml Upgrade msgpack dep to avoid broken pipe panic May 7, 2019
LICENSE update license to Apache2.0 May 5, 2018
Makefile Fix android release Makefile script typo May 14, 2019
README-zh.md Update drivers URL May 21, 2019
README.md Update drivers URL May 21, 2019
alltest.sh Merge remote-tracking branch 'origin/develop' into feature/beta_upstream Mar 27, 2019
analysisVendor.sh add analysisVendor.sh to generate dep tree png May 2, 2018
cleanupDB.sh Update cleanup db script Mar 22, 2019
codecov.yml Combine delete path func. Mar 5, 2019
docker-compose.yml Rename docker observer to explorer. May 24, 2019
fmtGoDoc.sed Format comment Aug 29, 2018
fmtGoImports.sh Add imports format script and format all codes Feb 25, 2019
genMarshalHash.sh Keep HashStablePack latest before generate code Jan 21, 2019

README.md

Go Report Card Coverage Build Status License GoDoc homebrew

中文简介

CovenantSQL(CQL) is a Decentralized, GDPR-compliant, Trusted, SQL database with Blockchain features:

  • ServerLess: Free, High Availabile, Auto Sync Database Service for Serverless App
  • GDPR-compliant: Zero pain to be GDPR-compliant.
  • SQL: Most SQL-92 support.
  • Decentralize: Running on Open Internet without Central Coordination.
  • Privacy: Access with Granted Permission and Encryption Pass.
  • Immutable: Query History in CQL is Immutable and Trackable.
  • Permission: Column Level ACL and SQL Pattern Whitelist.

What is CQL?

  • Open source alternative of Amazon QLDB
  • Low cost DBaaS
  • Just like filecoin + IPFS is the decentralized file system, CQL is the decentralized database

Quick Start

CQL client supports:

  • macOS X 10.9+
  • Linux 2.6.23+ (x86, x86_64, armeabi-v7a, arm64-v8a)
Developer Guide

MacOS

  • 🍺 Homebrew users can just run:

    brew install cql
  • non-Homebrew users can run:

    sudo bash -c 'curl -L "https://mac.gridb.io/cql" | tar xzv -C /usr/local/bin/ --strip-components=1'

Linux

  • Just run:

    sudo bash -c 'curl -L "https://linux.gridb.io/cql" | tar xzv -C /usr/local/bin/ --strip-components=1'

To continue: TestNet Quickstart

SDKs

CovenantSQL Testnet is already released, have a try.

How CQL works

3 Layers Arch

CovenantSQL 3 Layer design

  • Layer 1: Global Consensus Layer (the main chain, the middle ring in the architecture diagram):
    • There will only be one main chain throughout the network.
    • Mainly responsible for database Miner and the user’s contract matching, transaction settlement, anti-cheating, shard chain lock hash and other global consensus matters.
  • Layer 2: SQL Consensus Layer (shard chain, rings on both sides):
    • Each database will have its own separate shard chain.
    • Mainly responsible for: the signature, delivery and consistency of the various Transactions of the database. The data history of the permanent traceability is mainly implemented here, and the hash lock is performed in the main chain.
  • Layer 3: Datastore Layer (database engine with SQL-92 support):
    • Each Database has its own independent distributed engine.
    • Mainly responsible for: database storage & encryption, query processing & signature, efficient indexing.
For more details

Consensus Algorithm

CQL supports 2 kinds of consensus algorithm:

  1. DPoS (Delegated Proof-of-Stake) is applied in Eventually consistency mode database and also Layer 1 (Global Consensus Layer) in BlockProducer. CQL miners pack all SQL queries and its signatures by the client into blocks thus form a blockchain. We named the algorithm Xenomint.
  2. BFT-Raft (Byzantine Fault-Toleranted Raft)bft-raft is applied in Strong consistency mode database. We named our implementation Kayak. The CQL miner leader does a Two-Phase Commit with Kayak to support Transaction.transaction

CQL database consistency mode and node count can be selected in database creation with command cql create '{"UseEventualConsistency": true, "Node": 3}'

Comparison

Ethereum Hyperledger Fabric Amazon QLDB CovenantSQL
Dev language Solidity, ewasm Chaincode (Go, NodeJS) ? Python, Golang, Java, PHP, NodeJS, MatLab
Dev Pattern Smart Contract Chaincode SQL SQL
Open Source Y Y N Y
Nodes for HA 3 15 ? 3
Column Level ACL N Y ? Y
Data Format File Key-value Document Filefuse, Key-value, Structured
Storage Encryption N API Y Y
Data Desensitization N N N Y
Multi-tenant DIY DIY N Y
Throughput (1s delay) 15~10 tx/s 3500 tx/s ? 11065 tx/s (Eventually Consistency)
1866 tx/s (Strong Consistency)
Consistency Delay 2~6 min < 1 s ? < 10 ms
Secure for Open Internet Y N Only in AWS Y
Consensus PoW + PoS(Casper) CFT ? DPoS (Eventually Consistency)
BFT-Raft (Strong Consistency)

FootNotes

  • BFT-Raft: A CQL leader offline needs CQL Block Producer to decide whether to wait for leader online for data integrity or promote a follower node for availability. This part is still under construction and any advice is welcome.

  • Transaction: Talking about ACID, CQL has full "Consistency, Isolation, Durability" and a limited Atomicity support. That is even under strong consistency mode, CQL transaction is only supported on the leader node. If you want to do "read v, v++, write v back" parallelly and atomically, then the only way is "read v from the leader, v++, write v back to leader"

  • FUSE: CQL has a simple FUSE support adopted from CockroachDB. The performance is not very ideal and still has some issues. But it can pass fio test like:

    fio --debug=io --loops=1 --size=8m --filename=../mnt/fiotest.tmp --stonewall --direct=1 --name=Seqread --bs=128k --rw=read --name=Seqwrite --bs=128k --rw=write --name=4krandread --bs=4k --rw=randread --name=4krandwrite --bs=4k --rw=randwrite

Network Stack

DH-RPC := TLS - Cert + DHT

Layer Implementation
RPC net/rpc
Naming Consistent Secure DHT
Pooling Session Pool
Multiplex smux
Transport Security Enhanced TLS
Network TCP or KCP for optional later

Test Tools we use

Papers

Our team members published:

that inspired us:

Performance

Strong consistency bench result (2 miners, 8 core aws c5.2xlarge): CovenantSQL bench

As seen above, the concurrency pressure on the database increased gradually in the first 5 hours, and the write TPS also increased. When the TPS no longer grows, the concurrent pressure is maintained for 100 hours.

Demos

Use cases

Traditional App & ĐApp use cases

Traditional App

Privacy data

If you are a developper of password management tools just like 1Password or LastPass. You can use CQL as the database to take benefits:

  1. Serverless: no need to deploy a server to store your user's password for sync which is the hot potato.
  2. Security: CQL handles all the encryption work. Decentralized data storage gives more confidence to your users.
  3. Regulation: CQL naturally comply with GDPR.

IoT storage

CQL miners are deployed globally, IoT node can write to nearest CQL miner directly.

  1. Cheaper: Without passing all the traffic through a gateway, you can save a large bandwidth fee. And, CQL is a shared economic database which makes storage cheaper.
  2. Faster: CQL consensus protocol is designed for Internet where network latency is unavoidable.

Open data service

For example, you are the most detailed Bitcoin OHLC data maintainer. You can directly expose an online SQL interface to your customers to meet a wide range of query needs.

  1. CQL can limit specific SQL query statements to meet the needs while also balancing data security;
  2. CQL can record SQL query records on the blockchain, which is very convenient for customers to check their bills for long-tail customers and billing, like this
  3. For customers with high performance requirements, Slave nodes can be deployed at the customer to meet the needs of customers with low latency queries while enabling almost real-time data updates.

Secure storage

Thanks to the CQL data history is immutable, CQL can be used as a storage for sensitive operational logs to prevent hacking and erasure access logs.

ĐApp

Storing data on Bitcoin or Ethereum is quite expensive ($4305 / MB on Ethereum 2018-05-15). Programming is very complicated due to the lack of support for structured data storage. CQL gives you a low-cost structured SQL database and also provides more room for ĐApp to exchange data with real-world.

TestNet

Contact

You can’t perform that action at this time.