When you absolutely must not lose any of your data.
rqlite is an easy-to-use, lightweight, distributed relational database, which uses SQLite as its storage engine. rqlite is simple to deploy, operating it is very straightforward, and its clustering capabilities provide you with fault-tolerance and high-availability. rqlite is available for Linux, macOS, and Microsoft Windows.
Check out the rqlite FAQ.
rqlite gives you the functionality of a rock solid, fault-tolerant, replicated relational database, but with very easy installation, deployment, and operation. With it you've got a lightweight and reliable distributed relational data store. Think etcd or Consul, but with relational data modelling also available.
You could use rqlite as part of a larger system, as a central store for some critical relational data, without having to run larger, more complex distributed databases.
Finally, if you're interested in understanding how distributed systems actually work, rqlite is a good example to study. Much thought has gone into its design and implementation, with clear separation between the various components, including storage, distributed consensus, and API.
rqlite uses Raft to achieve consensus across all the instances of the SQLite databases, ensuring that every change made to the system is made to a quorum of SQLite databases, or none at all. You can learn more about the design here.
- Trivially easy to deploy, with no need to separately install SQLite.
- Super-simple to use, with a straightforward HTTP API. A command-line interface is also available, as are various client libraries.
- Fully replicated production-grade SQL database, with full access to SQLite full-text search and JSON document support.
- Multiple options for node-discovery and automatic clustering, including integration with Kubernetes, Consul, etcd and DNS, allowing clusters to be dynamically created.
- Extensive security and encryption support, including node-to-node encryption.
- Choice of read consistency levels, and support for choosing write performance over durability.
- Optional read-only (non-voting) nodes, which can add read scalability to the system.
- A form of transaction support.
- Hot backups, as well as load directly from SQLite.
The quickest way to get running is to download a pre-built release binary, available on the Github releases page. Once installed, you can start a single rqlite node like so:
rqlited -node-id 1 ~/node.1
This single node automatically becomes the leader. You can pass
rqlited to list all configuration options.
docker run -p4001:4001 rqlite/rqlite
Check out the rqlite Docker page for more details on running nodes via Docker.
brew install rqlite
Forming a cluster
While not strictly necessary to run rqlite, running multiple nodes means you'll have a fault-tolerant cluster. Start two more nodes, allowing the cluster to tolerate failure of a single node, like so:
rqlited -node-id 2 -http-addr localhost:4003 -raft-addr localhost:4004 -join http://localhost:4001 ~/node.2 rqlited -node-id 3 -http-addr localhost:4005 -raft-addr localhost:4006 -join http://localhost:4001 ~/node.3
This demonstration shows all 3 nodes running on the same host. In reality you probably wouldn't do this, and then you wouldn't need to select different -http-addr and -raft-addr ports for each rqlite node.
With just these few steps you've now got a fault-tolerant, distributed relational database. For full details on creating and managing real clusters, including running read-only nodes, check out this documentation.
Node Discovery and Automatic Clustering
rqlite can use Consul, etcd, and DNS, for node discovery. This allows nodes to automatically connect and form a cluster. This can be much more convenient, allowing clusters to be dynamically created. Check out the documentation for more details.
Check out the Kubernetes deployment guide.
Let's insert some records via the rqlite CLI, using standard SQLite commands. Once inserted, these records will be replicated across the cluster, in a durable and fault-tolerant manner. Your 3-node cluster can suffer the failure of a single node without any loss of functionality or data.
$ rqlite 127.0.0.1:4001> CREATE TABLE foo (id INTEGER NOT NULL PRIMARY KEY, name TEXT) 0 row affected (0.000668 sec) 127.0.0.1:4001> .schema +-----------------------------------------------------------------------------+ | sql | +-----------------------------------------------------------------------------+ | CREATE TABLE foo (id INTEGER NOT NULL PRIMARY KEY, name TEXT) | +-----------------------------------------------------------------------------+ 127.0.0.1:4001> INSERT INTO foo(name) VALUES("fiona") 1 row affected (0.000080 sec) 127.0.0.1:4001> SELECT * FROM foo +----+-------+ | id | name | +----+-------+ | 1 | fiona | +----+-------+
You can learn more about rqlite performance, and how to improve it, here.
By default rqlite uses an in-memory SQLite database to maximise performance. In this mode no actual SQLite file is created and the entire database is stored in memory. If you wish rqlite to use an actual file-based SQLite database, pass
-on-disk to rqlite on start-up.
Does using an in-memory SQLite database put my data at risk?
Since the Raft log is the authoritative store for all data, and it is stored on disk by each node, an in-memory database can be fully recreated on start-up from the information stored in the Raft log. Using an in-memory database does not put your data at risk.
In-memory databases are currently limited to 2GiB (2147483648 bytes) in size. You can learn more about possible ways to get around this limit in the documentation.
Because rqlite peforms statement-based replication certain non-deterministic functions, e.g.
RANDOM(), are rewritten by rqlite before being passed to the Raft system and SQLite. To learn more about rqlite's support for non-deterministic functions, check out the documentation.
This has not been extensively tested, but you can directly read the SQLite file under any node at anytime, assuming you run in "on-disk" mode. However there is no guarantee that the SQLite file reflects all the changes that have taken place on the cluster unless you are sure the host node itself has received and applied all changes.
In case it isn't obvious, rqlite does not replicate any changes made directly to any underlying SQLite file, when run in "on disk" mode. If you change the SQLite file directly, you may cause rqlite to fail. Only modify the database via the HTTP API.
SQLite dot-commands such as
.tablesare not directly supported by the API, but the rqlite CLI supports some very similar functionality. This is because those commands are features of the
sqlite3command, not SQLite itself.
For reliable operation, particularly in production, it's important to monitor rqlite. You can learn how to check rqlite status and diagnostics here.
Backup and restore
You can learn about securing access, and restricting users' access, to rqlite here.
rqlite Slack Channel
Join the Slack channel to learn more about rqlite.
There is a Google Group dedicated to discussion of rqlite.
How do I pronounce rqlite? For what it's worth I try to pronounce it "ree-qwell-lite". But it seems most people, including me, often pronouce it "R Q lite".