- PR #1502: Create temporary SQLite files in same directory as actual SQLite file.
- PR #1498: rqlite shell
.helpshould show commands in alphabetical order. - PR #1499: Bump golang.org/x/crypto from 0.16.0 to 0.17.0.
- PR #1501: Upgrade rqlite disco-client to handle IPv6 addresses. Fixes issue #1500. Thanks @jtackaberry
- PR #1497: Don't re-open and close database on shutdown. It's unnecessary.
This version sees the minor version incremented to indicate the importance of this release. Bcrypted passwords were not secure in the Credentials Configuration, as they could also be used directly as passwords. This has been fixed, and bcrypted passwords are no longer supported going forward. All users using bcrypted hashes in Credential files should upgrade to this release and migrate to using plaintext passwords in those files.
If you were using bcrypted passwords, this is probably a breaking change. You should recreate any Credentials Configuration files to use plaintext passwords instead of bcrypt hashes, and ensure the Credentials Configuration files are secured from unauthorized access.
- PR #1492: Remove faulty bcrypt hashed password support. Fixes issue #1488. Thanks @jtackaberry
- PR #1494: Confirm restored data looks like a SQLite file.
- PR #1490: Guard against
nilHistory in rqlite shell. Fixes issue #1486.
- PR #1489: Add
.bootcommand to rqlite shell, to support Booting nodes.
- PR #1487: Small improvements to rqlite shell.
This release introduces a new, high-performance, initialize-from-SQLite option. Known as Boot loading, it allows an rqlite node to be seeded with large (multi-GB) datasets, with the time taken to seed the node determined only by your disk performance.
- PR #1485: Add Boot loading.
- PR #1484: Remove ability to trigger chunked-loading.
Release v8.0.6 protects against a malformed database being loaded as a result of using /db/load. All 8.x users should upgrade to this release.
- PR #1482: Don't load an invalid database.
- PR #1481: Revert to simpler one-shot load for SQLite files.
- PR #1471, PR #1472: Refactor query parameters into own code.
- PR #1473:
go modupdates. - PR #1478: Upgrade Go to 1.21 in
go.mod. Fixes issue #1476. - PR #1475: Minor rqlite CLI improvements.
- PR #1480: Don't completely truncate the log, due to apparent unsigned overflow issues in Raft log.
- PR #1466: Close all dechunkers at Store shutdown.
- PR #1467: Introduce explicit FSM type.
- PR #1468, PR #1469: Fix failure-to-restart issue after chunked loading.
Fix an edge case related to Raft Snapshotting when a chunked load is in progress at the same time.
- PR #1456: Standardize on chunk size.
- PR #1456: Set
TrailingLogs=0to truncate log during user-initiated Snapshotting. - PR #1462: Refactor redirect logic in HTTP service.
- PR #1463: Synchronize Snapshotting and chunked loading.
- PR #1464: Handle snapshotting of empty WAL files.
- PR #1465: Move uploader goroutine into Uploader.
This release fixes an edge case issue during restore-from-SQLite. It's possible if a rqlite system crashes shortly after restoring from SQLite it may not have loaded the data correctly.
- PR #1456: Wrap Snaphot Store FullNeeded logic in a function.
- PR #1457: Allow FullNeeded to be explicity set to true.
- PR #1458: Perform full snapshot after chunked load.
This release introduces support for much larger data sets. Previously the Raft snapshotting process became more memory intensive and time-consuming as the SQLite database became larger. This set an practical upper limit on the size of the SQLite database. With the 8.0 release rqlite has been fundamentally redesigned such that snapshotting consumes approximately the same amount of resources, regardless of the size of the SQLite database.
This release also eases operations, as well as adding new features and bug fixes.
Release 8.0 supports (mostly) seamless upgrades from the 7.x series, and upgrading from 7.x has been tested. However, it is still strongly recommended you backup any production cluster before attempting an upgrade. A more conservative approach would be to create a brand new 8.0 system, and load your backup into that cluster. Then switch production traffic over to the new 8.0 cluster.
8.0 and 7.x nodes should be able to interoperate, so a rolling upgrade should work fine as long as all nodes are fully caught up with the Leader node. Note you also cannot join a new 8.x node to a pre-existing 7.x cluster. Otherwise upgrade should operate but, again, it is strongly recommended you test this first. It is also not recommended that you run a cluster with a mix of 7.x and 8.0 code for any significant length of time, just the time required for a rolling upgrade.
Important things to note if you decide to upgrade an existing 7.x system:
- Backup your 7.x cluster first.
- it is strongly recommended you upgrade your 7.x cluster to the 7.21.4 release before upgrading to the 8.0 series.
- 8.0 always runs with an on-disk database, in-memory databases are no longer supported. Improvements made late in the 7.0 series mean there is little difference in write performance between in-memory and on-disk modes, but supporting both modes just meant confusion and higher development costs. If you were previously running in in-memory mode (the previous default), you don't need to do anything. But if you were previously passing
-on-disktorqlitedso that rqlite ran in on-disk mode, you must now remove that flag. - When forming a new cluster using 8.0, pass the Raft addresss of the remote node to the
-joincommand, not the HTTP API address. If your cluster is already formed, upgrades will work without changing anything (-joinoptions are ignored if nodes are already members of a cluster). You may need to change any scripting or automatic-configuration generation however. - Bcrypted password hashes are no longer supported, due to security flaws in the 7.x release. You should regenerate any Credentials file, and use plaintext passwords only (and prevent unauthorized access to the Credentials file).
- A few rarely, if ever, used
rqlitedcommand-line flags have been removed. These flags just added operational overhead, while adding little value.
- PR #1362: Enable SQLite FTS5. Fixes issue #1361
- PR #1405: Support a configurable HTTP connection timeout in the rqlite CLI. Thanks @jtarchie
- PR #1418: Add basic CORS support. Fixes issue #687. Thanks @kkoreilly
- PR #1422: Add mTLS support to rqlite CLI. Fixes issue #1421
- PR #1427: Upgrade to SQLite 3.44.0.
- PR #1433: Support an optional better form for the
nodes/output. Fixes issue #1415 - PR #1447: Remove-on-shutdown now supports authentication.
- PR #1451: Support optional
VACUUMof SQLite database file before upload to Cloud storage. - PR #1452: Support optional
VACUUMof requested backup SQLite file.
- PR #1368: Switch to always-on expvar and pprof.
- PR #1337: Store can now load from an io.Reader.
- PR #1339, PR #1340, PR #1341: Add
LoadRequestchunker/dechunker. - PR #1343: Remove previously-obsoleted supported command-line options.
- PR #1342: Integrate chunked-loading, applying to auto-restore from the Cloud.
- PR #1347: Migrate HTTP layer to chunked loading.
- PR #1355: Database layer can run an integrity check.
- PR #1385: Remove support for in-memory databases.
- PR #1360: 'go mod' updates, and move to go 1.21.
- PR #1369, PR #1370: Use singleton, sync'ed, random source.
- PR #1367: Move to a WAL-based Snapshot store, which unlocks support for much larger data set support.
- PR #1373: Remove compression-control command-line options.
- PR #1377: Automatically upgrade 7.x snapshots.
- PR #1380: Compress snapshots when transmitting over the network.
- PR #1382, PR #1383: Add basic stats for Snapshot store.
- PR #1384: Use less-racy function to retrieve Leader Address and ID.
- PR #1386: Ensure full sync'ing and closing of files during WAL replay.
- PR #1388: Ensure databases open with WAL checkpoint disabled. Thanks @benbjohnson.
- PR #1390: Add Restore functions to Snapshot Store.
- PR #1394: Use only one RW database connection.
- PR #1395: More DB-level and Snapshotting statistics.
- PR #1399: Better trailing flags error message.
- PR #1404: Add an interface between Store and Snapshot Store.
- PR #1410, PR #1412: Implement simpler WAL-based snapshotting.
- PR #1413: Remove
-raft-no-freelist-synccommand line flag. - PR #1420, PR #1431: Nodes join a cluster using the Raft address, not the HTTP API.
- PR #1426: 'go mod' updates, including moving to Raft 1.6.
- PR #1430: Check that any supplied Join addresses are not HTTP servers.
- PR #1437, PR #1438, PR #1439: Actually timeout if needed during
nodes/access. Fixes issue #1435. Thanks @dwco-z - PR #1440: Add a Compacting WAL rewriter. Thanks @benbjohnson.
- PR #1441, PR #1443: Integrate Compacting WAL writer
- PR #1444: Trivial clean-ups related to backups.
- PR #1445: Count Snapshot upgrades.
- PR #1336: Remove on-disk-startup control. It's no longer needed as on-disk performance is now very close to in-memory performance, thanks to the switch to synchronous off mode and the use of the SQLite WAL.
- PR #1329: Try a different version of V2 Snapshot codec.
- PR #1332: Upgrade dependencies.
- PR #1333: Set "types" for expressions e.g.
COUNT. Fixes issue #1330
- PR #1321: Check for errors in responses during load testing.
- PR #1323: Add codec for v2 snapshots.
- PR #1324: Close Snapshot after we're finished restoring from it.
- PR #1325: Handle getting an error when asking for database stats.
This release changes the mode of SQLite, when rqlite is running in on-disk mode. SQLite now runs in WAL mode, when previously it was in DELETE mode. Testing shows this results in a ~30% increase in write-performance.
- PR #1314: More preparations for WAL mode when running on-disk.
- PR #1315, PR #1316: Enable WAL when running in on-disk mode.
- PR #1317: DB-layer now supports WAL replay.
- PR #1318: Periodically record actual applied index. Addresses a possible edge case with on-disk startup.
- PR #1311: Support 'key' param on the
/statusHTTP endpoint. - PR #1313: Sysdump now retrieves data from all nodes if possible.
- PR #1309: Factor Snapshot creation into own module.
- PR #1305: Upgrade dependencies via
go get. - PR #1306: Add some (currently unused) WAL control code.
- PR #1307, PR #1308: Add full WAL-support to database layer. Not yet enabled by full application.
- PR #1302: Add some important PRAGMA state to database-level status reporting.
- PR #1303: Reduce disk space usage by retaining only a single Raft SQLite snapshot.
This release changes the "syncing" mode SQLite uses to OFF when rqlite runs in "on-disk" mode. The SQLite docs state that this risks database corruption in the event of a crash, but that's a non-issue for rqlite, as rqlite always deletes any SQLite database on startup and rebuilds it from the Raft log. Testing shows this change results in (at least) a 3x speed-up in write performance when operating in "on-disk" mode.
- PR #1301: Set synchronous mode to
OFFfor SQLite on-disk files.
- PR #1298: Move FSMSnapshot to own source file.
- PR #1300: Check for WAL-enabled SQLite files during load and restore.
- PR #1296: Use correct connection when checking a SQL statement for "read-only" status, otherwise "database locked" could result. Also refactors much of the DB-level unit tests.
- PR #1291: Allow bootstrap-join even with preexisting state. Fixes issue #1290
- PR #1288: Upgrade to SQLite 3.42.0.
- PR #1286: More validation of passed-in network addresses.
- PR #1278: Enable support for more CPU architectures. Fixes issue #901
- PR #1275: Node-removal performs retries.
- PR #1277: Upgrade dependencies.
- PR #1279: Move to custom image to speed up testing on CircleCI.
- PR #1269: Add WaitForRemoval() to Store.
- PR #1270: Confirm self-removal changes cluster config.
- PR #1272: Refactor node self-removal on shutdown.
- PR #1273: Make WaitForLeader() more consistent.
- PR #1274: Do DNS boostrapping even if there is pre-existing state. Fixes issue #1247
This release also includes some small logging improvements, related to node-shutdown.
- PR #1266, PR #1268: Add network information to
/status. - PR #1267: Reduce self-remove timeout to 5 seconds.
This release adds a new HTTP endpoint, located at /db/request. This endpoint accepts both read and write requests, including mixing both together in a single request. When requests are sent to this endpoint, rqlite will automatically perform the correct operation for each SQL statement in the request. This endpoint may be more convenient for some use cases, and means that client code doesn't have to decide on whether it should send requests to /db/execute or /db/query. Many thanks to VOXO for funding this development.
7.18.0 also includes some small improvements to diagnostics and instrumentation.
- PR #1256, PR #1258, PR #1260, PR #1261, PR #1265: Support a Unified Endpoint, which can accept both read and write requests. Fixes issue #263.
- PR #1253: Node optionally removes itself from the cluster automatically when gracefully shutting down. See the documentation for full details.
- PR #1252: Stop the HTTP server first on shutdown.
This release introduces the ability for a node to automatically recover from a backup in AWS S3. See the documentation for full details.
- PR #1243: Support automatically restoring from AWS S3. Thanks to VOXO for funding this development.
- PR #1244: Disco configs now support Environment variable expansion.
- PR #1245: Support continuing on failure to download from AWS S3.
- PR #1246: Add support for custom S3 endpoint.
- PR #1239: Remove erroneous scaling factor from etcd and Consul reporting service.
- PR #1240: Add support for controlling and reading Store readiness.
- PR #1241: Check Store is ready in key places.
- PR #1248: Refactor autobackup and autorestore.
- PR #1249: Retry certain cluster-client operations.
- PR #1233: Close file handle after upload.
- PR #1229, PR #1232: Add support for automatic backups to AWS S3. Many thanks to VOXO for funding this development.
- PR #1218: Check for more possible errors in peers.json. Thanks @Tjstretchalot
- PR #1220: Support Notify over Raft connection.
- PR #1221: Support Join over Raft connection.
- PR #1222: Joiner expands all targets to include protocols.
- PR #1224: Fix credentials load error checking. Thanks @phmx
- PR #1227: Upgrade dependencies, including moving to Hashicorp Raft 1.5.
This release is the first to includes various bug fixes and optimizations thanks to running much of the code through Chat GPT-4, most of which are not explicitly listed in the CHANGELOG, but you can check the commit history for details. Future releases of rqlite will probably include more such changes.
- PR #1179: go mod updates.
- PR #1180: Support large numbers in requests.
- PR #1186: Improve read-only (non-voting) node management. Fixes issue #1182.
- PR #1189: Migrate to a Protobuf data model for Join Requests.
- PR #1190: Migrate to a Protobuf data model for Notify Requests.
- PR #1207: Decompose end-to-end testing into distinct CircleCI jobs.
- PR #1218: Check for more possible errors in peers.json. Thanks @Tjstretchalot
- PR #1174: Fix command-line help for x509 resources.
- PR #1178: Fix parsing of Named Parameters with
NULLas value. Fixes issue #1177. Thanks @wellescastro
This release sees the addition of mutual TLS support, and corrects some other issues related to the use of TLS and Certificate Authority configuration.
- PR #1171: Support mutual TLS for both HTTP connections and inter-node traffic. See issue #1167. Thanks @otto-dev
- PR #1173: go mod updates.
- PR #1156: Better error message when rqlite shell can't connect to a node.
- PR #1162: Fix Consul-based discovery issue when using TLS.
- PR #1163: go mod updates.
- PR #1146: Delete history file if
RQLITE_HISTFILESIZEenvironment variable is zero. Fixes issue #1145. Thanks @jamielinux - PR #1148: go mod updates.
- PR #1141: Store and load CLI history across sessions.
- PR #1135: Silently ignore self-joins if nothing has changed on the joining node.
- PR #1136: Stop HTTP server gracefully on node shutdown.
- f6c4b17: By default, Leader node will stepdown if that node is shutting down.
- PR #1139: Cache hashed passwords. Fixes issue #1138.
- PR #1140: Use SQLite with corrected in-memory database locking (SQLite forum post). Fixes issue #1103.
- PR #1121: Transparently forward node-removal requests to Leader.
- PR #1125: Support fetching a subset of expvar information.
- PR #1134: Support stepping down as Leader before shutting down.
- PR #1120: go mod updates.
- PR #1126: Add Queue instrumentation.
- PR #1127: Clearer Queued Writes loop logic.
- PR #1128: Add ResetStats to some modules.
- PR #1114, PR #1118: Support automatically removing non-reachable nodes after a configurable period. Fixes issue #728.
- PR #1116, PR #1117: Support associative form for query responses. Fixes issue #1115.
- PR #1097, PR #1110: Start HTTP server as soon as possible after launch.
- PR #1098: Bootstrapper doesn't need to know the bootstrap-expect value.
- PR #1099: Add explicit
.exitoption to CLI. - PR #1102: Use BasicAuth redaction functionality from the standard library.
- PR #1108: Add more upgrade testing.
- PR #1109: Higher Queued Writes defaults
- PR #1113: go mod updates
- PR #1096: Upgrade to SQLite 3.39.4.
- PR #1094: Update packages to resolve CVE-2022-32149 and CVE-2022-27664. Thanks @sgalsaleh
- PR #1095: Log, and add to version output, the SQLite release.
- PR #1096: go mod updates.
This release addresses a shortcoming in inter-node communications. Nodes now consistently encode the length of those communications using 8-byte values. If any node in a cluster is upgraded to this release, then all nodes in that cluster should be.
- PR #1089: Move to 8-byte Protobuf lengths for cluster communications. Fixes issue #1088.
- PR #1090: Correct error handling for remote load and backup failures.
- PR #1086: Restoring via follower should have same HTTP response body.
- PR #1087: Notified and joined node checks address resolution.
This release makes it more convenient to load SQLite files directly into rqlite, as any node can now process the request. For this to work however, all nodes in your cluster must be running 7.9.0 (or later). Otherwse 7.9.0 is fully compatible with earlier release, so a rolling upgrade process is an option.
- PR #1084: Transparently forward SQLite data Restore requests to Leaders.
- PR #1085: Improved logs during joining.
This release makes it more convenient to retrieve a backup. Now any node can provide a backup of the underlying SQLite database. For this to work however, all nodes in your cluster must be running 7.8.0 (or later). Otherwse 7.8.0 is fully compatible with earlier release, so a rolling upgrade process is an option.
- PR #1081: Transparently forward Backup requests to Leaders.
- PR #1079: Use a Protobuf model for Backup requests.
- PR #1078: Decrease bootstrap polling interval from 5 seconds to 2 seconds.
- PR #1082: Small refactor of backup code.
- PR #1075: Upgrade to latest SQL parser. Fixes issue #1072
- PR #1074: Support
NULLas a paramterized value. Fixes issue #1073
This release adds support for SQLite RANDOM(), the first such support for non-deterministic functions. It does this via statement-rewriting.
- PR #1046: Add rewriting of SQLite
RANDOM()so statements with this function are safe to use.
- PR #1064: Upgrade dependencies, and move to requiring Go 1.18 (or later) for building.
- PR #1058:
rqlitedterminates if passed unroutable advertised Raft or HTTP addresses. - PR #1057, PR #1059: Perform credential checking with intra-cluster communications. Fixes issue #1051. Thanks @ngharrington
- PR #1055: Add new
join-read-onlypermission.
- PR #1049: Ignore freshness when serving queries on Leader. Fixes issue #1048. Thanks to @Tjstretchalot for the bug report.
- PR #1043: Allow cluster-connect timeout to be configurable. Fixes issue #1042.
This release adds a new, higher-performance, option for writing to the database, known as Queued Writes. This allows users to trade off durability for high-performance writes to the database. See the documentation for full details.
- PR #1019: CLI supports restoring from SQLite database files.
- PR #1024, PR #1025, PR #1031, PR #1033: Add support for Queued Writes. Fixes issue #1020.
- PR #1036: rqbench supports queued writes.
- PR #1027: go mod (dependencies) updates, including upgrading SQLite to 3.38.5.
- PR #1030, PR #1032: Handle more connection errors. Fixes issue #1029.
With this release rqlite supports restoring a node from an actual SQLite file, which is very much faster than restoring using the SQL dump representation of the same SQLite database.
- PR #1017, PR #1018: Support restoring from SQLite data files. Fixes issue #1005.
- PR #1015: go mod (dependencies) updates.
- PR #999: Add end-to-end encrypted node test.
- PR #1008: Upgrade to SQLite 3.38.0. Fixes issue #1005.
- PR #1009: Don't HTML escape JSON output.
Fixes an issue in the 7.3.0 release that prevented clusters, which used TLS for internode communications, from operating correctly. All deployments using TLS should be upgraded to this version.
- PR #998: Node TLS mux needs to use advertised Raft address. Thanks to Nathan Ferch for the bug report.
With this release rqlite networking is based on whatever network identifiers are passed in at launch. If hostnames are passed they are no longer resolved into IP addresses internally by rqlite, and only the low-level networking layers will do so. This allows rqlite to operate correctly even when cluster node IP addresses change, as long as the hostnames remain the same. This is common in deployment environments such as Kubernetes, particularly when using StatefulSets.
Upgrading an earlier cluster to this release is possible, simply by explicitly setting the IP address of each node at the command line. Alternatively, simply backup your previous node and restore it into a new 7.3.0 node. In any event, backing up your data is always recommended before any upgrade takes place.
- PR #993: Support IP addresses changing by not explicitly resolving network addresses. Fixes issue #695, issue #774, and issue #991. Thanks to @git001 for help testing this change.
This release introduces supported for DNS-based and DNS SRV -based autoclustering. rqlite can now lookup a DNS record for specified host, and use the network addresses returned to bootstrap the cluster. This can make it much easier to create rqlite clusters using the kinds of Service-Discovery mechanisms you find on Consul and Kubernetes, which are often based on DNS. See the documentation for full details on using DNS-based autoclustering.
- PR #979: Add support for DNS-based autoclustering. Fixes issue #554
- PR #985: Add support for DNS SRV -based autoclustering. Fixes issue #554
- PR #976: Improve
/readyzresponse. - PR #978: Return error on join request if node ID is the same as receiving node.
- PR #980: Move config validation to Config type.
- PR #981: Add curent time to node
/statusoutput. - PR #982:
/readyzcan skip leader check vianoleaderquery param. - PR #984: Count number of
/statusand/readyzrequests via expvar. - PR #986: Refactor join code with new Joiner type.
- PR #987: Upgrade to SQLite 3.37.0.
This release introduces a new automatic clustering approach, known as Bootstrapping, which allows rqlite clusters to form without assistance from an external system such as Consul. This can be very useful for certain deployment scenarios. See the documentation for full details on using the new Bootstrapping mode. Special thanks to Nathan Ferch for his advice regarding the design and development of this feature.
- PR #974: Add support for automatically bootstrapping clusters from just rqlite nodes.
- PR #957: Correct rqlite command line options in log message.
- PR #973: Use correct JSON field name for Consul disco scheme
This release introduces new node-discovery integration with Consul and etcd. By using one of those systems with rqlite, automatic clustering of rqlite is much easier. The legacy Discovery mode is not supported by release 7.0, but may be supported in a future release. So, for now, if you wish to continue using legacy Discovery, you will need to run rqlite 6.x, or earlier.
See the new documentation for full details on using Consul and etcd.
This release uses a new database for the Raft system, which should be compatible with earlier releases. However it is strongly recommended you backup any existing Leader nodes before you run 7.0, in the event there are any issues.
There are also some breaking changes in release 7.0, related to command-line arguments:
- The disco-related command-line arguments have changed to support Consul and etcd. If you wish to continue to use legacy Discovery, you can't upgrade to 7.0 -- or consider using Consul or etcd for node-discovery.
- The command-line argument
-RaftWaitForLeaderhas been removed. If you need to wait for a node to have a Leader, you should poll the/readyzendpoint.
- PR #957: Support autoclustering via use Consul and etcd KV stores.
- PR #947: CLI takes list of hosts, so it can try another node if first node is unresponsive. Fixes issue #157. Thanks @chermehdi
- PR #957: Refactor
rqlitedcommand-line argument code. - PR #965: Stop using deprecated protobuf package.
- PR #967: Replace BoltDB with etcd's fork, bbolt.
- PR #968: Control whether bbolt syncs freelist to disk.
- PR #959: Return clearer error if no database results set.
- PR #945: Minor refactor of Store creation.
- PR #946: Use actual duration types for duration
rqlitedcommand line flags. - PR #944: Refactor how nodes bootstrap.
- PR #954: Much better random file names.
- PR #956: Actually return errors from command marshaling code.
This release enhances the authentication and authorization system, making it more convenient to use. It also fixes some minor bugs, and migrates the end-to-end test suite to Python 3.
- PR #940: Support specifying the user to join a cluster with, via
-join-as. - PR #940: Add support for pseudo username
*to authentication configuration, providing more control over access.
- PR #934: Use correct MIME media type for JSON text. Fixes issue #933.
- PR #939: Migrate end-to-end testing to Python 3.
- PR #941: Upgrade go mod dependencies.
- PR #932: Add support for named parameters. Fixes issue #675.
- PR #929: Support disabling in-memory initialization of on-disk databases. Fixes issue #928.
- Fix URL for
.readyCLI command. - PR #925: Ignore disco ID if there is preexisting state. Fixes issue #347.
- PR #926: Ignore own join address if passed. Fixes issue #713.
- PR #921: Support fetching
/readyzvia CLI.
- PR #920: Minor log message fixes.
- PR #922, PR #924: Fix display of numbers during JSON deserialization. Thanks @tiswong
- PR #918: Add support for timestamp column types.
- PR #916: More helpful log messages on start-up.
- PR #915: Record compiler in logs and status output.
This release is the first to statically link libc for the Linux version of rqlite. While this does increase the size of binary, one of the main goals of rqlite is ease of deployment. With fully static linking, official releases will run on an even wider range of distributions.
- PR #911: Statically link all requirements, including
libc, for rqlite on Linux. macOS retains dynamically linking.
- PR #909: Support intra-cluster responses up to 4MB in size. Fixes issue #908.
- PR #907: Make it clearer when writes are sent to the wrong endpoint. See issue #903.
- PR #898: Support recovering clusters that have permanently lost quorum. Fixes issue #897.
- PR #896: Add
/readyzendpoint for easy ready-to-respond checks.
- PR #885: Improved responses on HTTP 500.
- PR #888: Expose stats about BoltDB on the
status/endpoint. - PR #889: Add OS-level information to
status/output. - PR #892: More Snapshot metrics.
- PR #894: Support timeout on
nodes/endpoint. - PR #895: Start HTTP service before attempting any join operation.
- PR #882: Some minor improvements related to on-disk SQLite use.
- PR #883: Add missing returns after HTTP errors.
- PR #884: Serialize on-disk databases by simply reading SQLite file.
- PR #880: Increase maximum in-memory database size to 2GiB, via upgraded dependencies.
- PR #879: Set timeout when fetching node API address.
- PR #878: CLI supports setting read consistency level.
- PR #876: Add round-trip time to each node to
nodes/endpoint output. - PR #877: Add Error information to
nodes/endpoint output.
This release introduces transparent request forwarding, which simplifies interacting with rqlite clusters. Client requests that must be served by the leader will no longer return HTTP 301, and will be forwarded transparently to the leader if necessary. Client software does not need to change to take advantage of this new functionality.
Systems running earlier 6.x software can be upgraded to this release without doing any special, but all nodes in the new cluster must be running this release. This release cannot communicate with nodes running earlier 6.x software.
- PR #859: Support transparent Execute and Query request forwarding. Fixes issue #330.
- PR #873: Support explicitly specifying SQLite on-disk file path.
- PR #863: Add gauge-like metric for Snapshot timings.
- PR #854: Use a connection pool for internode communications.
- PR #867: Add cluster status to Status output.
- PR #869: Cluster client uses resolved address, and improved status output.
- PR #851, PR #855: rqlite CLI properly supports PRAGMA directives.
- PR #853: Support enabling Foreign Key constraints via command-line options.
- PR #857: Use Protobufs as core data model.
- PR #858: Create dedicated client for talking to a cluster service.
- PR #862: Add detailed SQLite memory statistics to status output.
This release makes significant changes to SQLite database connection handling, resulting in proper support for high-performance concurrent reads of in-memory databases (an in-memory SQLite database is the default option for rqlite).
- PR #848: Enable DBSTAT table and JSON1 support. Fixes issue #843.
- PR #841: Remove support for specifying SQLite DSN.
- PR #842: Use
vfs=memdballowing proper concurrent reads of in-memory databases. Special thanks to @rittneje. - PR #842: Use read-only database connections for read queries, ensuring write SQL commands are not executed on the wrong endpoint.
- PR #842: Remove explicit support for Foreign Key constraint control and journal mode. Those controls are best left to the rqlite system now.
- PR #845: Add SQLite compile-time options to status output.
- PR #846: New DB and FSM indexes to track state.
This release addresses a significant issue related to SQLite connection handling and multithreading. All users should upgrade to this version.
- PR #827: Upgrade dependencies, including SQLite to 3.36.
- PR #835: Use Go standard libary sql/database abstraction. Fixes issue #830.
- PR #835: Use SQLite connection pool and add pool statistics to status output.
- PR #836: Add current SQLite journal mode to status output.
- PR #839: Limit in-memory connection pool to 1 connection.
- PR #840: Upgrade to rqlite/go-sqlite3 v1.20.4. See this issue.
- PR #822: Don't ignore
-joineven if previous state exists. Fixes issue #818.
This release implements a significant design change, which improves rqlite cluster reliability. With this change a rqlite node can more reliably direct read and write requests to the correct node.
In the 5.0 series, Follower nodes learned the HTTP API address of the cluster Leader via information - known as Metadata - that each node wrote to the Raft log. This Metadata was then available to each node in the cluster, if that node needed to redirect queries to the cluster Leader (assuming that node wasn't the Leader at that time). However that design was somewhat complex, and required the tracking of extra state, in addition to the SQLite database. It also meant that if the Metadata got out of sync with the Raft state, the cluster could be in a degraded state.
In this new design, a node now queries the Leader as needed, when that node needs to learn the Leader's HTTP API address. As a result, the Metadata component has been removed from rqlite, since it is no longer needed. And without any possibility of discrepancy between Metadata and Raft state, a whole class of potential bugs is removed. Any request for the Leader HTTP API address means the requesting node node connects to a TCP port already open on the Leader for Raft connections, so does not introduce any new failure modes. This multiplexing of the Raft TCP port is performed via the mux package.
This new design does mean that nodes running earlier software cannot communicate with 6.0 nodes, as 6.0 software no longer performs Metadata updates. As a result, rqlite clusters running 5.x software or earlier must be explicitly upgraded. To upgrade from an earlier version to this release you should backup your Leader node, and restore the database dump into a new 6.0 cluster.
The data API and cluster-management API remain unchanged however, so client code that communicates with rqlite should not need any changes.
- PR #796:
nodes/API reports real-time status of other nodes in cluster. Fixes issue #768. - PR #802: Add
.sysdumpcommand to rqlite CLI. - PR #807: rqlite CLI displays build information. Fixes issue #768.
- PR #792: Fetch leader HTTP API addresses on demand.
- PR #797: Remove
redirectkey from HTTP status output.
- PR #791: Reinstate node CA cert support which was erroneously removed in an earlier change.
- PR #788: Upgrade to SQLite 3.35.4.
- PR #790: Upgrade dependencies, including Hashicorp Raft to v1.3.0.
- PR #783: Create GZIP writer for every compression request. Fixes issue #781.
- PR #776, PR #777: Support specifying Dialer's local address when performing Join request. Fixes issue #774. Thanks @osxlinux
- PR #782: Better error messages for command unmarshaling.
- PR #772: Log launch command.
- PR #769: Upgrade to rqlite/go-sqlite3 1.20.1, to address a significant memory leak.
- PR #742: Add one-shot option to rqbench.
- PR #745: TLS version 1.0 and 1.1 disabled by default, but can be re-enabled at the command line. Fixes issue #743.
- PR #738: Don't use temp file when snapshotting database.
- PR #739: Don't use temp file when restoring an in-memory database.
- PR #738: Switch to rqlite fork of mattn/go-sqlite3. The SQLite C code remains unchanged.
- PR #741: Tighten up Store-level locking.
- PR #747: Time snapshot, restore, and startup times.
- PR #750: Build on-disk databases in-memory first. Fixes issue #731.
- PR #754: Support Noop commands in Raft Log.
- PR #759, PR #760: Close BoltDB on Store close.
- PR #757: More extensive system-level testing of Snapshot and Restore.
- PR #762: Convert Raft stats to numbers where possible. Fixes issue #763.
- PR #764: Add total Raft directory size to Store stats.
- PR #764: Close SQLite database only after Raft has been shut down.
- PR #765: Add auth-ok and auth-fail events to stats.
- PR #734: Better control over waiting for Leader at startup, via command line options.
- PR #724: rqlite CLI displays node version at startup.
- PR #726: Count number of legacy commands unmarshaled.
- PR #733: Clearer Raft log status during startup.
- PR #735: Implement policy over trailing Raft logs.
- PR #716: Support HTTP/2 protocol over TLS. Fixes issue #516.
- PR #711, PR# 712: Ignore join addresses if node already part of cluster. Fixes issue #710.
- PR #715: Compress SQLite database in Raft snapshot.
- PR #717: Add SQLite database page-centric size to status output.
- PR #719: Exit if any arguments passed at command line after data directory. Fixes issue #718.
- PR #696: Benchmarking tool now supports query testing.
- PR #694: Display, in the CLI, the HTTP response body on HTTP status 503.
- PR #703: Fix potential panic during request parsing.
- PR #705: Use Protobuf for encoding Raft Log commands.
- PR #692: Support setting Raft leader lease timeout.
- PR #693: Upgrade Go mod dependencies, including upgrading Hashicorp Raft to v1.2.0.
- PR #680, PR #681: Add missing calls to set BasicAuth in CLI. Fixes issue #678.
- PR #682: Explicitly handle "no leader" during backup, and return redirect if necessary.
- PR #683: Restore request should re-read file every attempt.
- PR #673: Support parameterized statements. Fixes issue #140.
- PR #672: Fix issue causing HTTPS-only redirects.
This release should not be used, due to a HTTP redirection bug.
- PR #660: Raft log size on disk now reported via status endpoint.
- PR #670: Add utilities for testing encrypted nodes.
- PR #671: Set Location for HTTP redirect properly for secure nodes.
- PR #654: Allow number of Join attempts and Join interval to be specified at startup. Fixes issue #653.
- PR #641: rqlite CLI now supports node removal.
This release fixes a very significant bug, whereby snapshotting was never occuring due to a zero snapshot-interval being passed to the Raft subsystem. This meant that the Raft log would grow without bound, and could result in very long start-up times if the Raft log was very large.
- PR #637: Allow the Raft snapshotting check interval to be set at launch time.
- PR #637: Set the Raft snapshot check interval to 30 seconds by default.
- PR #636: Check consistency level before freshness check. Thanks @wangfenjin
- PR #613: Support read-only, non-voting nodes. These provide read scalability for the system.
- PR #614: Support specifying minimum leader freshness with None consistency queries.
- PR #620: Log level can be specified for Raft module.
This release uses a new Raft consensus version, with the move to Hashicorp Raft v1. As a result the Raft system in 5.0 is not compatible with the 4.0 series. To upgrade from an earlier version to this release you should backup your 4.0 leader node, and restore the database dump into a new 5.0 cluster.
The rqlite server also supports explicitly setting the node ID. While it's not required to set this, it's recommended for production clusters. See the cluster documentation for more details.
The HTTP Query and Insert API remains unchanged in the 5.0 series relative to the 4.0 series.
- PR #597: Don't ignore any Join error, instead return it.
- PR #598: Ensure backup is correctly closed.
- PR #600: Move to Hashicorp Raft v1.1.1.
- PR #601: By default use Raft network address as node ID.
- PR #602: Add method to Store that returns leader ID.
- PR #603: Fix up status key name style.
- PR #604: JSON types are also text.
- PR #605: Broadcast Store meta via consensus.
- PR #607: Various Redirect fixes.
- PR #609: Simplify rqlite implementation.
- PR #610: Write node backup directly to HTTP response writer. Thanks @sum12.
- PR #611: Add varadic perm check functions to auth store.
This release adds significant new functionality to the command-line tool, including much more control over backup and restore of the database. Visit the Releases page to download this release.
- PR #592: Add dump database command (to SQL text file) to CLI.
- PR #585: Add backup command (to SQLite file) to CLI: Thanks @eariassoto.
- PR #589: Add restore command (from SQLite file) to CLI. Thanks @eariassoto.
- PR #584: Support showing query timings in the CLI. Thanks @joaodrp.
- PR #586: rqlite CLI now supports command-recall via cursor key. Thanks @rhnvrm.
- PR #564: rqlite server supports specifying trusted root CA certificate. Thanks @zmedico.
- PR #593: rqlite CLI now supports HTTP proxy (via Environment). Thanks @paulstuart
- PR #587: Add expvar statistics to store.
- PR #591: Store layer supports generating SQL format backups.
- PR #590: DB layer supports generating SQL format backups.
- PR #588: Abort transaction if load from backup operation fails.
- If joining a cluster via HTTP fails, try HTTPS. Fixes issue #577.
- PR #551: rqlite CLI supports specifying trusted root CA certificate. Thanks @zmedico.
- Allow the Raft election timeout to be set.
- Allow a node to be re-added with same IP address and port, even though it was previously removed. Fixes issue #534.
- PR #397, PR #399: Support hashed passwords. Fixes issue #395. Thanks @sum12.
- PR #389: Log Store directory path on startup.
- PR #392: Return redirect if node removal attempted on follower. Fixes issue #391.
- PR #383: Fix unit tests after underlying SQLite master table changes.
- PR #384: "status" perm required to access Go runtime information.
- PR #367: Remove superflous leading space at CLI prompt.
- PR #368: CLI displays clear error message when not authorized.
- PR #370: CLI does not need to indent JSON when making requests.
- PR #373, PR #374: Add simple INSERT-only benchmarking tool.
- PR #342: Add missing Store parameters to diagnostic output.
- PR #343: Support fetching expvar information via CLI.
- PR #344: Make read-consistency query param value case-insensitive.
- PR #345: Add unit test coverage for status and expvar endpoints.
- PR #337: Include any query params with 301 redirect URL.
- PR #316: CLI doesn't need pretty nor timed responses from node.
- PR #334: Set Content-Type to "application/octet-stream" for backup endpoint. Fixes issue #333.
The 4.0 release has renamed command-line options relative to earlier releases. This means that previous commands used to launch rqlited will not work. However the command-line changes are cosmetic, and each previous option maps 1-to-1 to a renamed option. Otherwise deployments of earlier releases can be upgraded to the 4.0 series without any other work.
- PR #309: Tweak start-up logo.
- PR #308: Move to clearer command-line options.
- PR #307: Support node-to-node encryption. Fixes issue #93.
- PR #310: HTTP service supports registration of Status providers; Mux is first client.
- PR #315: Add status and help commands to CLI.
- PR #304: Switch to Go 1.8.1.
- PR #296: Log Go version at startup.
- PR #294: Support multiple explicit join addresses.
- PR #297: CLI should explicitly handle redirects due to Go1.8. Fixes issue #295.
- PR #291: Don't access Discovery Service if node already part of cluster.
- PR #286: Tweak help output.
- PR #283: Main code should log to stderr.
- PR #280, PR #281: Integrate with new Discovery Service.
- PR #282: Retry cluster-join attempts on failure.
- PR #289: rqlite HTTP clients should not automatically follow redirects.
- PR #268: Allow Store to wait for application of initial logs. Fixes issue #260.
- PR #272: Add commit, branch, GOOS, and GOARCH, to output of
--version. - PR #274: Use Hashicorp function to read peers. Thanks @WanliTian
- PR #278: Add support for dot-commands
.tablesand.schemato rqlite CLI. Fixes issue #277.
- PR #253: Handle nil row returned by SQL execer. Fixes issue #253.
- PR #258: Remove check that all queries begin with SELECT. Fixes issue #255.
- PR #247: Simplify loading of SQLite dump files via single command execution. Fixes issue #246.
- PR #247: Correct SQLite dump load authentication check.
- PR #239: Add an API to the
Storelayer for custom peers storage and logging. Thanks @tych0 - PR #221: Start full support for various SQLite text types. Fix issue #240.
- PR #242: Support direct copying of the database via the Store. Thanks @tych0.
- PR #243: Use Store logging everywhere in the Store package.
- PR #230: Move Chinook test data to idiomatic testdata directory.
- PR #232, PR #233: rqlite CLI now supports accessing secured rqlited nodes. Thanks @tych0.
- PR #235: Return correct error, if one occurs, during backup. Thanks @tych0.
- PR #237, PR #238: Support observing Raft changes. Thanks @tych0
- With this release rqlite is moving to Go 1.7.
- PR #206, #217: Support loading data directly from SQLite dump files.
- PR #209: Tweak help output.
- PR #229: Remove explicit control of foreign key constraints.
- PR #207: Database supports returning foreign key constraint status.
- PR #211: Diagnostics show actual foreign key constraint state.
- PR #212: Add database configuration to diagnostics output.
- PR #224, PR #225: Add low-level database layer expvar stats.
- PR #195: Set Content-type "application/json" on all HTTP responses.
- PR #193: Allow joining a cluster through any node, not just the leader.
- PR #187: Support memory profiling.
- Go cyclo complexity changes.
- With this release Windows compatibility is checked with every build.
- PR #185: Enable foreign key constraints by default.
- PR #175: Simplify error handling of Update Peers API.
- PR #170: Log any failure to call
Serve()on HTTP service. - Go lint fixes.
- Go cyclo complexity changes.
- PR #159: All HTTP responses set X-RQLITE-VERSION.
- PR #151: Support configurable Raft heartbeat timeout.
- PR #149: Support configurable Raft snapshot thresholds.
- PR #148: Support pprof information over HTTP.
- PR #154: CLI now redirects to leader if necessary.
- PR #155: CLI now handles "no rows" correctly.
- PR #143: Use DELETE as HTTP method to remove nodes.
- PR #142: Use correct HTTP methods on all endpoints.
- PR #137: Use resolved version of joining node's address.
- PR #136: Better errors on join failures.
- PR #133: Add Peers to status output.
- PR #132: Support removing a node from a cluster.
- PR #131: Only convert []byte from database to string if "text". Thanks @bkeroackdsc
- PR #129: Verify all statements sent to query endpoint begin with "SELECT".
- PR #141: Store methods to expose node Raft state. Thanks @bkeroack
- PR #117: Use Raft advertise address, if exists, during join.
The Raft log format in 3.0 is not compatible with the 2.0 series. To upgrade from an earlier version to this release you should backup your 2.0 leader node, and replay the database dump into a new 3.0 cluster. The HTTP API remains unchanged however.
- PR #116: Allow HTTP advertise address to be set.
- PR #115: Support advertising address different than Raft bind address.
- PR #113: Switch to in-memory SQLite databases by default.
- PR #109: Nodes broadcast meta to cluster via Raft.
- PR #109, PR #111: Leader redirection
- PR #104: Handle the
-joinoption sensibly when already member of cluster.
- PR #95: Correctly set HTTP authentication.
- PR #84: Encrypted API (HTTPS) now supported.
- PR #85: BasicAuth support.
- PR #85: User-level permissions support.
- Print rqlited logo on start-up.
- End-to-end single-node and multi-node unit tests.
- PR #76: Obey timing information display at database level.
- PR #77: Add version information to binary.
timingsURL param to control presence of timing information in response.- PR #74: Use SQLite connection directly. Thanks @zmedico.
- Update operations return last-inserted ID.
- Column-oriented API responses.
- Types in API response body.
- Query times in addition to sum of query times.
- New Raft consensus module, built on Hashicorp's implementation.
- Hot backup support.
- Selectable read-consistency levels of none, weak, and strong.
- SQLite file size added to HTTP API status endpoint.
- expvar support added to HTTP server.
Check out this tag for full details.