“The exception confirms the rule in cases not excepted." ― Cicero.
Due to significant commits we had this last week, we decided to make an exception in our release schedule and cut one more release in 2023!
Still, the TigerBeetle team wishes everyone happy holidays! 🎁
-
Some CI-related stuff plus the
-Dreleaseflag, which will bring back the joy of using the compiler from the command line 🤓. -
Added value count to
TableInfo, allowing future optimizations for paced compaction.
-
The simulator found a failure when the WAL gets corrupted near a checkpoint boundary, leading us to also consider scenarios where corrupted blocks in the grid end up "intersecting" with corruption in the WAL, making the state unrecoverable where it should be. We fixed it by extending the durability of "prepares", evicting them from the WAL only when there's a quorum of checkpoints covering this "prepare".
-
Fix a unit test that regressed after we changed an undesirable behavior that allowed
prefetchto invoke its callback synchronously. -
Relaxed a simulator's verification, allowing replicas of the core cluster to be missing some prepares, as long as they are from a past checkpoint.
-
A highly anticipated feature lands on TigerBeetle: it's now possible to retrieve the transfers involved with a given account by using the new operation
get_account_transfers.Note that this feature itself is an ad-hoc API intended to be replaced once we have a proper Querying API. The real improvement of this PR is the implementation of range queries, enabling us to land exciting new features on the next releases.
-
Bump the client's maximum limit and the default value of
concurrency_maxto fully take advantage of the batching logic.
As the last release of the year 2023, the TigerBeetle team wishes everyone happy holidays! 🎁
-
We've established a rotation between the team for handling releases. As the one writing these release notes, I am now quite aware.
-
Fix panic in JVM unit test on Java 21. We test JNI functions even if they're not used by the Java client and the semantics have changed a bit since Java 11.
-
Move client sessions from the Superblock (database metadata) into the Grid (general storage). This simplifies control flow for various sub-components like Superblock checkpointing and Replica state sync.
-
An optimization for removes on secondary indexes makes a return. Now tombstone values in the LSM can avoid being compacted all the way down to the lowest level if they can be cancelled out by inserts.
-
Clients automatically batch pending similar requests 🎉! If a tigerbeetle client submits a request, and one with the same operation is currently in-flight, they will be grouped and processed together where possible (currently, only for
CreateAccountandCreateTransfers). This should greatly improve the performance of workloads which submit a single operation at a time.
-
Defense in depth: add checkpoint ID to prepare messages. Checkpoint ID is a hash that covers, via hash chaining, the entire state stored in the data file. Verifying that checkpoint IDs match provides a direct strong cryptographic guarantee that the state is the same across replicas, on top of existing guarantee that the sequence of events leading to the state is identical.
-
Gate the main branch on more checks: unit-tests for NodeJS and even more fuzzers.
-
Code cleanups after removal of storage size limit.
-
Fix free set index. The free set is a bitset of free blocks in the grid. To speed up block allocation, the free set also maintains an index --- a coarser-grained bitset where a single bit corresponds to 1024 blocks. Maintaining consistency between a data structure and its index is hard, and thorough assertions are crucial. When moving free set to the grid, we discovered that, in fact, we don't have enough assertions in this area and, as a result, even have a bug! Assertions added, bug removed!
-
LSM tree fuzzer found a couple of bugs in its own code.
-
Remove format-time limit on the size of the data file. Before, the maximum size of the data file affected the layout of the superblock, and there wasn't any good way to increase this limit, short of recreating the cluster from scratch. Now, this limit only applies to the in-memory data structures: when a data files grows large, it is sufficient to just restart its replica with a larger amount of RAM.
-
We finally have the "installation" page in our docs!
-
Use Zig's new
if (@inComptime())builtin to compute checksum of an empty byte slice at compile time. -
Fix unit tests for the Go client and add them to not rocket science set of checks.
-
When validating our releases, use the
releasebranch instead ofmainto ensure everything is in sync, and give the Java validation some retry logic to allow for delays in publishing to Central. -
Pad storage checksums from 128-bit to 256-bit. These are currently unused, but we're reserving the space for AEAD tags in future.
-
Remove a trailing comma in our Java client sample code.
-
Switch
bootstrap.shto use spaces only for indentation and ensure it's checked by our shellcheck lint. -
Update our
DESIGN.mdto better reflect storage fault probabilities and add in a reference. -
Add
CHANGELOG.mdvalidation to our tidy lint script. We now check line length limits and trailing whitespace. -
In keeping with TigerStyle, rename
reserved_noncetononce_reserved. -
Note in TigerStyle that callbacks go last in the list of parameters.
-
Add an exception for line length limits if there's a link in said line.
-
Recursively check for padding in structs used for data serialization, ensuring that no uninitialized bytes can be stored or transmitted over the network. Previously, we checked only if the struct had no padding, but not its fields.
-
Minor adjustments in the release process, making it easier to track updates in the documentation website when a new version is released, even if there are no changes in the documentation itself.
-
Fix outdated documentation regarding 128-bit balances.
-
Fix a bug discovered and reported during the Hackathon 2023, where the Node.js client's error messages were truncated due to an incorrect string concatenation adding a null byte
0x00in the middle of the string. -
Update the Node.js samples instructions, guiding the user to install all dependencies before the sample project.
-
We've doubled the
Headers size to 256 bytes, paving the way for future improvements that will require extra space. Concurrently, this change also refactors a great deal of code. Some of theHeader's fields are shared by all messages, however, eachCommandalso requires specific pieces of information that are only used by its kind of message, and it was necessary to repurpose and reinterpret fields so that the same header could hold different data depending on the context. Now, commands have their own specialized data type containing the fields that are only pertinent to the context, making the API much safer and intent-clear. -
With larger headers (see #1295) we have enough room to make the cluster ID a 128-bit integer, allowing operators to generate random cluster IDs without the cost of having a centralized ID coordinator. Also updates the documentation and sample programs to reflect the new maximum batch size, which was reduced from 8191 to 8190 items after we doubled the header.
-
Implement last-mile release artifact verification in CI.
-
Bump the simulator's safety phase max-ticks to avoid false positives from the liveness check.
-
Fix a crash caused by a race between a commit and a repair acquiring a client-reply
Write. -
Fix a crash caused by a race between state (table) sync and a move-table compaction.
Both bugs didn't stand a chance in the Line of Fire of our deterministic simulator!
-
Specify which CPU features are supported in builds.
-
Improve
shell.zig's directory handling, to guard against mistakes with respect to the current working directory. -
Interpret a git hash as a VOPR seed, to enable reproducible simulator smoke tests in CI.
-
Explicitly target glibc 2.7 when building client libraries, to make sure TigerBeetle clients are compatible with older distributions.
-
Revive the TigerBeetle VOPRHub! Some previous changes left it on it's Last Stand, but the bot is back in business finding liveness bugs: #1266
-
Set the latest Docker image to track the latest release. Avoids language clients going out of sync with your default docker replica installations.
-
Move website doc generation for https://docs.tigerbeetle.com/ into the main repo.
-
Addressed some release quirks with the .NET and Go client builds.
-
Prove a tighter upper bound for the size of manifest log. With this new bound, manifest log is guaranteed to fit in allocated memory and is smaller. Additionally, manifest log compaction is paced depending on the current length of the log, balancing throughput and time-to-recovery.
-
Recommend using ULID for event IDs. ULIDs are approximately sorted, which significantly improves common-case performance.
-
Rewrite Node.js client implementation to use the common C client underneath. While clients for other languages already use the underlying C library, the Node.js client duplicated some code for historical reasons, but now we can leave that duplication in the past. This Is A Photograph.
-
Increase block size to reduce latencies due to compaction work. Today, we use a simplistic schedule for compaction, which causes latency spikes at the end of the bar. While the future solution will implement a smarter compaction pacing to distribute the work more evenly, we can get a quick win by tweaking the block and the bar size, which naturally evens out latency spikes.
-
The new release process changed the names of the published artifacts (the version is no longer included in the name). This broke our quick start scripts, which we have fixed. Note that we are in the process of rolling out the new release process, so some unexpected breakage is expected.
-
Speed up secondary index maintenance by statically distinguishing between insertions and updates. Faster than the speed of night!
-
Include Docker images in the release.
-
Simplify superblock layout by using a linked list of blocks for manifest log, so that the superblock needs to store only two block references.
P.S. Note the PR number!
This is the start of the changelog. A lot happened before this point and is lost in the mist of git history, but any notable change from this point on shall be captured by this document.
-
Remove bloom filters. TigerBeetle implements more targeted optimizations for both positive and negative lookups, making bloom filters a net loss.
-
Increase alignment of data blocks to 128KiB (from 512 bytes). Larger alignment gives operators better control over physical layout of data on disk.
-
Overhaul of CI and release infrastructure. CI and releases are now driven by Zig code. The main branch is gated on integration tests for all clients.
This is done in preparation for the first TigerBeetle release.
For archeological inquiries, check out the state of the repository at the time of the first changelog: