0.14.176: Merge pull request #1445 from tigerbeetle/matklad/release-2024-01-22

0.14.176

Automated build. Do not use in production.

NOTE: You must run the same version of server and client. We do
not yet follow semantic versioning where all patch releases are
interchangeable.

Server

• Binary: Download the zip for your OS and architecture from this page and unzip.
• Docker: docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.176
• Docker (debug image): docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.176-debug

Clients

NOTE: Because of package manager caching, it may take a few
minutes after the release for this version to appear in the package
manager.

• .NET: dotnet add package tigerbeetle --version 0.14.176
• Go: go mod edit -require github.com/tigerbeetle/tigerbeetle-go@v0.14.176
• Java: Update the version of com.tigerbeetle.tigerbeetle-java in pom.xml to 0.14.176.
• Node.js: npm install tigerbeetle-node@0.14.176

Changelog

Welcome to 2024!

Safety And Performance

• #1425,
  #1412,
  #1410,
  #1408,
  #1395.

  Run more fuzzers directly in CI as a part of not rocket science package.

• #1413

  Formalize some ad-hoc testing practices as proper integration tests (that is, tests that interact
  with a tigerbeetle binary through IPC).

• #1404

  Add a lint check for unused Zig files.

• #1390

  Improve cluster availability by including conservative information about the current view into
  ping-pong messages. In particular, prevent the cluster from getting stuck when all replicas become
  primaries for different views.

• #1365

  Test both the latest and the oldest supported Java version on CI.

• #1389

  Fix a data race on close in the Java client.

Features

• #1403

  Make binaries on Linux about six times smaller (12MiB -> 2MiB). Turns tigerbeetle was
  accidentally including 10 megabytes worth of debug info! Note that unfortunately stripping all
  debug info also prevents getting a nice stack trace in case of a crash. We are working on finding
  the minimum amount of debug information required to get just the stack traces.

• #1423,
  #1426.

  Cleanup error handling API for Java client to never surface internal errors as checked exceptions.

• #1405

  Add example for setting up TigerBeetle as a systemd service.

• #1400

  Drop support for .Net Standard 2.1.

• #1397

  Don't exit repl on help command.

Internals

• #1422,
  #1420,
  #1417

  Overhaul documentation-testing infrastructure to reduce code duplication.

• #1398

  Don't test NodeJS client on platforms for which there are no simple upstream installation scripts.

• #1388

  Use histogram in the benchmark script to reduce memory usage.

TigerTracks 🎧

• Stripped

0.14.175: Merge pull request #1428 from tigerbeetle/matklad/release-2024-01-15

0.14.175

Automated build. Do not use in production.

NOTE: You must run the same version of server and client. We do
not yet follow semantic versioning where all patch releases are
interchangeable.

Server

• Binary: Download the zip for your OS and architecture from this page and unzip.
• Docker: docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.175
• Docker (debug image): docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.175-debug

Clients

NOTE: Because of package manager caching, it may take a few
minutes after the release for this version to appear in the package
manager.

• .NET: dotnet add package tigerbeetle --version 0.14.175
• Go: go mod edit -require github.com/tigerbeetle/tigerbeetle-go@v0.14.175
• Java: Update the version of com.tigerbeetle.tigerbeetle-java in pom.xml to 0.14.175.
• Node.js: npm install tigerbeetle-node@0.14.175

Changelog

Welcome to 2024!

Safety And Performance

• #1425,
  #1412,
  #1410,
  #1408,
  #1395.

  Run more fuzzers directly in CI as a part of not rocket science package.

• #1413

  Formalize some ad-hoc testing practices as proper integration tests (that is, tests that interact
  with a tigerbeetle binary through IPC).

• #1404

  Add a lint check for unused Zig files.

• #1390

  Improve cluster availability by including conservative information about the current view into
  ping-pong messages. In particular, prevent the cluster from getting stuck when all replicas become
  primaries for different views.

• #1365

  Test both the latest and the oldest supported Java version on CI.

• #1389

  Fix a data race on close in the Java client.

Features

• #1403

  Make binaries on Linux about six times smaller (12MiB -> 2MiB). Turns tigerbeetle was
  accidentally including 10 megabytes worth of debug info! Note that unfortunately stripping all
  debug info also prevents getting a nice stack trace in case of a crash. We are working on finding
  the minimum amount of debug information required to get just the stack traces.

• #1423,
  #1426.

  Cleanup error handling API for Java client to never surface internal errors as checked exceptions.

• #1405

  Add example for setting up TigerBeetle as a systemd service.

• #1400

  Drop support for .Net Standard 2.1.

• #1397

  Don't exit repl on help command.

Internals

• #1422,
  #1420,
  #1417

  Overhaul documentation-testing infrastructure to reduce code duplication.

• #1398

  Don't test NodeJS client on platforms for which there are no simple upstream installation scripts.

• #1388

  Use histogram in the benchmark script to reduce memory usage.

TigerTracks 🎧

• Stripped

0.14.174: Merge pull request #1385 from tigerbeetle/dj-release-release-fix

0.14.174

Automated build. Do not use in production.

NOTE: You must run the same version of server and client. We do
not yet follow semantic versioning where all patch releases are
interchangeable.

Server

• Binary: Download the zip for your OS and architecture from this page and unzip.
• Docker: docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.174
• Docker (debug image): docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.174-debug

Clients

NOTE: Because of package manager caching, it may take a few
minutes after the release for this version to appear in the package
manager.

• .NET: dotnet add package tigerbeetle --version 0.14.174
• Go: go mod edit -require github.com/tigerbeetle/tigerbeetle-go@v0.14.174
• Java: Update the version of com.tigerbeetle.tigerbeetle-java in pom.xml to 0.14.174.
• Node.js: npm install tigerbeetle-node@0.14.174

Changelog

“The exception confirms the rule in cases not excepted." ― Cicero.

Due to significant commits we had this last week, we decided to make an exception
in our release schedule and cut one more release in 2023!

Still, the TigerBeetle team wishes everyone happy holidays! 🎁

Internals

• #1362,
  #1367,
  #1374,
  #1375

  Some CI-related stuff plus the -Drelease flag, which will bring back the joy of
  using the compiler from the command line 🤓.

• #1373

  Added value count to TableInfo, allowing future optimizations for paced compaction.

Safety And Performance

• #1346

  The simulator found a failure when the WAL gets corrupted near a checkpoint boundary, leading us
  to also consider scenarios where corrupted blocks in the grid end up "intersecting" with
  corruption in the WAL, making the state unrecoverable where it should be. We fixed it by
  extending the durability of "prepares", evicting them from the WAL only when there's a quorum of
  checkpoints covering this "prepare".

• #1366

  Fix a unit test that regressed after we changed an undesirable behavior that allowed prefetch
  to invoke its callback synchronously.

• #1381

  Relaxed a simulator's verification, allowing replicas of the core cluster to be missing some
  prepares, as long as they are from a past checkpoint.

Features

• #1054

  A highly anticipated feature lands on TigerBeetle: it's now possible to retrieve the transfers
  involved with a given account by using the new operation get_account_transfers.

  Note that this feature itself is an ad-hoc API intended to be replaced once we have a proper
  Querying API. The real improvement of this PR is the implementation of range queries, enabling
  us to land exciting new features on the next releases.

• #1368

  Bump the client's maximum limit and the default value of concurrency_max to fully take
  advantage of the batching logic.

TigerTracks 🎧

• Everybody needs somebody

0.14.171: Merge pull request #1364 from tigerbeetle/matklad/artisinal-releases

0.14.171

Automated build. Do not use in production.

NOTE: You must run the same version of server and client. We do
not yet follow semantic versioning where all patch releases are
interchangeable.

Server

• Binary: Download the zip for your OS and architecture from this page and unzip.
• Docker: docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.171
• Docker (debug image): docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.171-debug

Clients

NOTE: Because of package manager caching, it may take a few
minutes after the release for this version to appear in the package
manager.

• .NET: dotnet add package tigerbeetle --version 0.14.171
• Go: go mod edit -require github.com/tigerbeetle/tigerbeetle-go@v0.14.171
• Java: Update the version of com.tigerbeetle.tigerbeetle-java in pom.xml to 0.14.171.
• Node.js: npm install tigerbeetle-node@0.14.171

Changelog

As the last release of the year 2023, the TigerBeetle team wishes everyone happy holidays! 🎁

Internals

• #1359

  We've established a rotation between the team for handling releases. As the one writing these
  release notes, I am now quite aware.

• #1357

  Fix panic in JVM unit test on Java 21. We test JNI functions even if they're not used by the Java
  client and the semantics have changed a bit since Java 11.

• #1351,
  #1356,
  #1360

  Move client sessions from the Superblock (database metadata) into the Grid (general storage). This
  simplifies control flow for various sub-components like Superblock checkpointing and Replica state
  sync.

Safety And Performance

• #1352

  An optimization for removes on secondary indexes makes a return. Now tombstone values in the LSM
  can avoid being compacted all the way down to the lowest level if they can be cancelled out by
  inserts.

• #1257

  Clients automatically batch pending similar requests 🎉! If a tigerbeetle client submits a
  request, and one with the same operation is currently in-flight, they will be grouped and
  processed together where possible (currently, only for CreateAccount and CreateTransfers).
  This should greatly improve the performance
  of workloads which submit a single operation at a time.

TigerTracks 🎧

• Carouselambra

0.14.169: Merge pull request #1350 from tigerbeetle/dj-tidy-tidy

0.14.169

Automated build. Do not use in production.

NOTE: You must run the same version of server and client. We do
not yet follow semantic versioning where all patch releases are
interchangeable.

Server

• Binary: Download the zip for your OS and architecture from this page and unzip.
• Docker: docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.169
• Docker (debug image): docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.169-debug

Clients

NOTE: Because of package manager caching, it may take a few
minutes after the release for this version to appear in the package
manager.

• .NET: dotnet add package tigerbeetle --version 0.14.169
• Go: go mod edit -require github.com/tigerbeetle/tigerbeetle-go@v0.14.169
• Java: Update the version of com.tigerbeetle.tigerbeetle-java in pom.xml to 0.14.169.
• Node.js: npm install tigerbeetle-node@0.14.169

Changelog

Safety And Performance

• #1339

  Defense in depth: add checkpoint ID to prepare messages. Checkpoint ID is a hash that covers, via
  hash chaining, the entire state stored in the data file. Verifying that checkpoint IDs match
  provides a direct strong cryptographic guarantee that the state is the same across replicas, on
  top of existing guarantee that the sequence of events leading to the state is identical.

Internals

• #1343,
  #1341,
  #1340

  Gate the main branch on more checks: unit-tests for NodeJS and even more fuzzers.

• #1332,
  #1348

  Code cleanups after removal of storage size limit.

TigerTracks 🎧

• Concrete Reservation

0.14.168: Merge pull request #1337 from tigerbeetle/matklad/release-2023-12-04

0.14.168

Automated build. Do not use in production.

NOTE: You must run the same version of server and client. We do
not yet follow semantic versioning where all patch releases are
interchangeable.

Server

• Binary: Download the zip for your OS and architecture from this page and unzip.
• Docker: docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.168
• Docker (debug image): docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.168-debug

Clients

NOTE: Because of package manager caching, it may take a few
minutes after the release for this version to appear in the package
manager.

• .NET: dotnet add package tigerbeetle --version 0.14.168
• Go: go mod edit -require github.com/tigerbeetle/tigerbeetle-go@v0.14.168
• Java: Update the version of com.tigerbeetle.tigerbeetle-java in pom.xml to 0.14.168.
• Node.js: npm install tigerbeetle-node@0.14.168

Changelog

Safety And Performance

• #1330,
  #1319

  Fix free set index. The free set is a bitset of free blocks in the grid. To speed up block
  allocation, the free set also maintains an index --- a coarser-grained bitset where a single bit
  corresponds to 1024 blocks. Maintaining consistency between a data structure and its index is
  hard, and thorough assertions are crucial. When moving free set to the grid, we discovered that,
  in fact, we don't have enough assertions in this area and, as a result, even have a bug!
  Assertions added, bug removed!

• #1323,
  #1336,
  #1324

  LSM tree fuzzer found a couple of bugs in its own code.

Features

• #1331,
  #1322,
  #1328

  Remove format-time limit on the size of the data file. Before, the maximum size of the data file
  affected the layout of the superblock, and there wasn't any good way to increase this limit, short
  of recreating the cluster from scratch. Now, this limit only applies to the in-memory data
  structures: when a data files grows large, it is sufficient to just restart its replica with a
  larger amount of RAM.

• #1321.

  We finally have the "installation" page in our docs!

Internals

• #1334

  Use Zig's new if (@inComptime()) builtin to compute checksum of an empty byte slice at compile
  time.

• #1315

  Fix unit tests for the Go client and add them to
  not rocket science
  set of checks.

TigerTracks 🎧

• Times Like These

0.14.167: Merge pull request #1325 from tigerbeetle/cb22/release-2023-11-27

0.14.167

Automated build. Do not use in production.

NOTE: You must run the same version of server and client. We do
not yet follow semantic versioning where all patch releases are
interchangeable.

Server

• Binary: Download the zip for your OS and architecture from this page and unzip.
• Docker: docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.167
• Docker (debug image): docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.167-debug

Clients

NOTE: Because of package manager caching, it may take a few
minutes after the release for this version to appear in the package
manager.

• .NET: dotnet add package tigerbeetle --version 0.14.167
• Go: go mod edit -require github.com/tigerbeetle/tigerbeetle-go@v0.14.167
• Java: Update the version of com.tigerbeetle.tigerbeetle-java in pom.xml to 0.14.167.
• Node.js: npm install tigerbeetle-node@0.14.167

Changelog

Internals

• #1306,
  #1308

  When validating our releases, use the release branch instead of main to ensure everything is
  in sync, and give the Java validation some retry logic to allow for delays in publishing to
  Central.

• #1310

  Pad storage checksums from 128-bit to 256-bit. These are currently unused, but we're reserving
  the space for AEAD tags in future.

• #1312

  Remove a trailing comma in our Java client sample code.

• #1313

  Switch bootstrap.sh to use spaces only for indentation and ensure it's checked by our
  shellcheck lint.

• #1314

  Update our DESIGN.md to better reflect storage fault probabilities and add in a reference.

• #1316

  Add CHANGELOG.md validation to our tidy lint script. We now check line length limits and
  trailing whitespace.

• #1317

  In keeping with
  TigerStyle,
  rename reserved_nonce to nonce_reserved.

• #1318

  Note in TigerStyle that callbacks go last in the list of parameters.

• #1325

  Add an exception for line length limits if there's a link in said line.

TigerTracks 🎧

• Space Trash

0.14.166: Merge pull request #1305 from tigerbeetle/batiati-release-2023-11-20

0.14.166

Automated build. Do not use in production.

NOTE: You must run the same version of server and client. We do
not yet follow semantic versioning where all patch releases are
interchangeable.

Server

• Binary: Download the zip for your OS and architecture from this page and unzip.
• Docker: docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.166
• Docker (debug image): docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.166-debug

Clients

NOTE: Because of package manager caching, it may take a few
minutes after the release for this version to appear in the package
manager.

• .NET: dotnet add package tigerbeetle --version 0.14.166
• Go: go mod edit -require github.com/tigerbeetle/tigerbeetle-go@v0.14.166
• Java: Update the version of com.tigerbeetle.tigerbeetle-java in pom.xml to 0.14.166.
• Node.js: npm install tigerbeetle-node@0.14.166

Changelog

Safety And Performance

• #1300

  Recursively check for padding in structs used for data serialization, ensuring that no
  uninitialized bytes can be stored or transmitted over the network. Previously, we checked only
  if the struct had no padding, but not its fields.

Internals

• #1299

  Minor adjustments in the release process, making it easier to track updates in the documentation
  website when a new version is released, even if there are no changes in the documentation itself.

• #1301

  Fix outdated documentation regarding 128-bit balances.

• #1302

  Fix a bug discovered and reported
  during the Hackathon 2023, where the Node.js
  client's error messages were truncated due to an incorrect string concatenation adding a null
  byte 0x00 in the middle of the string.

• #1291

  Update the Node.js samples instructions, guiding the user to install all dependencies before
  the sample project.

• #1295

  We've doubled the Headers size to 256 bytes, paving the way for future improvements that will
  require extra space. Concurrently, this change also refactors a great deal of code.
  Some of the Header's fields are shared by all messages, however, each Command also requires
  specific pieces of information that are only used by its kind of message, and it was necessary to
  repurpose and reinterpret fields so that the same header could hold different data depending on
  the context. Now, commands have their own specialized data type containing the fields that are
  only pertinent to the context, making the API much safer and intent-clear.

• #1304

  With larger headers (see #1295) we have enough room to make the cluster ID a 128-bit integer,
  allowing operators to generate random cluster IDs without the cost of having a centralized ID
  coordinator. Also updates the documentation and sample programs to reflect the new maximum batch
  size, which was reduced from 8191 to 8190 items after we doubled the header.

TigerTracks 🎧

• She smiled sweetly

0.14.165: Merge pull request #1305 from tigerbeetle/batiati-release-2023-11-20

0.14.165

Automated build. Do not use in production.

NOTE: You must run the same version of server and client. We do
not yet follow semantic versioning where all patch releases are
interchangeable.

Server

• Binary: Download the zip for your OS and architecture from this page and unzip.
• Docker: docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.165
• Docker (debug image): docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.165-debug

Clients

NOTE: Because of package manager caching, it may take a few
minutes after the release for this version to appear in the package
manager.

• .NET: dotnet add package tigerbeetle --version 0.14.165
• Go: go mod edit -require github.com/tigerbeetle/tigerbeetle-go@v0.14.165
• Java: Update the version of com.tigerbeetle.tigerbeetle-java in pom.xml to 0.14.165.
• Node.js: npm install tigerbeetle-node@0.14.165

Changelog

Safety And Performance

• #1264

  Implement last-mile release artifact verification in CI.

• #1268

  Bump the simulator's safety phase max-ticks to avoid false positives from the liveness check.

• #1270

  Fix a crash caused by a race between a commit and a repair acquiring a client-reply Write.

• #1278

  Fix a crash caused by a race between state (table) sync and a move-table compaction.

  Both bugs didn't stand a chance in the Line of Fire
  of our deterministic simulator!

Internals

• #1244

  Specify which CPU features are supported in builds.

• #1275

  Improve shell.zig's directory handling, to guard against mistakes with respect to the current
  working directory.

• #1277

  Interpret a git hash as a VOPR seed, to enable reproducible simulator smoke tests in CI.

• #1288

  Explicitly target glibc 2.7 when building client libraries, to make sure TigerBeetle clients are
  compatible with older distributions.

0.14.164: Merge pull request #1299 from tigerbeetle/matklad/empty-docs

0.14.164

Automated build. Do not use in production.

NOTE: You must run the same version of server and client. We do
not yet follow semantic versioning where all patch releases are
interchangeable.

Server

• Binary: Download the zip for your OS and architecture from this page and unzip.
• Docker: docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.164
• Docker (debug image): docker pull ghcr.io/tigerbeetle/tigerbeetle:0.14.164-debug

Clients

NOTE: Because of package manager caching, it may take a few
minutes after the release for this version to appear in the package
manager.

• .NET: dotnet add package tigerbeetle --version 0.14.164
• Go: go mod edit -require github.com/tigerbeetle/tigerbeetle-go@v0.14.164
• Java: Update the version of com.tigerbeetle.tigerbeetle-java in pom.xml to 0.14.164.
• Node.js: npm install tigerbeetle-node@0.14.164

Changelog

Safety And Performance

• #1264

  Implement last-mile release artifact verification in CI.

• #1268

  Bump the simulator's safety phase max-ticks to avoid false positives from the liveness check.

• #1270

  Fix a crash caused by a race between a commit and a repair acquiring a client-reply Write.

• #1278

  Fix a crash caused by a race between state (table) sync and a move-table compaction.

  Both bugs didn't stand a chance in the Line of Fire
  of our deterministic simulator!

Internals

• #1244

  Specify which CPU features are supported in builds.

• #1275

  Improve shell.zig's directory handling, to guard against mistakes with respect to the current
  working directory.

• #1277

  Interpret a git hash as a VOPR seed, to enable reproducible simulator smoke tests in CI.

• #1288

  Explicitly target glibc 2.7 when building client libraries, to make sure TigerBeetle clients are
  compatible with older distributions.