Skip to content

Release v2.16.1

Latest

Choose a tag to compare

@rekhoff rekhoff released this 20 Nov 04:51
· 8 commits to master since this release
8932a5a

Features

🚀 Tiered storage: your database is no longer limited by RAM 🚀

Until now, every SpacetimeDB database has had a hard limit on database size: all of its data had to fit in memory.

As of today, that limit is gone. With tiered storage, a table's data can be stored in remote object storage and cached across memory, local disk, and remote storage in a tiered structure. SpacetimeDB keeps your hot data in memory and moves colder data to cheaper tiers, so your database can grow far beyond the RAM of any single machine.

Turning it on

Tiered storage is opt-in per table in this release, so existing modules keep exactly the performance they have today. To opt a table in, give it a cache policy:

Policy What it means
required_in_memory Default. The whole table stays in memory, as before.
required_on_disk Data can leave memory but always stays on local disk.
no_requirement Full tiered storage. Let the host decide where to cache data.

Rust

#[spacetimedb::table(accessor = chat_message, public, cache_policy = no_requirement)]
pub struct ChatMessage {
    #[primary_key]
    #[auto_inc]
    id: u64,
    sender: Identity,
    text: String,
}

TypeScript

const chatMessage = table(
  { name: 'chat_message', public: true, cachePolicy: 'no_requirement' },
  {
    id: t.u64().primaryKey().autoInc(),
    sender: t.identity(),
    text: t.string(),
  }
);

C#

[SpacetimeDB.Table(Accessor = "ChatMessage", Public = true,
    CachePolicy = SpacetimeDB.TableCachePolicy.NoRequirement)]
public partial struct ChatMessage
{
    [SpacetimeDB.PrimaryKey, SpacetimeDB.AutoInc]
    public ulong Id;
    public Identity Sender;
    public string Text;
}

You can change the policy on an existing table too: update the attribute and republish, and the migration is handled for you.

This feature is great for use cases such as message history, event logs, match replays, inventories, and any large table where only a small slice is hot at once.

⚠️⚠️⚠️ Heads up: in the next major version (3.0), no_requirement will become the default. To keep the current all-in-memory behavior for a table after that, set required_in_memory explicitly.

⚠️⚠️⚠️ Heads up: As of this update, all Maincloud databases now have a default limit of 24 GiB of resident memory. If you see the error Database resident memory budget exceeded (limit: 24.0 GiB). while inserting into your database, you can either delete data or opt-in to tiered storage.

What happens to cold data during a transaction?

When a reducer touches data that has been moved out of memory, SpacetimeDB brings it back into memory automatically. Your code will continue to work unchanged. You call ctx.db.chat_message.id.find(42) the same way whether that row is in memory, on disk, or in remote storage.

The tricky part is the commit. Fetching data from disk or remote storage can fail, for example because of a network error. If that happens SpacetimeDB must abort the transactions.

In order to ensure atomicity, SpacetimeDB commits in two steps:

  1. Gather. It loads everything the transaction will modify into memory and locks it there, so nothing can be moved back out while the commit is in progress. If loading fails here, nothing has been written yet and the transaction rollsback.
  2. Apply. With everything already in memory, it writes the changes. This step is infallible, so every transaction is applied completely or not at all.

Tiered storage changes where your data lives, but not the transactional guarantees SpacetimeDB has always given you.

(#6305, #6443, #6470, #6497, #6509, #6578)

What's next

This is just the first version. Coming soon:

  • Rollback-and-retry, which smooths out transactions that need to fault in cold data.
  • Tiered index storage, so indexes can leave memory too.
  • Database forking, so you can clone databases cheaply.
  • A bunch of internal changes to allow cool features like time travel!

Auto-reconnect in the C++/Unreal SDK

The Unreal SDK now automatically reconnects after a dropped connection, including token refresh. The Blackholio demo has been updated to use it.

(#6318)

Breaking Changes

Republishing previously used database names is more restricted

Permissions for republishing a previously used database name have been tightened. A publish that previously succeeded may now be rejected if you don't hold the required permissions for that name.

(#6547)

TypeScript SDK: errors from app callbacks now fail the connection

Errors thrown from application callbacks no longer escape the SDK. Instead, a bad message or callback error fails the connection gracefully. Code that relied on catching errors escaping SDK calls will no longer see them propagate.

(#6010)

spacetime generate emits mod.rs instead of lib.rs

Rust module receiver bindings now generate mod.rs as their root file instead of lib.rs. If your workflow references the generated lib.rs by name, update it to mod.rs.

(#6593)

Bug Fixes

TypeScript and C# SDKs: old clients now skip newly added columns

Adding a column to the end of a table previously broke older clients: TypeScript threw RangeError and C# silently returned wrong values. Rows are now read using the server-provided row boundaries, so trailing columns unknown to the client's bindings are skipped, matching the Rust SDK's behavior. Removing, reordering, or retyping a column still breaks old clients.

(#6091)

TypeScript SDK: fix crash on React Native during automatic reconnect

(#6511)

V8 module startup protected from heap exhaustion

Module compilation, evaluation, and schema extraction now apply the existing memory-limit termination mechanism, so a module that exhausts its heap fails startup cleanly instead of crashing the instance.

(#6572)

Fix seeding collisions in the TypeScript module RNG

(#6323)

CLI: fix the logout endpoint path

(#6606)

Improvements

Faster first lookups in the TypeScript SDK cache

Indexes are now built on first lookup rather than eagerly, and unique single-column indexes use a flat map, improving memory use and lookup performance.

(#6614)

Rust SDK: disable Nagle's algorithm

TCP_NODELAY is now set on Rust SDK connections, reducing latency for small, frequent messages.

(#6552)

Rust SDK: disable brotli's default ffi-api feature

(#6444)

Procedures interrupted when a database is paused or suspended

Running procedures are now cancelled when their database is paused or suspended. Cancellation takes effect at the next with_tx, HTTP, or sleep_until boundary; a procedure with none of these calls is still not stopped.

(#6537)

What's Changed

  • Integrated tiered storage in #6305
  • Added auto-reconnect to the C++/Unreal SDK in #6318
  • Made old TypeScript and C# clients skip columns added to a table in #6091
  • Fixed TypeScript SDK crashing Node on a bad message in #6010
  • Fixed TypeScript SDK crash on React Native with automatic reconnect in #6511
  • Protected V8 module startup from heap exhaustion in #6572
  • Fixed seeding collisions in the TypeScript module RNG in #6323
  • Fixed the CLI logout endpoint path in #6606
  • Fixed database name publication in #6547
  • Built TypeScript SDK cache indexes on first lookup with a flat map for unique single-column indexes in #6614
  • Disabled Nagle's algorithm in the Rust SDK in #6552
  • Disabled brotli's default ffi-api feature in the Rust SDK in #6444
  • Interrupted procedures when a database is paused or suspended in #6537
  • Emitted mod.rs as the Rust receiver bindings root in #6593
  • Bumped the openssl crate to support openssl 4 in #6583

Full public changelog: v2.15.0...v2.16.1