Spice v2.4.0-rc.1 (October 8, 2026)

Spice v2.4.0-rc.1 is the first release candidate for v2.4.0. It adds performance improvements, S3 event-driven ingestion, SQL results-cache warmup, and adaptive HTTP rate controls. The release also upgrades to DataFusion v55, Ballista v55, Arrow v59, Vortex v0.86, Iceberg v0.11, and Turso v0.81.

Highlights in v2.4.0-rc.1 include:

• Performance & Query Engine: DataFusion v55, Vortex v0.86.1, Iceberg v0.11.0, and Ballista v55 update query planning, storage, and distributed execution.
• S3 Event-Driven Ingestion — ingest newly created objects through SQS notifications.
• Cache Warmup and Shared Fetches — warm the SQL results cache after the first refresh and coalesce concurrent source fetches.
• Adaptive HTTP Rate Controls — bound request waits and coordinate request limits across instances.
• ORC Support — query ORC files through listing connectors.
• Hugging Face Datasets: Query and accelerate files from the Hugging Face Hub.

What's New in v2.4.0-rc.1

Performance & Query Engine

This release upgrades Apache DataFusion to the v55.2.0 dependency line and Apache Arrow to v59.3.0. It also upgrades Vortex to v0.86.1, Apache Iceberg to the v0.11.0 fork, and Apache Ballista to v55.

Apache DataFusion v55

The DataFusion v55 release adds the following improvements:

• Sort pushdown and TopK pruning: Parquet scans reevaluate each unread row group as the threshold for ORDER BY ... LIMIT tightens. They skip groups that cannot contribute to the result. TopK pruning also supports multiple sort columns.
• Join planning: The optimizer converts eligible inner joins to semi joins and removes redundant sides of outer joins. It also orders filter predicates by estimated cost.
• Aggregation and expressions: Multi-column GROUP BY uses column-oriented storage for all supported key types, such as fixed-size binary UUIDs. More string functions preserve dictionary encoding, and IN lists use specialized paths for small integer types.
• Parquet reads: Scans skip nested fields that the declared schema does not contain. They also skip page-index reads when a file has no page index.
• Spill handling: Sorts bound the number of streams in a merge. If memory is insufficient, they spill the largest stream again in smaller batches.
• SQL diagnostics and functions: EXPLAIN accepts PostgreSQL-style options and FORMAT pgjson. New array functions cover element-wise addition, subtraction, scaling, sums, and averages.

Spice carries these changes through its query plans and preserves statistics across plan wrappers. Cayenne keeps Vortex scans below 10 MiB unsplit to avoid repeated footer reads. See #14612.

Vortex v0.79.0 to v0.86.1

Spice v2.3.2 used the Vortex v0.79.0 fork. This upgrade covers the full upstream range from v0.79.0 through v0.86.1, not only the v0.86 changes:

• Types and arithmetic: Vortex adds native Map arrays, Arrow map conversion, and operations and compression for maps. It also adds union arrays and decimal addition, subtraction, multiplication, and division.
• Row selection: Piecewise-sequence indices represent contiguous selections without an expanded index for every row. Specialized paths handle chunked arrays, fixed-size lists, variable-length lists, and binary values.
• Scans and expressions: Layout scans gain a physical plan and expression optimization. Filters can pass through scalar functions with multiple arguments. Filters on wide lists restrict child elements to the selected range, and constant masks can resolve from metadata.
• Compression: Binary arrays support FSST compression with variable-length offsets. OnPair becomes a stable encoding for reads and gains a storage-backed dictionary. Vortex can convert run-end arrays of lists and decimals.
• File metadata: Files can store custom metadata. Readers cache decoded type descriptions, and file-format editions define the supported encodings and types.
• Memory and execution: Builders append nested values in batches, preserve list views, and propagate buffer allocators. Expression rewrites retain unchanged nodes, and row functions support batch execution.
• Correctness and validation: NULL handling changes cover dictionary predicates, BETWEEN bounds, and empty arrays. Readers add validation for footer offsets, compression metadata, and array indices. Other corrections cover nested scalar hashes, decimal operations, and variable-length binary selections above 4 GiB.

See the complete Vortex v0.79.0 to v0.86.1 changelog for every upstream change. Changes to standalone Vortex bindings and GPU execution do not imply new Spice features.

Apache Iceberg v0.11.0

Spice updates its Iceberg reader and catalog integrations from v0.10.1 to the v0.11.0 fork for DataFusion 55 and Arrow 59. The upstream v0.11 changes extend reads and catalog compatibility:

• Iceberg v3 reads: The reader applies deletion vectors from Puffin files and carries row identifiers and sequence numbers through scans.
• Metadata-only scans: Metadata-only projections do not require data-column reads. Manifest reads reuse partition types, and positional-delete processing buffers runs instead of allocating a key per row.
• REST catalogs: Clients negotiate server-advertised endpoints and use session-scoped OAuth2 authentication.

The upgrade also aligns Iceberg storage with OpenDAL v0.58. See #14771.

Apache Ballista v55

Spice.ai Enterprise feature. See the Enterprise documentation.

The Ballista v55 upgrade adds virtual-core resource accounting for distributed tasks. A protocol-version handshake detects incompatible schedulers and executors. Cancellation identifies tasks by their task IDs, and task-state records use an append-only model.

Schedulers and executors must run the same version. Upgrade all cluster components together.

S3 Event-Driven Ingestion

S3 listing datasets can use refresh_mode: changes with S3 event notifications delivered through SQS:

datasets:
  - from: s3://my-bucket/events/
    name: events
    params:
      file_format: parquet
      s3_region: us-east-1
      s3_changes_queue_url: ${ secrets:events_queue_url }
    acceleration:
      enabled: true
      engine: cayenne
      mode: file
      refresh_mode: changes

New object notifications append the object's rows. A periodic listing backfill covers missed or expired notifications. Each dataset needs its own queue and permissions to read and delete SQS messages, alongside its S3 read/list permissions.

This is object ingestion: removal notifications are ignored by default. Set s3_on_object_removed: rebuild to rebuild the entire prefix when an object is removed. An overwrite of an already applied object key is not ingested again; use new object keys for incoming data. See #14121.

SQL Results-Cache Warmup and Shared Fetches

The SQL results cache can persist query plan shapes and replay them after a dataset's first full or append refresh:

runtime:
  caching:
    sql_results:
      enabled: true
      warmup: on_first_refresh

For example, run these queries against an accelerated orders dataset with warmup enabled:

SELECT id, status FROM orders WHERE id = 1;
SELECT id, status FROM orders WHERE id = 2;

Spice records one query shape because only the equality-filter value differs. After a restart and the dataset's first full or append refresh, warmup reruns that shape with distinct id values from the refreshed dataset, filling the cache before the dataset becomes ready. Keep the local .spice/data directory across restarts, or configure runtime.state.location, to retain recorded query shapes.

Warmup replays up to ten distinct recorded query shapes and tries up to 1,024 distinct filter-value combinations per shape, stopping when the cache is full. A dataset stays not ready until warmup completes. Later refreshes do not repeat warmup. The feature requires the default plan-based cache key; cache_key_type: sql is incompatible. See #14178.

Concurrent cache-miss fetches for the same request now share a source fetch. SQL results caching also includes changes for tables updated during query execution and for stale results served while revalidation runs. HTTP dataset caching supports RFC 5861 stale-if-error handling. See #14142, #14710, #14708, and #14134.

SQL, search, and embedding caches now use Spice's sharded cache backend. Existing engine: moka and engine: pingora values are accepted for configuration compatibility but no longer select a backend.

Adaptive HTTP Rate Controls

HTTP rate controls now adapt admission to upstream failures while staying within configured request limits. rate_control_acquire_timeout bounds how long a request waits for capacity and defaults to the connector's client ...

Spice v2.3.2 (September 22, 2026)

Spice v2.3.2 makes point lookups and repeated queries faster with Spice Cayenne. SQL results-cache improvements apply to every cached query, regardless of its data source or accelerator. Across two benchmark rounds, the time for a small cache hit fell by 50-52% over HTTP and 44-48% over Flight SQL. For small cache hits, CPU time per server request fell by 58-61% over HTTP and 49-51% over Flight SQL.

Highlights in v2.3.2 include:

• Indexed Point Lookups — Cayenne now honors indexes, so a lookup reads the matching rows instead of scanning
• Cayenne Data Layout — cayenne_cluster_by groups related rows together across storage tiers
• Faster Queries After a Full Refresh — refreshed tables are written so that filtered queries read fewer files
• Faster Cached Responses — these improvements apply to every query in the SQL results cache, regardless of its data source or accelerator. For small results, cache-hit time fell by 44-52%
• Faster Repeat Queries — Cayenne reuses its prepared view of a table until the data changes
• Localpod Fix — a localpod dataset no longer serves data its source has replaced
• Catalog Fix — tables no longer go missing when datasets load at the same time
• Safer Cayenne Configuration — an ambiguous metastore configuration is reported instead of appearing as lost data
• Distributed HTTP Queries — async distributed queries can read HTTP datasets

What's New in v2.3.2

Faster Point Lookups with Cayenne Indexes

Every other accelerator already accepted indexes, and Cayenne logged that it ignored them. Cayenne now builds an index for each entry, in both mode: file and mode: memory:

acceleration:
  engine: cayenne
  mode: file # or memory
  indexes:
    '(TenantId, ServiceId)': enabled

A query that pins every column of an index to a value, such as WHERE TenantId = 7 AND ServiceId = 'a', now reads the matching rows directly. Without an index, Cayenne can only skip files whose minimum and maximum values rule the key out, which rarely helps when related rows are spread across the table — a lookup on a 5.4M-row test dataset had to open about half its files for nearly every key.

Indexes only reduce what a query reads, so results are identical either way. A query that uses a range, an IN list, an OR, or a cast on the indexed column reads the table as before. Index definitions are not stored with the table, so adding or removing one takes effect the next time the dataset loads. Floating-point columns cannot be indexed and are reported at load time.

EXPLAIN shows whether a query used an index, and the cayenne_lookup_index_probe_total metric counts lookups by outcome.

Control How Cayenne Lays Out Data

cayenne_cluster_by stores rows with similar values near each other, so a filtered query reads fewer files. It applies to every storage tier, replacing cayenne_datalake_clustering_columns, which affected only the coldest tier.

acceleration:
  engine: cayenne
  params:
    cayenne_cluster_by: 'tenant_id, event_time'

CREATE TABLE ... CLUSTER BY (column, ...) is also supported, and several cases it previously rejected — a single column in parentheses, and names whose capitalization differs from the column definition — now work. A column name that does not exist is reported when the dataset loads rather than failing later.

Faster Queries After a Full Refresh

A full refresh previously spread each key across every file it wrote, so a filtered query had to open all of them even when it wanted a single row. A refreshed table is now written so that each file holds a distinct range of the data, and queries filtering on that range read only the files that can match.

This needs no configuration and applies to any dataset whose accelerated table is replaced by a refresh. Datasets that already set cayenne_sort_columns or cayenne_cluster_by keep their existing layout, and the very first load is unchanged.

Faster Cached Query Responses

Before this release, the runtime planned a query before it checked the SQL results cache. It also repeated other work that a cache hit does not need. The runtime now checks the cache first and returns cached answers directly. These improvements apply to every cached query, regardless of its data source or accelerator.

Across rounds of benchmarking queries that returned a seven-row GROUP BY result, cache-hit time over HTTP fell by 50-52%. Over Flight SQL, cache-hit time fell by 44-48%. In one HTTP round, cache-hit time fell from 63.8 µs to 30.8 µs. In one round over Flight SQL, it fell from 55.3 µs to 30.9 µs.

With task history enabled, the server used 125 µs of CPU time for a small HTTP hit before the change. It used 49 µs after the change. For a small result over Flight SQL, the server used 330 µs before the change. It used 168 µs after the change. These changes reduced CPU time by 61% and 49%, respectively.

For a wide result over HTTP, the server used 241 µs of CPU time before the change. It used 116 µs after the change. This is a 52% reduction. Wide results did not reduce the runtime's cache-hit time in those runs. Some measurements were slower.

Raw cache entries now share each stored batch with streaming JSON HTTP and Flight SQL responses. Buffered HTTP formats — including CSV, plain text, vnd.* envelopes, and JSON with union columns — still clone batches while encoding.

A stream benchmark with eight 20-column batches fell from 1.8722 µs to 226.23 ns. This is an 8.3x speedup. One 200-column batch fell from 2.0190 µs to 131.94 ns. This is a 15x speedup. These figures measure the construction and full consumption of the stream. They do not measure query latency from start to finish.

For compressed entries, the first read keeps the entry compressed. The second read promotes it to raw data when the raw size fits the cache. Later reads avoid decompression. In a cache benchmark, a later read took 17.1-17.8 µs across three payload sizes. A decode took 39.1-154.8 µs. Cached answers, expiry times, and cache settings are unchanged.

Faster Repeat Queries on Cayenne

Cayenne prepares a view of a table's current contents before it can answer a query. It now reuses that preparation until the data actually changes, instead of rebuilding it, so repeated queries against a table that is not being written return faster and stop querying the metastore entirely once warm.

Datasets fed by continuous change data capture — a cdc: or debezium: source, or refresh_mode: changes — reuse the preparation for up to one second so a burst of incoming changes can share it. Datasets that accept writes always see their own writes immediately. No configuration is required.

Better Pruning for Decimal, Binary, and Time Columns

Cayenne did not record minimum and maximum values for decimal, binary, time, and 16-bit float columns, so queries filtering on them could not skip files and had to read more data than necessary. Those columns now carry the same statistics as every other type, and clustering on a decimal column works as intended.

After a change that widens a decimal column's scale, Cayenne discards the affected statistics and rebuilds them, so queries read a little more until that completes. Results are unaffected.

Localpod Datasets Stay in Sync with Their Source

A localpod dataset reads through another dataset in the same Spicepod. When the source dataset was reloaded — because its configuration changed — the localpod dataset kept reading the replaced copy, so it answered with data the source no longer had, and both copies kept refreshing.

A localpod dataset now reloads whenever the dataset it reads through does, including through several levels of chaining, and its cached results are cleared at the same time. This also works when the source is named with its full path, such as localpod:spice.public.parent. Fixes #3288.

Catalog Tables No Longer Go Missing

When two datasets in the same schema loaded at the same time, one could be silently discarded, and every later query against it failed with Table not found. Both datasets reported that they had loaded successfully, and which one went missing varied between restarts. Datasets that share a schema now always both register.

Cayenne Detects an Ambiguous Metastore at Startup

Cayenne datasets that each set a different cayenne_file_path, without a shared cayenne_metadata_dir, could open the wrong metadata directory after a restart. Cayenne then started up empty even though the data was still on disk, which looks like a total loss of accelerated data.

This configuration is now rejected at startup, naming each dataset and path involved and linking to the documentation. If a Spicepod uses several Cayenne data paths, set the same cayenne_metadata_dir on each dataset before upgrading.

Distributed Queries over HTTP Datasets

An async distributed query submitted to /v1/queries failed if it read an unaccelerated HTTP dataset. These queries now run, with credentials, headers, and pagination behaving as they do for a non-distributed query. Fixes #14104.

Other Fixes

• **Part...

Spice v2.3.1 (September 15, 2026)

Spice v2.3.1 is a patch release that improves performance of filtered Vortex scans, cache eviction, and query planning. It also improves installation reliability, subquery fallback, numeric conversion, and Cayenne writes. The MCP server and client support the 2026-07-28 revision of the Model Context Protocol.

Highlights in v2.3.1 include:

• Ordered Full Refreshes — Cayenne applies sort_columns when a full refresh replaces a table
• CTE Materialization — Cayenne computes a multi-reference CTE once under runtime.query.cte_materialization: auto
• Faster Vortex Scans — selective filters read fewer bytes, and zones skip unrelated scan splits
• Faster IN Lists — Vortex uses a set probe for large constant lists
• Reusable Parameterized Plans — all parameter values use the same cached plan without sharing results
• Reliable Cache Eviction — declared keys and timestamp ties do not prevent eviction
• Installer and Container Reliability — the installer validates each archive, and scratch images provide a writable temporary directory
• Cayenne Memory-Mode DML — DELETE, UPDATE, and INSERT is now supported when the acceleration uses the default in-memory mode
• MCP 2026-07-28 — the MCP server and client speak the new revision, and legacy initialize clients keep working
• Connector Fixes — SMB share roots load on Samba, regexp_match answers faithfully on DuckDB, and S3 retry warnings read as one line

What's New in v2.3.1

Cayenne Applies sort_columns During Full Refreshes

Cayenne previously applied sort_columns only during compaction. A dataset with refresh_mode: full replaces its table, so it did not run that compaction. Cayenne now sorts the replacement stream before it writes the refreshed table, and the ordered files let min/max statistics exclude unrelated files from selective queries.

In the reported TPC-H SF1 benchmark, point-lookup p50 latency improved by 16%, and the same query used 37% less CPU. The sort adds work to each full refresh, so only datasets that configure sort_columns pay this cost. See Full Refresh for full-refresh configuration.

Cayenne Materializes Multi-Reference CTEs

DataFusion inlines a WITH body when it binds a query, so it plans and executes a twice-referenced CTE twice. This release adds the setting runtime.query.cte_materialization, which accepts disabled and auto. The default is disabled, and it keeps the previous behavior.

Under auto, Cayenne computes an expensive multi-reference CTE once into a memory-accounted buffer, and each reference reads that buffer. A CTE body counts as expensive when it contains an aggregation, a join, a window function, a distinct, a sort, an unnest, or a union. A pass-through CTE stays inlined, so projection pushdown can still prune its columns. The setting has no effect on a query that does not scan a Cayenne-accelerated dataset.

runtime:
  query:
    cte_materialization: auto # disabled (default) | auto

See the Runtime reference for the other query settings.

Faster Vortex Scans

Vortex now waits for a pushed-down filter before it prepares projected column readers, so a split with no matching rows does not prepare those readers. Selective queries in the reported benchmarks read 45% to 58% fewer bytes, and bandwidth-constrained storage showed the largest latency improvements.

Vortex also checks each scan split against its zone statistics, and it returns an empty stream when the split cannot contain a matching row. In the reported TPC-H SF1 benchmark, point-lookup p50 latency improved by 12%, and CPU use per query fell by 53%. Nonselective scans do not use these shortcuts and remain unchanged within the reported run-to-run variation.

Large IN Lists Run Faster

Vortex previously evaluated a constant IN list once for every row and every list item, so runtime grew with both dimensions. Vortex now builds a set once and probes it in one pass, and it uses value intervals to reject files before a scan.

A 32,768-value list over 1,048,576 rows fell from 44.57 seconds to 225 milliseconds in the reported benchmark. Small lists keep their existing evaluation path.

Parameterized Queries Reuse Logical Plans

The logical-plan cache previously included each bound parameter value in its key, so each value tuple created another entry for the same SQL text. The cache now stores an unbound plan for each SQL statement, and Spice binds the current values after it reads the plan. The SQL results cache still includes parameter values in its keys, so a result only matches a query with the same values.

The reported point-lookup benchmark improved p50 latency by 12.4% and p99 latency by 31.7%, and parameterized traffic creates fewer duplicate entries. Indexed primary-key lookups now report an exact row count to the query planner, so DataFusion can avoid an unnecessary repartition for a single-row probe.

Caching Accelerations Evict Reliably

A refresh_mode: caching dataset can use size and row-count limits, and two defects could let the cache exceed these limits. A declared primary_key could expand the eviction key to the entire stored row, and an index marked as unique had the same fault. A Map column in the expanded key then made every sweep fail. Spice now keeps the group key limited to the request columns, and a refreshed entry also removes rows that disappeared from its latest response.

Many responses can share the same one-second fetch timestamp. The previous eviction predicate could then remove only 512 entries when this tie crossed the survivor boundary. The eviction sweep now combines a timestamp range with named boundary entries. The range removes the older bulk, so eviction converges under sustained ingestion.

Federation and Runtime Improvements

• DuckDB and SQLite accelerations now handle more subqueries with on_zero_results: use_source. This includes mixed fallback policies and ANY or ALL comparisons.
• Decimal-to-float casts now preserve the expected rounded value after a federated aggregate. For example, a PostgreSQL average of 47.5 returns 47.5.
• Cayenne uses asynchronous filesystem calls from its asynchronous code. A slow data directory does not block the Tokio runtime at these call sites.

Installer and Container Reliability

The runtime installer now retries unsuccessful HTTP responses and invalid archives, and it accepts an archive only when it contains a regular spiced executable. The installer sends a configured token to GitHub release requests, and it does not send that token to runtime download hosts. On native Windows, spice upgrade now rejects the unsupported runtime upgrade instead of continuing with a partial upgrade.

The scratch-based standard and release images now provide a writable temporary directory at /app/tmp, which supports operations such as acceleration snapshot uploads.

Cayenne Memory-Mode DML Works as Expected

A Cayenne acceleration uses mode: memory by default. Before this release, filtered DML statements did not change its in-memory rows as requested.

• A filtered DELETE reported zero affected rows and left each matching row in place.
• An UPDATE could create a duplicate live row for the same primary key.
• An INSERT over an existing primary key could keep both versions of the row.

DELETE, UPDATE, and INSERT now update the in-memory tier. The append path also enforces each declared primary_key according to its on_conflict setting. The SQL DML reference lists the supported statements.

Cayenne Uses Less Memory for Primary-Key Work

Cayenne now releases the keyset bytes that an abandoned primary-key checkout counted. A cancelled validation stream, or an apply that fails before the store, previously left a phantom reservation behind. That reservation counted against the table's memory account and the keyset ceiling until the next publish replaced the figure. Sibling tables then received a smaller keyset budget than their real use warranted. Cayenne always discarded the keys themselves, so no query returned wrong rows.

Cayenne also allocates less during primary-key validation and deletion filtering:

• The membership set for incoming keys holds only the precomputed digests.
• Conflict and bloom checks borrow the encoded row. Cayenne copies the key bytes only for a row that it retains.
• A scan reuses its row-encoding buffers between batches. The runtime charges each retained buffer to the query memory pool.

MCP Specification 2026-07-28

Spice supports revision 2026-07-28 of the Model Context Protocol on /v1/mcp. A modern client calls server/discover and sends per-request _meta, and it can call tools/list and tools/call without a prior initialize request. These requests carry no session, so the runtime mints no Mcp-Session-Id. A legacy client keeps its current behavior. It sends initialize and notifications/initialized, receives an Mcp-Session-Id, and sends that header on each later request. GET /v1/mcp and DELETE /v1/mcp stay legacy-only.

The runtime accepts every revision from...

Spice v2.3.0 (September 9, 2026)

Spice v2.3.0 brings performance improvements, broader query federation, and expanded data connector capabilities. The release improves cache reuse and reduces memory use for cached SQL results. It also extends BigQuery support and adds GitHub review, release, and repository data for SQL analysis. Google models now use Vertex AI, so deployments with credentials for Google AI Studio require migration.

Highlights in v2.3.0 include:

• SQL Federation Improvements — bug fixes for function translation, NULL handling, and timezone declarations
• BigQuery Federation — temporal expressions, recursive CTEs, correlated subqueries, and window aggregates now run as one BigQuery job
• Caching Accelerator Limits — caching_max_size and caching_max_items bound what a refresh_mode: caching accelerator holds
• Stale-While-Revalidate Across a Refresh — configured stale-while-revalidate windows retain cached results across acceleration refreshes
• GitHub Review Data — eight new tables, 17 new pulls columns, and a repo / owner column on every row
• Google Models on Vertex AI — a from: google model authenticates as a GCP service account

What's New in v2.3.0

SQL Federation Improvements

This release improves SQL translation and function handling for accelerated and federated datasets. The following bug fixes cover string functions, NULL handling, correlated subqueries, and timezone declarations.

• A trim call failed on DuckDB, SQLite, and MySQL because DataFusion emitted its canonical name, btrim. DuckDB now receives trim, with an explicit space argument for the one-argument form. The native SQLite and MySQL paths evaluate btrim locally.
• DuckDB federation now lower-cases to_hex output to match local evaluation.
• DuckDB federation now decodes sha256 output into a 32-byte digest instead of a hexadecimal string.
• concat uses || on DuckDB to consistently propagate NULL arguments.
• inner_product now returns NULL for an undefined dot product on DuckDB for consistency.
• Operations that DuckDB cannot handle (such as regex with U or R flags) are executed locally.
• User-defined functions registered after startup now execute locally.
• Catalog connectors now use the same list of local Spice-only functions as the data connectors.
• Two EXISTS shapes emitted SQL which evaluated the correlation over the whole relation, so the bound selected nothing. A semi or mark join then reported a match on a row the plan never read. Both shapes now refuse pushdown and run locally.
• DuckDB labels a TIMESTAMPTZ column with the connection's own timezone. The connector built its pool and never pinned that setting. A dataset's schema therefore carried the host timezone, and the same query returned different rows on different machines. A connector session is now pinned to UTC.

BigQuery Federation

BigQuery federation runs more query shapes as one remote job.

• Temporal expressions, recursive CTEs, and integer division now keep their results and stay in one federated statement. These shapes previously failed remotely, split into several queries, or produced wrong cohort boundaries.
• Three more statement shapes now run. The first is a grouped query that projects a wrapped form of its grouping expression. The second is a query whose federated tables all sit inside a correlated subquery. The third is any aggregate window function.
• A query that reads BigQuery tables from several datasets of one project now runs as one BigQuery query. It also no longer returns rows from the wrong dataset when two datasets hold a table of the same name.
• A dataset that stores JSON in a STRING column now pushes down scalar json_as_text expressions and json_get(...) IS NULL checks.
• The built-in date_trunc is preserved, and the dialect and the federation policy now agree on which aggregate and window calls are eligible.
• regexp_match null checks federate safely.
• A distinct union now renders as UNION DISTINCT. BigQuery rejects a bare UNION, so such a query failed outright. UNION ALL is unchanged.
• A numbering function such as ROW_NUMBER no longer carries a window frame, which BigQuery rejects. An aggregate window function keeps its frame.
• Percentile functions and grouping keys now render in the form BigQuery accepts. A reported 29-statement workload that failed on these shapes now runs in full.
• array_element translates a non-negative integer literal index that fits in Int64 as SAFE_ORDINAL. Other indexes evaluate locally. This preserves DataFusion's end-relative semantics for negative indexes.

Cancellation: A BigQuery query whose client goes away now stops. The BigQuery job ends as cancelled, and the pooled connection returns immediately. Before this release the query ran to completion, and the connection stayed busy for its whole duration. Enough cancellations left an application unable to query at all.

The Caching Accelerator Accepts Explicit Limits

A refresh_mode: caching accelerator had nothing bounding what it held. Retention was derived from caching_ttl plus caching_stale_while_revalidate_ttl, and only when caching_stale_if_error was disabled. A dataset that set caching_stale_if_error therefore got no policy at all, and nothing was ever evicted. Nothing capped the acceleration by size or by count either.

Two settings now bound the acceleration. Each one refuses an unparseable value rather than falling back to a default:

• caching_max_size — a byte budget, such as 512MiB.
• caching_max_items — a row budget.

caching_ttl is also accepted as caching_item_ttl, which is the spelling the SQL results, search results, and embeddings caches use. Eviction is entry-granular. A cached response can span several rows, so the runtime ranks entries by their oldest page and removes all of an entry's rows together. The storage schema is unchanged, and no existing acceleration needs a rebuild.

datasets:
  - from: https://api.example.com/v1/items
    name: items
    acceleration:
      enabled: true
      engine: duckdb
      refresh_mode: caching
      primary_key: '(request_query, request_path)'
      params:
        caching_ttl: 5m
        caching_max_size: 512MiB
        caching_max_items: 50000

caching_stale_if_error now fires on the failure it exists for. It keyed off a fetch that returned an error. The HTTP connector reports a failing origin as a successful fetch whose rows carry a 429 or 5xx status once it exhausts max_retries. An operator who enabled the setting received the origin's error body instead of the cached response.

A caching accelerator that has nothing bounding it now says so at startup.

SQL Results Cache Improvements

Stale results remain available across a refresh when configured. An acceleration refresh evicted every dependent SQL results cache entry, and any successful refresh counted as a change. A refresh_mode: full update still flushed the whole per-table cache. For a workload with consistently high QPS, each refresh turned a population of cached results into simultaneous synchronous misses.

When stale_while_revalidate_ttl is configured, an invalidation now marks dependent entries stale as of the refresh instead of evicting them. Inside the stale window the runtime serves the previous result with Results-Cache-Status: STALE and starts one background revalidation per key. Past the stale window the request is a miss, exactly as before. With no stale window configured, invalidation stays hard.

Cache accounting covers more retained memory. A cache with a million empty results reported 0.09 GiB against 1.85 GiB of retained memory. Its max_size accounting omitted parts of each entry. The cache now shares schemas and input-table sets, copies foreign buffers, and accounts for per-buffer allocation overhead.

In the reported benchmark, each entry retained 1%–80% less memory across 20 combinations of result shape and source. The benchmark ran on macOS/arm64 with snmalloc. Reported size ranged from 0.65x to 1.73x of retained memory after the change. The lowest ratio before the change was 0.31x. These figures compare the fix with its merge base, not v2.2.1, and are not guarantees for every workload.

This release also corrects memory accounting for the search results and embeddings caches.

Pingora cache engine: Table invalidation read every entry with a destructive get, so an invalidation promoted every key in the cache. Scan order replaced recency, and each visited key became a momentary miss to concurrent readers. A read also served a hit destructively, so a second reader reported a miss for a key the cache holds. Both reads are now non-destructive.

GitHub Data Connector: Review, Release, and Repository Tables

The GitHub Data Connector adds eight tables, 17 columns on pulls, and repository identity on every row. An application can now answer a code-review question in SQL. Before this release, pulls.reviews_count was a bare integer with no state and no reviewer, and pulls.review_comments recorded only inline comments.

Path	Rows
github.com/{owner}/{repo}/reviews	One per pull request review, with state, author, submitted_at, and commit_sha ...

Spice v2.2.1 (September 1, 2026)

Spice v2.2.1 is a patch release that improves accelerated data management, query federation, and search. Spice Cayenne now reclaims the storage that upserts and compaction leave behind, and new metrics report what each maintenance pass did. Retention policies apply to more Cayenne datasets. Federated queries push more work down to BigQuery. DuckDB and HTTP datasets keep their memory bounded. Vector search with HuggingFace embedding models returns more relevant results.

What's New in v2.2.1

Cayenne Reclaims Storage Automatically

A Spice Cayenne acceleration now reclaims the space that a write or a compaction supersedes. Each maintenance pass removes four kinds of state that earlier releases kept:

• tombstones for superseded upsert rows
• snapshot directories that a later commit replaced
• manifest and catalog rows for snapshots that compaction merged away
• deletion vectors that a compaction orphaned

Cayenne also releases the metastore pages that a pass frees. On a table with frequent refreshes or CDC upserts, disk and metastore usage now stay bounded. No configuration change is required.

Retention Policies Apply to More Cayenne Datasets

Cayenne now applies retention_sql on datasets with refresh_mode: full and on CDC-accelerated datasets with refresh_mode: changes. Before this release, only the append refresh path ran the retention pass. A full refresh reloaded every source row, and the rows that a policy had removed came back.

A retention_sql predicate that calls now() also works now. Cayenne prepared the predicate once when the table opened, at a point where DataFusion cannot evaluate now(), so the pass failed and removed nothing. Cayenne now resolves now() at the start of each pass. See Data Refresh for the retention settings.

datasets:
  - from: postgres:events
    name: events
    acceleration:
      engine: cayenne
      retention_sql: DELETE FROM events WHERE created_at < now() - INTERVAL '7 days'

Cayenne also accepts refresh_append_overlap now. It warns when a dataset sets retention_sql together with mode: memory, or sets indexes, because it cannot apply either setting.

Cayenne Maintenance and Storage Metrics

Cayenne now reports its maintenance decisions and its storage footprint as metrics on the cayenne meter. The runtime serves them at /metrics when spiced starts with --metrics <addr>. An operator can now tell a pass that found nothing from a pass that declined. The metrics also show where disk and metastore space goes. See Metrics for the metrics endpoint.

• cayenne_compaction_outcome_total and cayenne_maintenance_outcome_total count each pass by outcome, such as committed, no_op, or declined_<reason>.
• cayenne_compaction_trigger_total names the threshold that requested a pass.
• cayenne_maintenance_reclaimed_files_total, cayenne_maintenance_reclaimed_bytes_total, and cayenne_maintenance_reclaimed_rows_total count what each pass reclaimed.
• cayenne_storage_files, cayenne_storage_bytes, and cayenne_storage_rows report the live footprint by storage tier.
• cayenne_data_dir_files, cayenne_data_dir_bytes, and cayenne_data_dir_snapshot_dirs report what the data directory holds on disk.
• cayenne_metastore_db_bytes, cayenne_metastore_wal_bytes, and cayenne_metastore_table_rows report the size of the metastore.

Query Federation and SQL Improvements

• ADBC federation: The ADBC Data Connector again keeps Spice-only functions such as json_get_str out of the SQL it sends to a source, and it honours query_federation: disabled. The v2.2.0 move of the connector into its own crate dropped both settings. A query with a Spice-only function against BigQuery then failed with an unknown-function error.
• JSON functions in BigQuery: BigQuery federation now translates json_get_int and json_get_float into BigQuery SQL. A filter or a join on a JSON value runs in BigQuery instead of in Spice.
• Negative JSON numbers: json_get_int and json_get_float now read a negative JSON number. Every earlier release returned NULL for one.
• Joins between described datasets: An INNER or FULL join between two datasets that both set a description now plans. DataFusion carries the description as schema metadata, and the join failed with a schema mismatch error. This release updates DataFusion with the fix.

Cayenne Query and Delete Fixes

• Predicate deletes on upsert tables: A DELETE with a non-primary-key predicate on a Cayenne upsert table now removes only the rows that match. The delete matched the predicate against superseded row versions as well as live rows. A key whose old version matched therefore lost its live row too. The delete now applies the same visibility rules as a query.
• Date filters on Cayenne data files: A pushed-down CAST(date_col AS TIMESTAMP) comparison failed the scan for a Date32 column. For a Date64 column, the same comparison skipped files that held matching rows, so the query returned fewer rows. The scan cast the file's date statistic through the wrong storage type. It now converts the statistic before it prunes on it.
• Primary keys after an interrupted index rebuild: A rebuild of the primary-key index that failed or was cancelled left the index disabled for the life of the process. A later upsert could then write a second live row for a declared primary key, and every write rebuilt the whole keyset. Cayenne now closes the rebuild window on every exit path.
• Row counts after a commit: A distributed COUNT(*) that Cayenne answered from its maintained row count could miss the rows of a commit that had just become visible. Cayenne now claims a commit's row-count delta before it publishes the rows.
• Point lookups on CDC datasets: A primary-key lookup with an ORDER BY on the same key failed to plan on a CDC-accelerated Cayenne dataset after its first transactional commit. An example is WHERE id = 7 ORDER BY id. The runtime rejected the query and returned an error. The scan now forwards its constant columns across the schema cast, and the query plans again.
• Arrow Map columns in file mode: A Cayenne accelerator with mode: file now stores Arrow Map columns, such as the response_headers column of an HTTP dataset. Every write of such a column failed, and the acceleration stayed empty.

DuckDB and HTTP Datasets Keep Memory Bounded

DuckDB: The bundled DuckDB moves from v1.5.5 to v1.4.4. On v1.5.5, a DuckDB-accelerated dataset that refreshed through an ON CONFLICT ... DO UPDATE upsert grew its memory by about 6 MiB per refresh. The row count stayed steady while the memory grew, and the write eventually failed with an out-of-memory error. The DuckDB project tracks the regression as duckdb/duckdb#25162, and it is present in every v1.5.x release. v1.4.4 holds a steady footprint. Queries that run during a file-mode DuckDB swap also now recover when the swap invalidates their statements. The configured database path stays attached across the swap.

HTTP: The HTTP Data Connector now bounds its response cache. The cache kept every fetched response for the life of the process. An API-proxy workload with request-shaped keys therefore grew by about 1 MB per distinct request.

The cache now holds at most response_cache_max_size_bytes per dataset, 64 MiB by default, and evicts the oldest entries first. It expires an entry by the origin's max-age and Age headers, and it refuses to store a response that sends no-store or no-cache. The new response_cache_fallback_ttl setting keeps a response from an origin that sends no Cache-Control header at all. Two new metrics, response_cache_size_bytes and response_cache_items_count, report occupancy per dataset.

datasets:
  - from: https://api.example.com/v1/items
    name: items
    params:
      response_cache_max_size_bytes: 16777216
      response_cache_fallback_ttl: 5m

More Relevant Vector Search with HuggingFace Embeddings

Vector search with a HuggingFace sentence-transformer embedding model, such as all-MiniLM-L6-v2, now returns relevant results. The runtime kept the fixed padding that some models declare in tokenizer.json, so it padded every input to 128 tokens. The padding then dominated the vector of a short input. The runtime now clears that padding and lets the embedding layer own the attention mask. On the MTEB SciFact benchmark with all-MiniLM-L6-v2, nDCG@10 rises from 0.018 to 0.640 (published: 0.645). Full-text search and model2vec static models were not affected.

A search filter on a timezone-aware timestamp column now matches the exact instant when the runtime pushes it down to a DuckDB-backed search index. The filt...

Spice v2.2.0 (August 24, 2026)

Spice v2.2.0 focuses on real-time data, performance, and stability. Cloud Connect links self-hosted runtimes to Spice.ai Cloud for management and observability. MySQL datasets and PostgreSQL catalogs can now stream source changes into accelerated datasets. Debezium sources can send CDC events directly without Kafka. Warm in-memory indexes improve vector and full-text search performance.

Highlights in v2.2.0 include:

• Cloud Connect — link self-hosted runtimes for BYOC (Bring Your Own Cloud) to Spice.ai Cloud for management and observability
• MySQL CDC — MySQL tables now stay in sync in real time using MySQL binlog replication, with no Kafka or Debezium infrastructure and no scheduled refreshes
• PostgreSQL Catalog CDC — accelerate an entire PostgreSQL database in real time with a few lines of configuration and no per-table setup
• Faster Search — vector and full-text search can now serve from an in-memory index by default, with no configuration change

What's New in v2.2.0

Cloud Connect

Spice Cloud Connect is a feature of Spice.ai Cloud.

Spice Cloud Connect links self-hosted runtimes for BYOC (Bring Your Own Cloud) to Spice.ai Cloud for management and observability. Run spice cloud link to enroll a standalone runtime.

Try it with the Cloud Connect on a Development Machine recipe.

MySQL Change Data Capture

The MySQL Data Connector now supports refresh_mode: changes. The runtime loads an initial snapshot of the table. It then applies inserts, updates, and deletes from the MySQL binlog as they commit. The acceleration stays current without scheduled refreshes.

datasets:
  - from: mysql:orders
    name: orders
    params:
      mysql_host: localhost
      mysql_db: mydb
      mysql_user: replicator
      mysql_pass: ${secrets:mysql_pass}
    acceleration:
      engine: cayenne
      refresh_mode: changes

• Failover-safe GTID positions: When the source server has GTID enabled, the runtime tracks the replication position as a GTID set. A GTID position stays valid across a replica promotion, so replication survives a failover to a new primary without a rebuild.

PostgreSQL Catalog CDC

The PostgreSQL Catalog Connector is now Beta with new Catalog CDC acceleration support at Alpha.

A PostgreSQL catalog can now use refresh_mode: changes. One configuration replicates every table that the include patterns match. Queries then read fresh PostgreSQL data with no per-table setup.

catalogs:
  - from: pg
    name: pg
    include:
      - 'public.*'
    params:
      pg_host: localhost
      pg_db: mydb
      pg_user: postgres
      pg_pass: ${secrets:pg_pass}
    acceleration:
      refresh_mode: changes

Known limitations while catalog CDC acceleration is Alpha: configuration may change; a table dropped and recreated in the source can serve rows captured before it was recreated (#12110); and a durable catalog acceleration can come back empty after a restart (#12729). Feedback is welcome in #11850.

Debezium CDC Without Kafka

Any Debezium source plugin can now stream change events directly into Spice, with no Kafka bus in between. A dataset with from: cdc:… accepts change events at POST /v1/datasets/{name}/cdc, in JSON or Avro. The existing Kafka path (from: debezium:…) is unchanged.

Faster Search with Warm In-Memory Indexes

Search adds a warm in-memory tier for vector and full-text indexes. The runtime writes each change to the warm tier and to the durable store together. Queries read the warm tier first and fall back to the durable store. The warm tier covers vector indexes, full-text indexes, .vectors datasets, views, and chunked Elasticsearch vector columns.

More search improvements:

• cosine_distance uses SIMD instructions through the simsimd library.
• Full-text search applies stemming by default.
• Full-text search pushes SQL filters down into the tantivy index.
• A delete now removes the document from full-text and vector indexes. BM25 statistics no longer count superseded documents.
• The runtime loads local rerankers from text-embeddings-inference models.
• The runtime validates vector search parameters before it runs the SQL query.

Operations & Observability

• Query timeout: The new runtime.query.timeout setting bounds the duration of every query. An expired query fails with HTTP 504 or gRPC DEADLINE_EXCEEDED.

runtime:
  query:
    timeout: 30s

• CPU sizing for Kubernetes pods without a CPU limit: A Spice runtime pod with a CPU request and no CPU limit now sizes itself to twice the request instead of every core on the node. Set SPICE_CPU_CORES=all (or runtime.cpu.cores: all) to burst to all cores, for example when a 0.5-core request runs on a 24-core node.
• Trace IDs in logs: Every log record from a query carries the query's trace ID. Filter the logs by one ID to see everything that query did.
• Improved dashboards: The Grafana and Datadog dashboards add new and improved panels, and support multi-replica and Kubernetes deployments.
• Helm: The Helm chart supports a custom Deployment strategy and StatefulSet updateStrategy.
• Dataset status: A non-accelerated dataset now shows the Error state when its source is unavailable, instead of appearing healthy while queries fail.

Other Notable Improvements and Bug Fixes

• Drasi (Alpha): The runtime forwards CDC changes and runtime tables to a Drasi source.
• Microsoft SQL Server: Kerberos integrated authentication now works on Unix.
• HTTP connector: OAuth2 client-credentials authentication.
• Oracle DATE: Values lost their time of day. They now map to a timestamp.
• Catalog exclude patterns: The Glue and Cayenne catalogs ignored exclude patterns. Both pattern lists now apply.
• Spice Cayenne schema changes: Partitioned accelerations could report a source schema change as applied while partitions kept the old schema. This mismatch caused lossy casts or append failures. Cayenne now rejects in-place evolution for partitioned accelerations. The configured schema-change policy then handles the change.
• Snowflake NUMBER values: The connector could remove fractional digits during schema discovery. It treated numeric metadata as absent and defaulted the scale to zero. It now preserves the source precision and scale.

See the Changelog for the full list of fixes.

SDK Updates

Updated SDKs release alongside v2.2.0:

• spice.js v3.2.0 — adds query cancellation (listActiveQueries/cancelActiveQuery), mTLS client certificates, HTTP fallback when Flight is unavailable, and typed parameter binding via Flight SQL prepared statements. Also fixes nsql() and search() response handling and named parameters over HTTP.
• spicepy v4.0.0 — adds streaming of large results, natural-language queries (nsql), vector and hybrid search, and query cancellation. Spice.ai Cloud users need this update: the legacy hostnames are retired in favor of region-specific endpoints.
• spice-rs v3.2.0 — adds search, NSQL, async queries, query cancellation, and mTLS.

Dependency Updates

Dependency / Component	Version
DataFusion	v54.1
iceberg-rust	v0.10.0
Turso	v0.7.2
Rust toolchain	v1.96.1

New Contributors

• @boittega made their first contribution in #11509
• @lesbass made their first contribution in #12253

Contributors

• @bjchambers
• @boittega
• @ewgenius
• @Jeadie
• @krinart
• @lesbass
• @lukekim
• @peasee
• @phillipleblanc
• @sgrebnov
• @v1gnesh

Breaking Changes

• CPU sizing for Kubernetes pods without a CPU limit: A Spice runtime pod with a CPU request and no CPU limit now sizes itself to twice the request instead of every core on the node. Set SPICE_CPU_CORES=all (or runtime.cpu.cores: all) to keep the previous behavior, or set runtime.cpu.cores to a specific core count.
• Partitioned DuckDB accelerations are removed: The DuckDB accelerator now rejects partition_by. Use the Cayenne or Arrow accelerator for partitioned datasets.
• Adaptive Cayenne tuning requires an explicit opt-in: An unset cayenne_tuning now resolves to auto. Set cayenne_tuning: adaptive to enable the closed-loop controller.
• ONNX ML inference is removed...

Spice v2.1.5 (August 12, 2026)

Spice v2.1.5 is a patch release focused on dependable cached data and predictable operation under load. Cached queries now reflect data changes more reliably, Cayenne workloads stay within configured memory limits, health checks remain responsive while cached results are updated, and cache dashboards provide a more complete view of activity.

What's New in v2.1.5

Cached Queries Stay Fresh as Data Changes

Cached results are now cleared reliably after refreshes, writes, retention changes, and updates to dependent local datasets. Expired entries are removed promptly, and entries for data that is no longer part of a Cayenne dataset are not reused by later queries.

These improvements prevent a completed data change from being followed by an older cached answer. No configuration changes are required.

More Predictable Cayenne Behavior Under Heavy Load

Cayenne now keeps track of the memory needed to prepare query results, including when several parts of a query are prepared at once. Multiple accelerated tables also share available memory instead of each planning as though it were the only table in the deployment.

Large and concurrent workloads are therefore less likely to exhaust the available memory. When a query cannot fit within the configured limit, it fails cleanly instead of putting the entire service at risk. Operators can also limit how much work one dataset does at once when it needs a smaller memory footprint, without slowing every query.

Health Checks Remain Responsive During Cache Updates

Refreshing or writing a dataset with many cached results no longer holds up Spice while it finds old answers that need to be cleared. Health checks and other requests can continue during this work, reducing avoidable service restarts under load.

More Trustworthy Cache Dashboards

Cache dashboards now show total space, space in use, stored results, requests, and successful reuse whenever the dashboard is refreshed, including for new or lightly used datasets. Counts for expired results, automatic size cleanup, and cleanup after data changes are also reported consistently, so a zero value represents no activity rather than missing information.

Contributors

• @bjchambers
• @krinart
• @lukekim
• @phillipleblanc

Breaking Changes

No breaking changes.

Cookbook Updates

No new cookbook recipes.

The Spice Cookbook includes more than 100 recipes to help you get started with Spice quickly and easily.

Upgrading

To upgrade to v2.1.5, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:2.1.5 image:

docker pull spiceai/spiceai:2.1.5

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.1.5

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed

Changelog

• fix: arrow IndexedMemTable serves stale results after DML/retention/sync (fixes #11262) by @claudespice in #11532
• fix(cache): evict the Pingora cache down to max_size instead of only recording it (fixes #12688) by @grokspice in #12694
• fix(cache): count an invalidation as an eviction, and export the counters before one fires (fixes #12687) by @grokspice in #12791
• fix(cache): never serve a result whose tables changed after it read them by @bjchambers in #12703
• Remove unnecessary allocations from results-cache and hot conversion paths by @phillipleblanc in #11895
• fix(cache): read and expire a Pingora entry under one hold of its shard (fixes #12832) by @grokspice in #12839
• fix(runtime): invalidate localpod children's cached results on parent refresh by @krinart in #12897
• fix(cache): run the Pingora invalidation scan off the calling runtime worker (fixes #12806) by @grokspice in #12808
• fix(cache): count the removals the Pingora engine performs itself (fixes #12792) by @grokspice in #12830
• feat(cayenne): charge scan materialization to the query memory pool by @lukekim in #12759
• feat(cayenne): bound the SUM of the per-table PK keyset caches by @lukekim in #12802
• Charge the query memory pool for concurrent Vortex split decodes by @lukekim in #12940
• fix(vortex): invalidate retired segment cache entries by @phillipleblanc in #12943
• fix(vortex): expose segment cache metrics on scrape by @phillipleblanc in #12944

Full Changelog: v2.1.4...v2.1.5

Spice v2.1.4 (August 5, 2026)

Spice v2.1.4 is a patch release that improves DuckDB acceleration and runtime stability: datasets with UTC timestamp columns, such as Iceberg tables, now load successfully when accelerated with DuckDB, retention policies run reliably, and the runtime is more resilient under heavy query load.

What's New in v2.1.4

DuckDB Acceleration Works with Iceberg Timestamps

A dataset with a UTC timestamp column — for example, any Iceberg timestamptz column — could previously fail to load when accelerated with DuckDB, leaving the dataset unhealthy and unqueryable. These datasets now load and become ready normally, with no configuration changes needed.

Reliable Retention Policies on DuckDB

Retention policies now apply cleanly on DuckDB-accelerated datasets, including policies that combine a time window with an additional condition. Expired rows are evicted on every retention interval, keeping accelerated data fresh and storage bounded.

Improved Runtime Stability

The runtime is now more robust when queries are cancelled — whether by a client disconnecting, a timeout, or a new refresh superseding an in-flight read. A rare crash in this path has been eliminated.

Contributors

• @phillipleblanc
• @peasee

Breaking Changes

No breaking changes.

Cookbook Updates

No new cookbook recipes.

The Spice Cookbook includes more than 100 recipes to help you get started with Spice quickly and easily.

Upgrading

To upgrade to v2.1.4, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:2.1.4 image:

docker pull spiceai/spiceai:2.1.4

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.1.4

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed

Changelog

• chore(deps): bump datafusion and table-providers for the DuckDB timezone and retention fixes (2.1 backport) by @phillipleblanc in #12546
• fix(deps): bump the Vortex pin to pick up the task-cancellation fix by @phillipleblanc in #12547

Full Changelog: v2.1.3...v2.1.4

Spice v2.1.3 (August 4, 2026)

Spice v2.1.3 is a patch release focused on resource efficiency: explicit CPU sizing with the new runtime.cpu.cores setting, more query memory for Cayenne deployments, and improved memory and crash diagnostics. It also fixes Cayenne acceleration of Iceberg datasets with timestamptz columns and restores WHERE filters on federated LEFT/RIGHT JOIN queries.

What's New in v2.1.3

CPU Sizing with runtime.cpu.cores

The new runtime.cpu.cores setting controls how many cores the runtime targets. Thread pools, query partitioning, and accelerator concurrency are all derived from it.

runtime:
  cpu:
    cores: 4 # `auto` (default) detects. Accepts 4, 3.5, 3500m

Also available as --cpu-cores and SPICE_CPU_CORES (precedence: flag > environment > Spicepod).

This is most useful on large, shared nodes. A pod that sets resources.requests.cpu without a CPU limit exposes no cgroup quota, so the runtime sizes itself for every core on the node rather than its allocated share. Setting the entitlement aligns parallelism and memory footprint with the CPU the pod actually receives.

The effective value, its source, and the derived sizing are logged at startup and exported as the spiced_cpu_budget_cores gauge.

More Query Memory for Cayenne Deployments

The Cayenne compaction memory pool is now reserved only for accelerations that can compact into it: file mode with a small-write refresh profile. Other deployments, including refresh_mode: full, keep the full memory limit available to queries — up to 6.4 GiB on a 32 GiB limit, with no configuration change.

Memory budgets are now derived from the process's own cgroup limit rather than total host memory.

Diagnostics

Three new gauges report memory in use: query_memory_pool_used_bytes, cayenne_compaction_memory_pool_used_bytes, and process_resident_memory_bytes.

Memory pool refusals now return ResourcesExhausted and HTTP 503, distinguishing them from query errors.

Fatal native signals (SIGSEGV, SIGBUS, SIGILL, SIGFPE) report the signal, faulting address, and thread before exit, so a crash can be diagnosed from logs.

Fixed a bug where setting runtime.task_history.enabled: false also disabled every query metric — query_duration_ms, query_execution_duration_ms, query_executions, query_failures, query_returned_rows, and query_returned_bytes. These are now reported regardless of the task history setting.

Cayenne Acceleration of Iceberg timestamptz Columns

Accelerating an Iceberg dataset with a timestamptz column using the Cayenne engine previously failed during the refresh write with an error resolving the time zone +00:00. Iceberg maps every timestamptz column to the fixed-offset Arrow time zone +00:00, which the file writer could not resolve when building column statistics. Fixed-offset time zones (±HH:MM, ±HHMM, and ±HH) are now resolved wherever time zones are handled, so these datasets accelerate correctly.

Federated Outer Join Filters

A federated query combining a LEFT JOIN with a WHERE filter on the left table previously returned all rows instead of the filtered rows: when the query was pushed down to the data source or accelerator, the filter was folded into the JOIN ON clause, where it no longer filters the left side (RIGHT JOIN was affected symmetrically). Filters now stay on the side of the join they came from, so these queries return the correct rows.

Contributors

• @lukekim
• @sgrebnov
• @phillipleblanc
• @bjchambers
• @Jeadie

Breaking Changes

No breaking changes.

Cookbook Updates

No new cookbook recipes.

The Spice Cookbook includes more than 100 recipes to help you get started with Spice quickly and easily.

Upgrading

To upgrade to v2.1.3, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:2.1.3 image:

docker pull spiceai/spiceai:2.1.3

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.1.3

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed

Changelog

• feat(runtime): size every CPU-derived pool from the CPU entitlement by @bjchambers in #12276
• fix(runtime): carve the Cayenne compaction memory pool only when a dataset can compact into it by @sgrebnov in #12326
• fix(runtime): size memory budgets from the process's own cgroup limit by @lukekim in #12263
• fix(telemetry): read the cgroup CPU quota along the whole cgroup path by @sgrebnov in #12300
• feat(runtime): expose the memory numbers that explain an OOM as gauges by @lukekim in #12195
• fix(runtime): report a memory-pool refusal as ResourcesExhausted and answer it with 503 by @sgrebnov in #12289
• fix(cayenne): the write-concurrency raise must respect the memory brake by @lukekim in #12317
• feat(spiced): report fatal signals before exit by @sgrebnov in #12334
• fix: report query metrics when task history is disabled by @sgrebnov in #12227
• chore(deps): repoint vortex at the 2.1 fixed-offset timezone fix by @phillipleblanc in #12455
• chore(deps): bump datafusion rev for outer-join unparser fix by @Jeadie in #12460

Full Changelog: v2.1.2...v2.1.3

Spice v2.1.2 (July 29, 2026)

Spice v2.1.2 is a patch release focused on improving the DuckDB data accelerator. It upgrades the DuckDB engine to v1.5.5 and introduces the on_full_refresh parameter, giving file-mode DuckDB accelerations compact, predictable disk usage across repeated full refreshes.

What's New in v2.1.2

Bounded DuckDB Acceleration File Growth with on_full_refresh

File-mode DuckDB accelerations using refresh_mode: full now reclaim disk space on every refresh, keeping the database file compact and disk usage predictable for long-running deployments. Each full refresh bulk-loads a fresh copy of the data, and the new on_full_refresh modes ensure the space held by prior copies is returned rather than accumulating in the file.

The new on_full_refresh acceleration parameter controls how disk space is reclaimed after each full refresh:

acceleration:
  engine: duckdb
  mode: file
  refresh_mode: full
  params:
    duckdb_file: /data/shared.duckdb
    on_full_refresh: replace_file # default: reuse_file

• reuse_file (default): Existing behavior — refresh into the existing database file.
• replace_file: Each full refresh streams data into a fresh staging database file, carries over every other object sharing the file (other datasets' tables, views, indexes, and Spice metadata), checkpoints it, and atomically replaces the configured file. Queries are never interrupted — in-flight queries drain against the old file while new queries see the new file — and space is fully reclaimed on every refresh.
• checkpoint_file: After each refresh, run a CHECKPOINT in place, escalating to FORCE CHECKPOINT when concurrent transactions block the plain attempt (waiting for in-flight transactions; never aborting them).

DuckDB 1.5.5

The DuckDB engine is upgraded from v1.5.3 to v1.5.5, bringing the latest upstream stability fixes.

Contributors

• @lukekim
• @sgrebnov
• @bjchambers
• @krinart

Breaking Changes

No breaking changes.

Cookbook Updates

No new cookbook recipes.

The Spice Cookbook includes more than 100 recipes to help you get started with Spice quickly and easily.

Upgrading

To upgrade to v2.1.2, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:2.1.2 image:

docker pull spiceai/spiceai:2.1.2

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.1.2

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed

Changelog

• feat(duckdb): on_full_refresh: replace_file — full refresh into a new database file, atomically replaced by @lukekim in #12135
• feat(duckdb): 'on_full_refresh: checkpoint_file' to bound acceleration file growth by @sgrebnov in #12139

Full Changelog: v2.1.1...v2.1.2