v0.0.16

Highlights

• Add direct partitioned Dask DataFrame reads through ADBC, with explicit query parameters and Arrow schemas, worker-local connections, and preservation of nullable types and empty partitions (#357).
• Add the dask Python extra, a runnable two-process Distributed example, and installation, partition sizing, and consistency guidance.
• Add the standalone redis_adbc DuckDB extension and make DuckDB 2.0 the primary build/runtime target (#344, #354). Its extension build and distribution follow the separate DuckDB workflow; the native driver release pipeline does not package DuckDB extension binaries.
• Improve release cluster readiness checks, bound distribution compatibility validation, reduce repeated CI builds, and add PyPI Trusted Publishing and package descriptions (#339, #345–#350).

Installation and limitations

Install the published wheel and Dask dependencies:

python -m pip install 'adbc-driver-redis[dask]==0.0.16'

The Dask helper requires caller-defined disjoint partitions and an explicit Arrow schema. Each partition is materialized in worker memory; reads and retries have no shared snapshot. Install matching packages and the native driver on every worker. See the integration guide.

Redis with Search is required. Writes remain autocommit-only; transaction rollback is unsupported. Platform wheels target Linux x86-64/ARM64 with glibc 2.28+ and macOS ARM64 12+. Native libraries, CLI tarballs, wheels, and SHA-256 files are attached below.

Validation and publication

Source: c737cb8fff88855bc1c12e0d48f3b4dd3c46a364 (merged #357 on main).

Local Dask validation passed 24 Python integration tests, with 2 explicitly optional tests skipped (DuckDB extension download and isolated Superset environment). The Distributed example passed with partition sizes [1, 2, 2, 0, 1]. Minimum-version validation passed all 8 Dask tests using Dask/Distributed 2024.5.0 and PyArrow 14.0.2. See the verification record.

The release workflow passed: Linux amd64/arm64 builds and Redis Software/OSS cluster tests, macOS arm64 build/load/unit checks, all ten packaged Linux distribution checks, and final asset upload succeeded.

PyPI Trusted Publishing passed. Version 0.0.16 exposes the Dask extra, and all three PyPI wheel hashes match the validated release artifacts. A clean installation of the macOS release wheel also passed the two-process Dask query example against disposable Redis using its bundled native library.

Full changelog: v0.0.15...v0.0.16

v0.0.15

Highlights

• Add platform Python wheels with adbc_driver_redis DB-API helpers, the adbc_driver_rediss TLS alias, Polars URI loading, option enums, and Redis/rediss manifest installers (#338).
• Add SQLAlchemy 1.4/2.x dialects and Superset engine specs with catalog reflection and PostgreSQL time grains. Add a Dagster IO manager for Arrow, pandas, polars, staged table replacement, and partition retries (#338).
• Document and test DuckDB's current ADBC scanner integration, including attached-table autocommit writes, bulk ingestion, schemas, read-only attachments, and rejection of explicit transactional writes (#338).
• Expand PostgreSQL SQL, session, metadata, catalog, and error compatibility, including byte-order COLLATE support and application HASH discovery (#231, #234, #332, #333).
• Add bounded multicore scans, shared worker budgets and measured worker tuning; improve exact aggregate spill merging and ordered top-k execution (#223, #225, #233, #304, #326, #328–#330).
• Improve Redis Software setup/recovery checks, benchmark validation/cleanup, and client integration documentation.

Upgrade notes and limitations

• Unquoted SQL identifiers fold to ASCII lowercase; quote legacy mixed-case object names. Small integer literals infer INTEGER before BIGINT/NUMERIC. Default descending sorts place NULLs first.
• Text read as TIMESTAMP without time zone now keeps the written wall-clock time and ignores a validated offset or zone, as PostgreSQL does. Previously stored timestamps are not rewritten. Review stored view/default/check expressions when upgrading from v0.0.14.
• Strings use PostgreSQL C/POSIX byte ordering; locale-aware collations remain unsupported.
• Writes remain autocommit-only. SQLAlchemy rollback cannot undo writes. Dagster table replacement has a DROP/RENAME window, and partition DELETE/append can partially apply. Real multi-statement transactions are not implemented.
• Python wheels are distributed here; PyPI publication and driver-registry registration are not configured. macOS release wheels target macOS 12+ with Go 1.26. Native manifest paths are absolute; rerun installation if the library moves.
• Redis Search is required. Redis Flex and unsupported Redis-compatible products remain outside the supported scope.

Validation and downloads

Release source: 11faf538d0499301c1cb79c7a9ab3e195221a9ea on main (merged #338).

Local #338 validation on macOS arm64 with a disposable Redis 8.6.2 instance passed 17 Python integration tests, 3 separate Superset/SQLAlchemy 1.4 checks, 4 clean installed-wheel tests, the Go unit suite, and focused native collation/catalog regressions. The main Python run skipped Superset because it was tested separately. Browser-rendered charts, a TLS handshake, and Columnar's separate DuckDB extension were not exercised.

PR CI was still running at publication. Publishing starts the release workflow, which builds Linux amd64/arm64 and macOS arm64 native tarballs, CLI tarballs, Python wheels, and SHA-256 files. Downloads appear after its build/test/distribution validation jobs succeed; publication itself does not mean those checks passed.

See the verification record and release workflow for evidence and build status.

What's Changed

• docs: clarify unsupported Redis-compatible products by @alberttwong in #222
• Use multiple cores for individual aggregate queries by @alberttwong in #223
• Share an adaptive worker budget across aggregate queries by @alberttwong in #225
• Refresh README performance measurements on the current driver by @alberttwong in #228
• Default to Redis Software and correct QPF benchmarks by @alberttwong in #229
• Replace README latency table with measured driver worker scaling by @alberttwong in #230
• Read text as a number, boolean, date or time as Postgres does, with its errors by @alberttwong in #231
• Add measured aggregate worker autotuning with bounded exploration by @alberttwong in #233
• Ignore an offset or zone in text read as a TIMESTAMP, as Postgres does by @alberttwong in #234
• Create the Redis Software cluster once its API answers by @alberttwong in #235
• Fix 15 PostgreSQL compatibility, ADBC, and pagination bug reports by @alberttwong in #281
• Add starter Postgres differential harness (#236) by @alberttwong in #282
• Test four Redis Search environments in parallel by @alberttwong in #284
• Expand differential corpus and fixture coverage (#283) by @alberttwong in #287
• Fix concurrent ALTER failures on sharded Redis Software by @alberttwong in #288
• Stabilize worker tuning and document stopped performance validation by @alberttwong in #289
• Preserve legacy mixed-case column resolution by @alberttwong in #290
• docs: list data tool integrations and testing status by @alberttwong in #291
• Isolate ClickBench trials with hard memory limits by @alberttwong in #292
• Bound exact ClickBench aggregates and recover from Redis group limits by @alberttwong in #304
• Fix ClickBench follow-up query plans, FinOps preflight, and typed references by @alberttwong in #314
• Reduce Q19 count-group overhead and preserve timeout stage evidence by @alberttwong in #315
• Fix ClickBench cleanup order for runner security groups by @alberttwong in #324
• Validate ClickBench LIMIT ties and bound cleanup polling by @alberttwong in #325
• Improve ClickBench exact execution and retain live diagnostics by @alberttwong in #326
• Parallelize exact top-K fallback for ClickBench Q26 by @alberttwong in #328
• fix: parallelize residual ordered top-k by @alberttwong in #329
• perf: avoid redundant aggregate spill merge passes by @alberttwong in #330
• fix: verify terminated runners without VPC metadata by @alberttwong in #331
• Add PostgreSQL session, metadata, and query compatibility by @alberttwong in #332
• Complete adopted text encoding, foreign indexes, and PostgreSQL catalogs by @alberttwong in #333
• docs: compare ADBC and Redis connectivity across 20 data tools by @alberttwong in #334
• docs: remove superseded README integration table by @alberttwong in #335
• docs: show Redis connection solution paths with icons by @alberttwong in #336
• docs: add project-grounded AI-native SDLC records by @alberttwong in #337
• Add Python packaging and analytics client integrations by @alberttwong in #338

Full Changelog: v0.0.14...v0.0.15

v0.0.14

Highlights

• Faster bulk ingestion through bounded batches and reusable pipeline commands; adds bounded ClickBench loading tools and benchmark reports (#221).
• Lower scan and aggregation overhead: narrow field fetches, incremental scan-page aggregation, exact COUNT(DISTINCT) tuple deduplication with collision checks, deferred SELECT projection after row selection, and shared typed integer aggregate inputs (#216–#220).
• RESP3 is now the default, with support for both RESP2 and RESP3 reply formats (#207).
• SQL compatibility fixes for result column names, repeated names in derived tables, text-to-integer conversion, and quoted literals in arithmetic operators (#198, #200, #203).
• Fixes row loss when ALTER COLUMN TYPE cleanup reindexes a cursor, and preserves table-not-found errors during rewrite verification (#207).
• Adds measured QPF benchmark results and Redis Flex compatibility findings. Redis Flex remains unsupported (#208, #210).

Upgrading from v0.0.13

The connection protocol now defaults to RESP3. Set protocol=2 in the Redis URI if you need RESP2 compatibility. Integer conversion errors and result column names now follow PostgreSQL more closely; applications depending on earlier behavior should review these changes.

Validation and assets

Release target: 8719760eff2844f49b5f242bf3c931075be1749e on main.

The target has the same source tree as #221's tested commit. Its Linux amd64 and arm64 CI jobs passed Go tests and the ADBC validation suite against Redis standalone and a three-shard cluster, along with the ClickBench helper tests.

The release workflow builds driver and redis-adbc tarballs for Linux amd64, Linux arm64, and macOS arm64, with SHA-256 files. It validates Linux artifacts on Ubuntu 20.04/22.04, AlmaLinux 8/9, and Amazon Linux 2023 before uploading them. macOS receives unit tests and shared-library load checks; Redis integration tests run on Linux.

What's Changed

• docs: refresh README validation results and Go requirement by @alberttwong in #205
• Name result columns as Postgres does, and let derived tables repeat names by @alberttwong in #198
• Read text as an integer as Postgres does, with its errors by @alberttwong in #200
• Type quoted literals in arithmetic operators as Postgres resolves them by @alberttwong in #203
• docs: document local Flex testing and Cloud Search provisioning blocker by @alberttwong in #208
• Default the Redis driver to RESP3 with RESP2 compatibility by @alberttwong in #207
• Document measured QPF search and aggregation performance by @alberttwong in #210
• perf: avoid wide HASH loads for narrow exact scans by @alberttwong in #216
• Consume scan pages incrementally during driver aggregation by @alberttwong in #217
• Add exact COUNT DISTINCT tuple deduplication with collision checks by @alberttwong in #218
• perf: late-materialize ordered residual SELECT results by @alberttwong in #219
• perf: reuse typed inputs across integer aggregate expressions by @alberttwong in #220
• Improve bulk ingestion throughput and add bounded ClickBench loading by @alberttwong in #221

Full Changelog: v0.0.13...v0.0.14

v0.0.13

Highlights

Adopt existing HASHes without changing them (#161, #171)

• redis-adbc adopt now identifies rows by their Redis keys. It no longer writes __rowid into application HASHes, and later application writes appear in the table automatically.
• When an application index has compatible attributes, an adopted table can read through that index instead of creating a second one. Use -index to select one or -own-index to force a driver-owned index.
• Upgrade action: tables adopted by redis-adbc v0.0.11 or v0.0.12 use the old __rowid layout. v0.0.13 refuses to read them. Drop the driver table (the application HASHes stay), then run redis-adbc adopt again.

Schema changes work much more like PostgreSQL (#168, #186, #191)

• PRIMARY KEY is now retained in table metadata, makes its columns NOT NULL, is exposed through GetObjects and information_schema, and can be added or dropped with ALTER TABLE.
• ALTER COLUMN ... SET/DROP NOT NULL validates existing rows and follows PostgreSQL's action ordering in multi-action ALTER TABLE statements.
• ALTER COLUMN ... TYPE ... [USING ...] converts existing values and rewrites rows safely, including recovery when a connection closes mid-rewrite.

PostgreSQL-compatible values, expressions, and errors

• Errors now carry PostgreSQL SQLSTATEs and map to more precise ADBC statuses, including InvalidData, Integrity, NotFound, AlreadyExists, NotImplemented, Cancelled, and Timeout (#181, #192, #201).
• Row and record values work in SELECT, comparisons, ORDER BY, nested rows, and multi-column subqueries (#193). Quoted string literals are typed from their context, as PostgreSQL does (#195).
• Name and scope resolution now matches PostgreSQL more closely across joins, aliases, GROUP BY, ORDER BY, CTEs, derived tables, and DML (#166, #170, #175, #183, #194, #197, #199).
• Interval support now includes PostgreSQL field parsing and qualifiers, microsecond rounding, multiplication/division, aggregates, grouping, and formatting (#174, #182, #189, #190, #196, #202).
• Time handling adds TIME WITH TIME ZONE, POSIX daylight-saving rules, PostgreSQL's default abbreviations, broader TO_DATE/TO_TIMESTAMP patterns, and creation-time zones for literals stored in views, defaults, and checks (#163, #165, #177, #185).

Correctness, performance, and operations

• New redis-adbc sweep finds rows left beneath released driver-owned key prefixes and deletes them only with -apply; its dry run explains what it will leave alone (#173).
• Cluster clients now honor URI pool, retry, connection, and cluster-specific parameters (#162).
• Large row-value IN lists use a hash set, keyset pagination can push bounds into the index, and index grouping/sorting is used only when Redis's numeric representation is exact (#164, #167, #180, #184).
• Re-keying renames, concurrent drops, adopted-table reads, and cancellation during CREATE TABLE AS gained additional race and recovery coverage (#169, #171, #172, #176).

Upgrading from v0.0.12

• Re-adopt tables created by the v0.0.11/v0.0.12 redis-adbc adopt command as described above.
• Primary keys declared by older versions were accepted but not stored or enforced. Add them explicitly with ALTER TABLE ... ADD PRIMARY KEY (...). Upgrade every client that writes table metadata together; older clients do not preserve the new primary_key field.
• SQLSTATEs and ADBC statuses are now more specific. Code that matched the old InvalidArgument/ProgrammingError classes for conversion, missing-object, or duplicate-object errors may need updating.
• Multi-action ALTER TABLE statements run actions in PostgreSQL's order instead of written order, which can change the result of statements that add and then alter or drop the same column.
• Cluster URI parameters such as pool_size, max_retries, and cluster seed/redirect settings now take effect instead of being ignored or rejected.

Tested with

Before the tag, main at 8df997d passed:

• all 509 top-level Go tests (504 driver, 5 redis-adbc) against Redis 8.6.2 standalone and a 3-shard cluster;
• the ADBC validation suite against both setups: 360 passed, 8 skipped, 3 subtests passed;
• the opt-in TestTimeoutServerPaused case; and
• a v0.0.13-stamped macOS arm64 shared-library build.

The release workflow builds and tests Linux amd64, Linux arm64, and macOS arm64 artifacts. It also validates every Linux tarball on Ubuntu 20.04/22.04, AlmaLinux 8/9, and Amazon Linux 2023 before attaching assets.

This release was not tested against Redis Cloud or Redis Flex.

Assets

The release workflow attaches driver and redis-adbc tarballs, plus SHA-256 files, for:

• macOS Apple Silicon;
• Linux x86-64; and
• Linux arm64.

What's Changed

• Say in the README that adopt makes existing HASHes a table in place by @alberttwong in #154
• Apply the URI's pool_size, max_retries and other parameters on a cluster too by @alberttwong in #162
• Read POSIX time zones with daylight saving time rules, and all of Postgres's default abbreviations by @alberttwong in #163
• Use a hash set for long row IN lists, and the index for keyset pagination by @alberttwong in #164
• Read every TO_CHAR pattern in TO_DATE and TO_TIMESTAMP, and negative MAKE_DATE years as BC by @alberttwong in #165
• Identify an adopted table's rows by their keys, not by __rowid by @alberttwong in #161
• Say to add LATERAL when a derived table reads an earlier FROM item by @alberttwong in #166
• Format an interval with TO_CHAR, and a time as an interval, as Postgres does by @alberttwong in #174
• Roll back a re-keying rename and release its lease when its connection is closed by @alberttwong in #169
• Delete the rows left under dropped tables' key prefixes with redis-adbc sweep by @alberttwong in #173
• Read an adopted table's rows through the application's own index by @alberttwong in #171
• Cancel TestSQLCancel's CREATE TABLE AS once it has written rows by @alberttwong in #176
• Don't let a FROM item's ON clause read the target of UPDATE or DELETE by @alberttwong in #175
• Let derived tables and CTEs read the queries around them by @alberttwong in #170
• Read local-time literals in views, DEFAULTs and CHECKs in the zone they were created in by @alberttwong in #177
• Sort rows that tie on the index's doubles in ORDER BY by @alberttwong in #167
• Read a tie by value only when the index printed its double exactly by @alberttwong in #180
• Set and drop NOT NULL with ALTER TABLE, and run its actions in Postgres's order by @alberttwong in #168
• Add a TIME WITH TIME ZONE type, and convert a time AT TIME ZONE, as Postgres does by @alberttwong in #185
• Drop a table that a plain RENAME TO renames while the DROP runs by @alberttwong in #172
• Group in the index only on values it replies exactly (aggregate_pushdown all) by @alberttwong in #184
• Keep PRIMARY KEY: make its columns NOT NULL, and list it in GetObjects and information_schema by @alberttwong in #186
• Sum and average intervals as intervals, as Postgres does by @alberttwong in #182
• Let a join's ON clause read only its two sides, and split FROM lists at commas by @alberttwong in #183
• Put values that compare equal in one group, as Postgres does by @alberttwong in #189
• Report values that don't convert as InvalidData, with Postgres's SQLSTATE on every error it has one for by @alberttwong in #181
• Multiply and divide intervals as Postgres does by @alberttwong in #190
• Fix the build: report SUM/AVG interval overflow with SQLSTATE 22008 by @alberttwong in #192
• Return rows as values: SELECT (a, b), ROW(…), ORDER BY (a, b) and nested rows by @alberttwong in #193
• Sort a bare ORDER BY name by the output column of that name, as Postgres does by @alberttwong in #194
• Read interval literals as Postgres does: exact fields, rounded to microseconds by @alberttwong in #196
• Group by the output column of a bare GROUP BY name no input column has, as Postgres does by @alberttwong in #197
• Change a column's type with ALTER COLUMN … TYPE, rewriting the rows as Postgres does by @alberttwong in #191
• Check the SQLSTATE of interval * and / errors by @alberttwong in #201
• ...

v0.0.12

Highlights

Cancelling a statement works (#149, #152; fixes #142, #150)

• Before, AdbcStatementCancel (cursor.adbc_cancel() in Python) returned OK, but a statement noticed the cancel only at its next Redis command:
  • Work the driver does in memory ran to the end: filtering, joins, aggregates, window functions, sorts, generate_series. A query with no Redis call after that work returned its full result. A join failed only once it was done.
  • A streamed result couldn't be cancelled: the cancel failed with INVALID_STATE ("no active query to cancel"), and the stream went on until it was released.
  • Every cancel was reported as IO (IO: [redis] failed to write rows: context canceled), except a rename_rekey rename's.
• Now a cancelled statement fails with CANCELLED:
  • Work in memory checks for the cancel at every row, so it stops within milliseconds. The issue's 9.3M-pair join, cancelled 1 s in, now fails within a second of the cancel; before, it ran 17.5 s and failed with IO. A sort under way still finishes first.
  • A streamed result: cancelling the statement deletes its FT.AGGREGATE cursor, and reading the stream then fails with CANCELLED. A read under way first waits for the cursor page it asked for.
  • Redis commands: a command already sent still waits for its reply. The error then names the step: CANCELLED: [redis] failed to write rows: context canceled.
  • A Go context past its deadline fails the statement with TIMEOUT.
• Writes: INSERT, UPDATE, DELETE and MERGE compute every change before they write one, so a statement cancelled before then changes nothing. There are no transactions, so one cancelled while it writes keeps the rows it has written, as does CREATE TABLE … AS. A rename_rekey rename is rolled back, as before.
• Speed when not cancelled is unchanged: each check is one atomic load per row (Go 1.26's cancelCtx.Err).
• Docs: the README has a new "Cancelling a statement" section, after "Timeouts".

Bulk ingest converts in the session time zone (#153, fixes #151)

• Before, with adbc.redis.time_zone set to anything but UTC, bulk ingest stored text without an offset, and Arrow timestamps without a zone, as UTC in a TIMESTAMP WITH TIME ZONE column. INSERT reads them as local times, as Postgres does. dbt seeds load with bulk ingest, so in New York a seed's 2024-01-10 02:00:00 landed 5 hours off.
• Now bulk ingest converts like INSERT … VALUES ($1, …) with the same batch bound:
  • Into TIMESTAMP WITH TIME ZONE: an Arrow string without an offset, an Arrow timestamp without a zone, and an Arrow date are local times. A skipped local time takes the offset before the transition, and a repeated one the offset after it.
  • A zoned Arrow timestamp written to a TIMESTAMP, DATE, TIME or text column stores its local time.
• Unchanged: text with an offset and zoned Arrow timestamps keep their instant. UTC sessions, the default, store the same values as before.

redis-arrow is gone (#147, #148)

• The redis-arrow command (Arrow IPC export and import) is removed. Its uses are covered elsewhere:
  • Query results as Arrow: the Redis Arrow Flight server, or the driver from Python with pyarrow.
  • Loading Arrow data: ADBC bulk ingest (adbc_ingest).
  • HASHes already in Redis: redis-adbc copy.
• Releases now attach the driver and redis-adbc tarballs only. The redis-arrow tarballs in v0.0.10 and v0.0.11 stay.

Upgrading from v0.0.11

• No change to existing tables' data or metadata. v0.0.11 and v0.0.12 clients can share a database.
• Behaviour changes:
  • Cancelling. A cancelled statement fails with CANCELLED where it failed with IO, or ran to the end. Code that matched context canceled in IO errors should check for the CANCELLED status instead.
  • Bulk ingest in a non-UTC session stores values without an offset or zone as local times (above). Rows ingested earlier aren't rewritten.
  • redis-arrow is no longer released.
• Known gaps:
  • A sort under way finishes before the cancel takes effect.
  • A Redis command already sent waits for its reply, up to the read timeout.
  • GetParameterSchema doesn't describe MERGE: its parameters are the null type.

Tested with

Before the tag, main at b44cec5 passed the full ADBC validation suite (330 passed, 10 skipped) and all 366 Go tests (362 for the driver, 4 for redis-adbc):

• TestFlexRefused was skipped, since it needs a Flex database.
• TestTimeoutServerPaused, which pauses the server, ran separately with REDIS_PAUSE_TESTS=1 and passed.
• The Go tests also passed in an amd64 build (GOARCH=amd64, under Rosetta).

Setup	Validation suite	Go tests
Redis 8.6.2, standalone	✓	✓
Redis 8.6.2, 3-shard cluster	✓	✓

The same commit also passed a full run of the release workflow before the tag. That run built and tested on linux-amd64, linux-arm64 and macOS. It then ran the validation suite, and redis-adbc check, copy and adopt, with each Linux tarball on all ten distribution/architecture pairs. The release workflow then tests the built binaries again; see Assets. This release wasn't run against Redis Cloud.

Assets

Each adbc_driver_redis tarball contains the shared library, its C header and the README. Each redis-adbc tarball contains the binary and the README. Each .sha256 file has its tarball's checksum.

Asset	Platform
adbc_driver_redis-v0.0.12-darwin-arm64.tar.gz	macOS, Apple Silicon
adbc_driver_redis-v0.0.12-linux-amd64.tar.gz	Linux x86-64: RHEL 8+, Ubuntu 20.04+, Amazon Linux 2023
adbc_driver_redis-v0.0.12-linux-arm64.tar.gz	Linux arm64: same distributions
redis-adbc-v0.0.12-darwin-arm64.tar.gz	macOS, Apple Silicon
redis-adbc-v0.0.12-linux-amd64.tar.gz	Linux x86-64, statically linked
redis-adbc-v0.0.12-linux-arm64.tar.gz	Linux arm64, statically linked

macOS quarantines a binary from a tarball downloaded with a browser and refuses to run it. Download with gh release download or curl instead, or run xattr -d com.apple.quarantine <binary>.

For other platforms:

cd go && make build

cd go && make adbc-cli

v0.0.11

Highlights

New: redis-adbc, for HASHes the driver didn't write (#138, #140, #143, #144)

• The driver only sees tables it created. redis-adbc makes HASHes that an application wrote (user:1, user:2, …, maybe with the application's own search index) queryable through the driver. It links the driver as a Go library; make adbc-cli builds it from go.

  • check guesses the table's structure from a sample of the HASHes:

    • types: BOOLEAN, BIGINT, NUMERIC(38, s), DOUBLE PRECISION, DATE, TIMESTAMP [WITH TIME ZONE], VARBINARY or VARCHAR;
    • integers with leading zeros, such as ZIP codes, stay text.

    It then prints a checklist for each way of making the HASHes a table. Each item is ok, warn, missing (the command provides it) or blocker (change something first), with a fix. The checklist ends with the command to run. Without -prefix, check lists the collections it finds. It writes nothing.

  • copy reads the HASHes and bulk-ingests them into a new driver table, through the driver. The originals stay as they are, and the table is a snapshot (-mode replace refreshes it). If a copy fails in create mode, it drops the table it made.

  • adopt makes the HASHes themselves a driver table, so SQL reads what the application writes. Without -apply it only says what it would do. With it, it:

    • creates a second index, in the driver's format, beside the application's;
    • writes __rowid (the key's suffix) into each HASH;
    • writes the table's metadata.

    If the index can't index a HASH, adopt undoes everything. Running it again finishes an adopt that was killed half-way. -refresh adds HASHes the application wrote since.

  redis-adbc check -prefix user:
  redis-adbc copy -prefix user: -table users
  redis-adbc adopt -prefix user: -table users_live -apply

• What an adopted table can't do. Its keys are the application's, so the driver refuses INSERT (and MERGE … INSERT and bulk ingest), TRUNCATE, ADD COLUMN and a re-keying RENAME.

  • DROP TABLE drops only the adopted index (without DD) and the metadata. The HASHes stay.
  • DROP COLUMN keeps the field.
  • No new table may have its rows under an adopted prefix.
  • UPDATE and DELETE work, and change the application's HASHes.
  • In place, a column is only numeric if every stored value is such a number. The driver stores booleans as 0/1 and dates as integers, so true/false text and ISO dates are VARCHAR.

• Prebuilt for the same platforms as the driver: redis-adbc-v0.0.11-linux-amd64, -linux-arm64 and -darwin-arm64. The Linux binaries are statically linked. On every distribution the driver is tested on, the release workflow runs check, copy and adopt with each tarball.

• ACL rules for each command are in the README, and were checked against users that have exactly those rules.

• redis-arrow is unchanged: export and import.

Query results stream (#139)

• Before, ExecuteQuery built a query's whole result in memory before it returned. So SELECT * FROM big_table held the table twice, in the cursor pages and in the Arrow batches.
• Now a query that reads one table through its index streams. The rows come from the FT.AGGREGATE cursor a page at a time as the reader is read, so the driver holds about one batch, whatever the table's size.
  • That is a single SELECT with no aggregates, GROUP BY, DISTINCT, set operations, window functions or correlated subqueries.
  • Its rows come in insertion order, or in an ORDER BY the index sorts exactly.
  • Other queries are built whole, as before.
• Row count and errors:
  • A result that fits in the first batch (65,536 rows by default) comes back whole, with its row count. A longer one is a stream with row count -1.
  • An error after the first batch ends the stream: Next returns false, and Err reports it.
• redis-arrow export of a streamed query writes batch by batch.
• Options: adbc.redis.stream_results=false (database or statement) turns streaming off. adbc.redis.stream_batch_rows sets the rows per batch.
• Redis's cursor limits now apply while a stream is open: search-cursor-max-idle, 300 s by default, and search-index-cursor-limit, 128 open cursors per index.

GetParameterSchema (#136, fixes #133)

• It returned NotImplemented. The Arrow Flight SQL JDBC driver (and so DBeaver, through the Redis Arrow Flight server) needs it for every parameterized PreparedStatement.

• Now it returns one nullable field per parameter, in order, with the type the parameter is used as. The type comes from:

  • comparisons, arithmetic, || and LIKE, casts, COALESCE and CASE;
  • for INSERT values and UPDATE … SET, the target column.

  A parameter whose type can't be told is the null type. The validation suite's test_parameter_schema now passes (325 passed).

CREATE TABLE no longer drops an application's index (#137, #138)

• The bug: before it creates a table's index (CREATE TABLE, bulk ingest, TRUNCATE, a re-keying rename), the driver drops an index that already has the new name with FT.DROPINDEX … DD, taking it for a leftover of an interrupted CREATE. It only checked its own registry. So an application's index that happened to be named like a new table's, such as idx:public:users, was dropped, and DD deleted every HASH it covered.
• Now the driver checks with FT.INFO first. A leftover is a HASH index on just the new prefix, with no filter and no documents.
  • Any other index of that name makes the table take the next names (public:users~2:, idx:public:users~2). So does an index on keys the ACL user can't read, which FT.INFO refuses with NOPERM.
  • The application's index and HASHes are left alone.

Repository

• The repository is alberttwong/redis-adbc-driver again, and the README's title is "ADBC driver for Redis". The README and code link that name (#145, #146); the -arrow-ipc URLs redirect.

Upgrading from v0.0.10

• No change to existing tables' data or metadata. v0.0.10 and v0.0.11 clients can share a database, with one exception: adopted tables.

• Don't let v0.0.10 clients near an adopted table. v0.0.10 doesn't know the adopted flag in a table's metadata, and ignores it:

  • Its DROP TABLE or TRUNCATE of an adopted table runs FT.DROPINDEX … DD, which deletes the application's HASHes.
  • Its INSERT writes rows among the application's keys.

  Upgrade every client of a database before you adopt HASHes in it. (Adopting adds adbc:{meta}:adopted and adbc:{meta}:adopting to the driver's metadata.)

• Behaviour changes:

  • Streaming. A result longer than the first batch has row count -1, and an error after the first batch comes from the reader's Err, not from ExecuteQuery. A stream holds an FT.AGGREGATE cursor until it is read to the end, released or its connection closed, so Redis's cursor limits (above) apply. adbc.redis.stream_results=false gives v0.0.10's behaviour.
  • GetParameterSchema returns a schema instead of NotImplemented.
  • A new table takes the ~N names when an index with its name isn't the driver's (before, that index was dropped with its HASHes).

• Known gaps:

  • AdbcStatementCancel doesn't reach a stream; Go callers cancel the context instead.
  • GetParameterSchema doesn't describe MERGE: its parameters are the null type.

Tested with

Before the tag, main at a779de7 passed the full ADBC validation suite (325 passed, 10 skipped) and all 367 Go tests (358 for the driver, 4 for redis-adbc, 5 for redis-arrow):

• TestFlexRefused was skipped, since it needs a Flex database.
• TestTimeoutServerPaused, which pauses the server, ran separately with REDIS_PAUSE_TESTS=1 and passed.
• The Go tests also passed in an amd64 build (GOARCH=amd64, under Rosetta).

Setup	Validation suite	Go tests
Redis 8.6.2, standalone	✓	✓
Redis 8.6.2, 3-shard cluster	✓	✓

The same commit also passed a full run of the release workflow before the tag. That run built and tested on linux-amd64, linux-arm64 and macOS, then ran the validation suite, a redis-arrow round trip, and redis-adbc check, copy and adopt with each Linux tarball on all ten distribution/architecture pairs. The release workflow then tests the built binaries again; see Assets. This release wasn't run against Redis Cloud.

Assets

Each adbc_driver_redis tarball contains the shared library, its C header and the README. Each redis-arrow and redis-adbc tarball contains the binary and the README. Each .sha256 file has its tarball's checksum.

Asset	Platform
adbc_driver_redis-v0.0.11-darwin-arm64.tar.gz	macOS, Apple Silicon
adbc_driver_redis-v0.0.11-linux-amd64.tar.gz	Linux x86-64: RHEL 8+, Ubuntu 20.04+, Amazon Linux 2023
adbc_driver_redis-v0.0.11-linux-arm64.tar.gz	Linux arm64: same distributions
redis-arrow-v0.0.11-darwin-arm64.tar.gz	macOS, Apple Silicon
redis-arrow-v0.0.11-linux-amd64.tar.gz	Linux x86-64, statically linked
redis-arrow-v0.0.11-linux-arm64.tar.gz	Linux arm64, statically linked
redis-adbc-v0.0.11-darwin-arm64.tar.gz	macOS, Apple Silicon
redis-adbc-v0.0.11-linux-amd64.tar.gz	Linux x86-64, statically linked
redis-adbc-v0.0.11-linux-arm64.tar.gz	Linux arm64, statically linked

macOS quarantines a binary from a tarball downloaded with a browser and refuses to run it. Download with gh release download or curl instead, or run xattr -d com.apple.quarantine <binary>.

For other platforms:

cd go && make build

cd go && make cli adbc-cli

v0.0.10

Highlights

New: redis-arrow, Arrow IPC export and import (#124, #127)

• A command that moves data between Redis and Arrow IPC through the driver. It links the driver as a Go library, so it needs neither cgo nor the shared library. make cli builds it from go.

  • export runs SQL and writes the result as an IPC file (.arrow, also read as Feather v2) or a stream (.arrows, or stdout). -format file|stream overrides the choice, and -compression lz4|zstd compresses the batches.
  • import bulk-ingests an IPC file or stream, from a path or stdin, telling the two apart by the file format's magic bytes. It takes -mode create|append|replace|create_append (as in ADBC bulk ingest), -schema and -index-columns.

  redis-arrow export -o sales.arrow "SELECT * FROM sales WHERE status = 'shipped'"
  redis-arrow export "SELECT * FROM sales" | redis-arrow import -table sales_copy -mode replace

• Prebuilt for the first time in this release, for the same platforms as the driver: redis-arrow-v0.0.10-linux-amd64, -linux-arm64 and -darwin-arm64. The Linux binaries are statically linked. The release workflow runs an export and import round trip with each tarball on every distribution the driver is tested on.

• Connecting: -uri, else $REDIS_URI, else redis://localhost:6379/0. Put a password in $REDIS_URI, not -uri, where other users of the machine can see it in the process list. -option key=value sets a database option.

• Behaviour to know:

  • A failed export leaves no partial file.
  • A statement without a result set (DDL, or DML without RETURNING) still runs, but export then fails and writes nothing.
  • The driver builds a query's whole result before it returns the first batch, so exporting a large table needs memory for all of it.
  • A file on stdin is read into memory first, since an IPC file's footer is at its end. Streams are read batch by batch.

• macOS quarantines a binary from a tarball downloaded with a browser and refuses to run it. Download with gh release download or curl instead, or run xattr -d com.apple.quarantine redis-arrow.

Outer-level aggregates are computed in the outer query (#130, fixes #120)

• The bug: an aggregate of only the outer query's columns inside a subquery, such as max(g.id) in (select max(g.id) from g x where x.id = 1), was computed over the subquery's rows. v0.0.9 rejected the cases it knew were wrong as not supported.
• Postgres's rule, now the driver's: an aggregate belongs to the innermost query whose columns its arguments, ORDER BY and FILTER read. For an ordered-set aggregate, those are its WITHIN GROUP value and FILTER.
  • Such an aggregate is computed over each of the outer query's groups. In the subquery it is a constant, also in its WHERE.
  • It makes a query without GROUP BY an aggregate query, which returns one row.
  • sum(g.v + x.id) also reads the subquery's columns, so it stays the subquery's.
• Results are now Postgres 16's. With g (id, v) holding (1, 10), (2, 10), (3, 20):

Query	v0.0.9	v0.0.10 (= Postgres)
select v, (select sum(g.v) from g x where x.id = 1) from g group by v	10|10, 20|10	10|20, 20|20
select (select max(g.id) from g x where x.id = 1) from g	1, 1, 1	3
select v, (select max(g.id) from g x where x.id = 1) from g group by v	not supported	10|2, 20|3
select count(*), (select max(g.id) from g x where x.id = 1) from g	not supported	3|3
select v from g group by v having (select count(g.id) from g x where x.id = 1) > 1	not supported	10
select (select sum(g.v)) from g	aggregates require a FROM clause	40

• Pushdown: the outer aggregates run in the index when the outer query's other aggregates do. For example, the outer SUM(g.v) of the first query is an FT.AGGREGATE … GROUPBY.
• Each aggregate's placement is checked at its own level, as in Postgres. select id from g where (select max(g.id)) > 1 is aggregate functions are not allowed in WHERE. The same goes for GROUP BY, JOIN conditions, FILTER, LATERAL items, functions in FROM, UPDATE, VALUES, RETURNING, MERGE WHEN conditions and recursive terms. max((select max(g.id))) is aggregate function calls cannot be nested.

A scalar subquery with more than one row gets Postgres's error (#131)

• select (select x.id from g x) from g now fails with more than one row returned by a subquery used as an expression, as in Postgres 16. Before, the message was the driver's own: scalar subquery returned N rows. The status is still InvalidArgument.
• This covers every place a scalar subquery can be: the select list, WHERE, HAVING, ORDER BY, function arguments, UPDATE … SET and INSERT … VALUES. A failed UPDATE or INSERT leaves the table unchanged.

NaN sums are computed in the index on amd64 servers (#125)

• The bug: on an x86 Redis server, a SUM or AVG of doubles that adds up to NaN (Infinity plus -Infinity) wasn't computed in the index. RediSearch replies -nan there, with the sign bit that x86 gives that NaN, and v0.0.9 didn't read -nan. So the driver computed the aggregate itself: the results were right, but slower. On a cluster, the check for a shard's NaN partial SUM missed -nan the same way.
• Now -nan and +nan, in any case, read as NaN, as in Postgres 16 ('-NaN'::float8 is NaN). This is the linux-amd64 test failure that kept v0.0.9's release run from attaching binaries.

Every pull request is tested like a release (#122, #129, #132)

• New Test workflow: every pull request that changes go/** runs the Go tests and the validation suite against standalone Redis and a 3-shard cluster, on linux-amd64 and linux-arm64. Before, linux-amd64 was first tested in the release run, which is how v0.0.8 and v0.0.9 shipped without binaries.
• The release workflow can skip a test in a run by hand (-f skip=<go test -skip pattern>), for a test that's wrong at that tag, not for a driver bug. v0.0.8 and v0.0.9 got their binaries this way.
• Found while preparing this release, and fixed by #132 before the tag. Both showed up on a 3-shard cluster, and in both the driver was right and the test was wrong:
  • A #130 test expected one particular row from order by <constant> limit 1, which any row satisfies.
  • A rename test raced with redis-arrow's tests. go test ./... runs the two packages at once against the same server. Any connection that opens rolls back an abandoned move, including the one the test had just set up. The go test runs that have a server now pass -p 1, and so does make test. Pass it yourself when running every test against a server.

Upgrading from v0.0.9

• No change to stored data or metadata. v0.0.9 and v0.0.10 clients can share a database.
• Behaviour changes:
  • Outer-level aggregates in subqueries are computed over the outer query's groups. Queries that held one used to return the subquery's values, or fail as not supported. A query without GROUP BY that holds one now returns one row.
  • New error text for aggregates in the wrong place, now Postgres's:
    • aggregates are not allowed in WHERE (and in GROUP BY and JOIN conditions) is now aggregate functions are not allowed in ….
    • … in WHEN conditions is now … in MERGE WHEN conditions.
  • Aggregates in UPDATE … SET, INSERT … VALUES and MERGE's SET and VALUES now fail with Postgres's error when the statement is planned. Before, they failed when it ran, with aggregate MAX is not allowed here.
  • A scalar subquery that returns more than one row has the message more than one row returned by a subquery used as an expression. It used to be scalar subquery returned N rows.
  • On amd64 servers, SUM and AVG that add up to NaN run in the index. Only performance changes; the results are the same.
• Known gaps:
  • An outer-level aggregate with a subquery in its arguments fails: (select max((select g.id)) from x) is outer-level aggregate with a subquery in its arguments is not supported. Postgres gives 10|2, 20|3. A query's own max((select g.id)) works as before.
  • (select count(*)) without FROM is still aggregates require a FROM clause. Postgres gives one row per outer row.

Tested with

Before the tag, main with #132 passed the full ADBC validation suite (324 passed, 10 skipped, 1 expected failure) and all 340 Go tests (335 for the driver, 5 for redis-arrow):

• TestFlexRefused was skipped, since it needs a Flex database.
• TestTimeoutServerPaused, which pauses the server, ran separately with REDIS_PAUSE_TESTS=1 and passed.
• The Go tests also passed in an amd64 build (GOARCH=amd64, under Rosetta).

Setup	Validation suite	Go tests
Redis 8.6.2, standalone	✓	✓
Redis 8.6.2, 3-shard cluster	✓	✓

Before the tag, the same code also passed GitHub Actions: #132's Test run (Go tests and validation suite on standalone and cluster, on linux-amd64 and linux-arm64) and a full run of the release workflow. The release workflow then tests the built binaries again; see Assets. This release wasn't run against Redis Cloud.

Assets

Each adbc_driver_redis tarball contains the shared library, its C header and the README. Each redis-arrow tarball contains the binary and the README. Each .sha256 file has its tarball's checksum.

Asset	Platform
adbc_driver_redis-v0.0.10-darwin-arm64.tar.gz	macOS, Apple Silicon
adbc_driver_redis-v0.0.10-linux-amd64.tar.gz	Linux x86-64: RHEL 8+, Ubuntu 20.04+, Amazon Linux 2023
adbc_driver_redis-v0.0.10-linux-arm64.tar.gz	Linux arm64: same distributions
redis-arrow-v0.0.10-darwin-arm64.tar.gz	macOS, A...

v0.0.9

Highlights

Binaries attached by a later run (#119, #121, #125, #129)

• Neither v0.0.8 nor v0.0.9 got binaries from its release run. v0.0.8's linux-amd64 job failed TestScalarTrigonometry and v0.0.9's failed TestSQLNaNAggregates, so none of the three platforms' tarballs were attached. Both releases' tarballs were attached later by a run of the release workflow by hand that skipped that one test (#129). They are built from each tag's own code.
• v0.0.8's failure was the test, not the driver: it expected arm64's last bits for TAN(PI() / 4) and COT(PI() / 4). Go fuses multiply-adds on arm64 but not on amd64, so tan(π/4) is 0.9999999999999998 on arm64, 1 on amd64 and 0.9999999999999999 in Postgres. v0.0.9's test compares the radian functions' inexact results with Postgres's within 1 ulp (#121). The degree functions are exact everywhere, as before.
• v0.0.9's failure is a slowdown, not wrong results: with an amd64 Redis server, SUM or AVG of doubles that adds up to NaN (Infinity plus -Infinity) isn't computed in the index. RediSearch replies -nan there, which v0.0.9 doesn't read, so the driver computes the aggregate itself. The results are right. Fixed after v0.0.9 by #125.

Schema-qualified column references are checked (#115)

• The bug (#111): the driver ignored the schema in s.t.col, so select n2.t.v from n1.t returned n1.t's v with no error. other.n1.t.v and a.b.c.d.e were accepted too.
• Postgres's rule: s.t.col (or redis.s.t.col) names only a table or view t of schema s that has no alias. pg_temp is the connection's temporary schema. CTEs, derived tables and aliased items are never visible schema-qualified. The same holds for s.t.* and in CHECK. t.col and alias.col are unchanged.
• Two tables t from different schemas can now be FROM items together without aliases. select n1.t.v, n2.t.v from n1.t join n2.t on n1.t.id = n2.t.id returns rows, where it failed with table name "t" specified more than once; use aliases. That works in every kind of join, including USING and NATURAL, in UPDATE … FROM, DELETE … USING and LATERAL, and in subqueries. A MERGE source still can't have the target's name.
• Errors are Postgres 16's:
  • invalid reference to FROM-clause entry for table "t"; there is an entry for table "t", but it cannot be referenced from this part of the query
  • table reference "t" is ambiguous
  • cross-database references are not implemented: …
  • improper qualified name (too many dotted names): …

Subqueries of a grouped query read only grouped columns (#117)

• The bug (#112): select v from g group by v having exists (select 1 from g x where x.id > g.id) returned rows, with the subquery reading one row's id per group. Postgres rejects it.
• Now it fails with Postgres's subquery uses ungrouped column "g.id" from outer query.
  • Where: the select list, HAVING, ORDER BY, DISTINCT ON, QUALIFY and window clauses, for every kind of subquery and at any depth.
  • What counts as grouped: only a GROUP BY item that is a column by itself. With aggregates or HAVING but no GROUP BY, nothing is.
  • Not checked, as in Postgres: aggregate arguments, WHERE, ON, GROUP BY and LATERAL.
• Outer-level aggregates of an ungrouped column gave wrong results. An example is max(g.id) in (select max(g.id) from g x where x.id = 1), in a query grouped by v. They now fail with outer-level aggregate of column "g.id" in a subquery is not supported.
• Column names in errors are t.c, as in Postgres. Grouping-set errors used to show s.s.g, t.t.a, or a NUL byte for some items.

NaNs in float columns are found by every query (#116)

• The bug (#110): RediSearch doesn't index a NUMERIC field holding NaN. So a row with a NaN in an indexed REAL or DOUBLE PRECISION column was written and never seen again: SELECT, COUNT(*), UPDATE and DELETE skipped it, and DROP TABLE and TRUNCATE left its HASH behind. Also, x = 'NaN' failed with Syntax error … near NaN.
• Storage: a NaN is now stored as infinity, which RediSearch indexes as +inf, and it reads back as NaN. Every write path does this, bulk ingest included.
• Results: comparisons, IN lists and joins with NaN or Infinity are re-checked by the driver, so they give Postgres's rows. NaN is above every number, Infinity included, and equals itself.
• Once an indexed float column holds a NaN, its metadata records that (nans), and the flag is never cleared. From then on, the driver itself sorts and groups by the column and, with aggregate_pushdown all, computes its MIN, MAX, SUM and AVG, because the index can't tell a NaN from +Infinity.
• Hash joins match NaN with NaN.
• Text of the infinities is Postgres's Infinity / -Infinity (was +Inf / -Inf) in casts, ||, CONCAT and errors. Arrow values don't change.
• NUMERIC still can't hold NaN or the infinities, but the error is now cannot convert NaN to NUMERIC(10,2): NUMERIC values can't be NaN or infinite.

Times keep their declared precision; CURRENT_TIMESTAMP(p) rounds (#118)

• The bug (#113): a p of 1, 2, 4, 5, 7 or 8 became 3, 6 or 9. cast(timestamp '2024-01-10 10:00:00.123456' as timestamp(2)) was …00.123, and a timestamp(2) column was a TIMESTAMP(3).
• Rounding: values are rounded to p digits, as #106 rounds to 0, 3, 6 or 9. That covers casts and every write, including CREATE TABLE … AS, defaults and bulk ingest.
• The type is reported with p: TIMESTAMP(2) in errors, in information_schema.columns and in GetObjects, which now sets xdbc_decimal_digits for every time and timestamp column. The Arrow unit is the next one up: a TIMESTAMP(2) is a timestamp[ms] of whole hundredths.
• Nothing is rounded twice: mixed precisions (COALESCE, CASE, set operations, …) take the most digits. Computed values (arithmetic, DATE_TRUNC, aggregates, …) have no p.
• Current times: CURRENT_TIMESTAMP(p), LOCALTIMESTAMP(p), CURRENT_TIME(p), LOCALTIME(p) and NOW(p) round to p, and their type and Arrow unit follow it. current_timestamp(0) is whole seconds, a timestamp[s, tz=UTC]. p must be an integer constant from 0 to 9.
• Typed literals with a precision are parsed: TIMESTAMP(2) '…', TIMESTAMPTZ(1) '…', TIME(1) '…'.

Upgrading from v0.0.8

• Upgrade every client that writes to a database together. A v0.0.8 client:
  • still writes a NaN as NaN, so its rows with NaNs stay out of the index;
  • drops a float column's NaN flag whenever it changes a table's metadata (an ALTER TABLE, a COMMENT ON, and some writes), after which v0.0.9 sorts, groups and aggregates that column's NaNs as +Infinity in the index;
  • ignores a time column's p and writes its Arrow unit's digits (3 for a TIMESTAMP(2)).
• Rows with NaNs written by v0.0.8 and earlier are still not in the index, and nothing moves them. The README's "NaNs in the index" section says how to find them (compare COUNT(*) with the keys under the table's prefix, or check FT.INFO's hash_indexing_failures) and how to repair one (UPDATE … WHERE __rowid = N).
• Time columns created by v0.0.8 and earlier keep the type they were given, values included: a TIMESTAMP(2) column stays a TIMESTAMP(3). Only columns created from v0.0.9 on have p.
• Behaviour changes:
  • Schema-qualified columns name only their schema's table. select n2.t.v from n1.t and select other.n1.t.v … used to return rows and now fail. A rel.* with a wrong qualifier now gets Postgres's error text.
  • Grouped queries whose subqueries read ungrouped columns fail, and so do outer-level aggregates of an ungrouped column. So does a subquery repeated by text in GROUP BY (select (select g.id) … group by (select g.id)), which Postgres accepts: write group by 1 or an alias instead.
  • The infinities' text is Infinity / -Infinity, and so are hashes of that text.
  • Float columns that have held a NaN are sorted, grouped and aggregated in the driver, not in the index. Comparisons with 'Infinity' are re-checked by the driver.
  • TIMESTAMP(p) and TIME(p) with a p of 1, 2, 4, 5, 7 or 8 hold p digits and are reported that way. For example, information_schema.columns.data_type is TIMESTAMP(2).
  • CURRENT_TIMESTAMP(p) and the other current times round to p, and their Arrow unit follows it: CURRENT_TIMESTAMP(0) is a timestamp[s, tz=UTC] (was microseconds). An argument that isn't an integer constant from 0 to 9 is an error.
  • New errors: TIMESTAMP(-1), VARCHAR(-1) and TIMESTAMP(3, 1).
  • NUMERIC errors for NaN and the infinities have new text.

Tested with

Before the release, main passed the full ADBC validation suite (324 passed, 10 skipped, 1 expected failure) and all 330 Go tests:

• TestFlexRefused was skipped, since it needs a Flex database.
• TestTimeoutServerPaused, which pauses the server, ran separately with REDIS_PAUSE_TESTS=1 and passed.
• The Go unit tests also passed in an amd64 build (GOARCH=amd64, under Rosetta). That build is how #119 was reproduced.

Setup	Validation suite	Go tests
Redis 8.6.2, standalone	✓	✓
Redis 8.6.2, 3-shard cluster	✓	✓

The release workflow then tests the built libraries; see Assets. This release wasn't run against Redis Cloud.

Assets

The release run failed TestSQLNaNAggregates on linux-amd64 (see the first highlight), so these were attached later, by a run of the release workflow by hand that skipped that one test (#129). They are built from v0.0.9's code, and every other test passed. Each tarball contains the shared library, its C header and the README; each .sha256 file has the tarball's checksum.

Asset	Platform
adbc_driver_redis-v0.0.9-darwin-arm64.tar.gz	macOS, Apple Silicon ...

v0.0.8

Highlights

ALTER TABLE takes several actions; BEGIN, COMMIT, SET and SHOW run (#96)

• Several actions: ALTER TABLE takes a comma-separated list of ADD [COLUMN], DROP [COLUMN], ALTER [COLUMN] … {SET | DROP} DEFAULT, ADD constraint and DROP CONSTRAINT. On v0.0.7, a second action was syntax error: unexpected ",". dbt writes this for on_schema_change and for snapshots whose source gains columns.
• All or nothing, in the order written: if any action fails, nothing changes. Postgres runs the DROPs first, so a few lists differ (see the README).
• Retried ADD COLUMNs work: they used to fail forever with Duplicate field in schema - x.
• Transaction statements do nothing, as in Postgres's autocommit mode: BEGIN, START TRANSACTION, COMMIT, END, ROLLBACK, ABORT and SET TRANSACTION. ROLLBACK undoes nothing. A script with SAVEPOINT, RELEASE or … PREPARED is refused before any of it runs.
• SET, RESET and SHOW, per connection. SET LOCAL lasts until the transaction or the script ends.
  • search_path picks the schema for unqualified names, like adbc.redis.default_schema but for this connection only.
  • TimeZone sets the session time zone (#106).
  • client_encoding, statement_timeout (not enforced), DateStyle and a few others are checked and shown, with no other effect.
• dbt: the adapter can drop its workarounds for ALTER and commit; (alberttwong/redis-dbt-adapter-with-project#31, #32).

Row constructors work in comparisons, IN and ANY / ALL (#98)

• The problem: (a, b) was a syntax error. dbt's delete+insert with a list unique_key writes where (k1, k2) in (select distinct k1, k2 from …), so every composite-key incremental model failed on its second run.
• Syntax: (a, b, …) and ROW(…), with Postgres's semantics, NULLs included:
  • =, <>, <, <=, > and >=; (1, NULL) = (1, 2) is NULL.
  • [NOT] IN over a list of rows or a subquery, ANY / SOME / ALL, a row against a one-row subquery, BETWEEN and simple CASE.
  • IS [NOT] NULL, and IS [NOT] DISTINCT FROM, which is new for single values too.
• Where: WHERE, ON, HAVING, CASE, the select list, ORDER BY, CHECK, views, and the conditions of UPDATE, DELETE and MERGE. Parameters can be items.
• dbt's delete+insert gives Postgres's rows (a target row with a NULL key item is kept), and the adapter can drop its delete … using override (alberttwong/redis-dbt-adapter-with-project#28).
• Performance: dbt's delete of 10,000 keys from a 200,000-row table took about 1 s (0.2–0.3 s for clustered keys), against 1.3–1.6 s for the DELETE … USING workaround.
• Errors are Postgres's (unequal number of entries in row expressions, …). Rows as values and nested rows aren't supported.

An aliased table is visible only by its alias (#108)

• The bug: inside (select … from t x …), t.v named the inner row instead of the outer t. So x.v > t.v compared each row with itself, and correlated subqueries and LATERAL returned wrong rows with no error.
• Postgres's rule: a FROM item with an alias is visible only by its alias, so t.col refers to an enclosing level. Unaliased items are still visible by their table name, and unqualified columns resolve innermost-first, as before. This holds everywhere, UPDATE, DELETE and MERGE included.
• Errors, Postgres's text:
  • select t.id from t x used to return rows, and update t o set … where t.id = 1 updated the row. Both now fail with invalid reference to FROM-clause entry for table "t"; perhaps you meant to reference the table alias "x".
  • A qualifier that names no visible item gives missing FROM-clause entry for table "t", not column "t.col" does not exist.

Sums of doubles are exact, and trigonometric functions are new (#107)

• The bug (#83): SUM, AVG and the variances of DOUBLE PRECISION added the values in floating point, in the order rows arrived. On a cluster that order changes, so the same query gave different last digits from run to run (502249.99999999994 … 502250.0000000010 for an exact 502250), and ROUND could show it.
• Now: the driver keeps the exact sum and rounds once, so results are correctly rounded and the same on every run. That covers GROUP BY, FILTER, DISTINCT, grouping sets and window frames. It costs about 25 ns per value, within the noise in benchmarks.
• Integers and decimals: AVG of them is now the exact mean, rounded once. SUM of integers fails only if the total doesn't fit BIGINT, not when a running total overflows, and a decimal SUM such as SUM(CASE WHEN … THEN 1 ELSE 2.5 END) no longer leaves out the integer values.
• NaN from pushed-down sums on a cluster (#101): SUM and AVG of a nullable column gave nan for a group whose rows on one shard all lacked the value: DOUBLE columns with adbc.redis.aggregate_pushdown=all, and AVG of an INTEGER column by default. Such a query now runs again so that every shard sends a number.
• New functions (#89): SIN, COS, TAN, COT, ASIN, ACOS, ATAN, ATAN2, the degree forms SIND … ATAN2D, RADIANS, DEGREES, PI() and CBRT, with Postgres's rules. The degree forms are exact at 0, 30, 45, 60 and 90 degrees; the radian forms can differ from Postgres in the last bit, as Postgres does between platforms. With them, dbt_utils' haversine_distance runs.

Longer timeouts, abandoned renames settled, ACLs documented (#100)

• Timeouts (#86): the client waits 5 minutes for each reply and each send. It used go-redis's 5 s (10 s in effect on a standalone server), so slow statements failed.
• New options: adbc.redis.read_timeout and adbc.redis.write_timeout (database and connection) take a duration (30s, 10m) or seconds, 0 for none. They override the URI's, and the write timeout follows the read timeout unless set.
• Timeout errors say what to raise: … i/o timeout (Redis did not reply within the read timeout, 1s: raise adbc.redis.read_timeout, or set it to 0 for no timeout).
• Commands that mustn't run twice are sent once. go-redis resent them after a timeout, so FT.CURSOR READ could lose a page of rows.
• Abandoned renames (#87): an adbc.redis.rename_rekey rename whose connection died could keep its table stuck. Now Open, or the next statement that changes the table, rolls the move back or finishes it, and other connections wait for that instead of failing.
• ACL users (#94): the README lists the commands and keys the driver needs, with a full ACL SETUSER example and a read-only user's rules. Per-schema key patterns keep a user out of other schemas' tables. FT.DROPINDEX … DD deletes rows even with read-only key access, so a user that mustn't change data must not get it.
• Refused commands in a transaction are named, with their key, instead of a bare EXECABORT.

Dates and times become text as in Postgres; time zones and TO_CHAR (#97)

• Text (#84): casts, ||, MD5, the other text functions and error messages write dates and times as Postgres does: seconds always, then the fraction without trailing zeros. 2024-01-10 10:00:00.000000 is now 2024-01-10 10:00:00. Equal values give the same text at every precision, so dbt_utils' generate_surrogate_key over a timestamp matches Postgres. Arrow results don't change.
• Years and intervals: BC years are 0044-03-15 BC (was -0043-03-15), years past 9999 keep all their digits, and both read back. '0000-01-01' is now invalid date. Intervals read -1 days +02:00:00.
• AT TIME ZONE and AT LOCAL (#88), with Postgres's precedence and semantics, plus TIMEZONE(), and CONVERT_TIMEZONE() with 2 or 3 arguments, dbt_date's default.
• Zones: 38 of Postgres's abbreviations (PST, CET, …) as fixed offsets, IANA names in any case, POSIX offsets (west of Greenwich, as in Postgres) and INTERVALs. The time zone database is built in, adding about 460 KB to the library.
• DST, as in Postgres: a local time in a spring-forward gap takes the offset before the change, one in a fall-back overlap the offset after it. '2024-03-10 02:30 America/Los_Angeles' is now 10:30 UTC (was 09:30).
• TO_CHAR (#90) has every pattern and modifier in Postgres's table, read as Postgres reads them.
  • New: WW, J, CC, RM, FF1 to FF6, BC, TZH, OF, the TH ordinals and more.
  • Fixed: FM applies only to the next pattern, patterns are case-sensitive (Yyyy is no longer YYYY), TZ of a plain timestamp is empty, and an empty format gives NULL.

A session time zone, and rounding to a lower time precision (#106)

• Rounding (#103): a time or timestamp cast or written to a lower precision is rounded half away from zero, as in Postgres, where it used to be truncated. cast(timestamp '2024-01-10 10:00:00.5' as timestamp(0)) is now 10:00:01.
  • That covers casts and every write, including CREATE TABLE … AS, defaults and bulk ingest.
  • Carries reach the next day, month or year, and time(0) of 23:59:59.9 is 24:00:00, as in Postgres. Halves before 2000 round down, since Postgres counts from 2000-01-01.
  • Text compared with a time or timestamp keeps its digits, so ts0 = '… 10:00:00.5' no longer matches a TIMESTAMP(0) of 10:00:00.
• Session time zone (#105): SET TIME ZONE, SET timezone, SET LOCAL, RESET and SHOW timezone, per connection. It takes IANA names in any case, POSIX offsets, numbers, INTERVALs and abbreviations such as PST. The new option adbc.redis.time_zone (database and connection) sets the default.
• What the zone changes, as in Postgres: the text of a timestamp with time zone (2024-01-09 21:00:00-08), TO_CHAR, input without an offset, conversions between timestamp, timestamptz and date, EXTRACT, DATE_TRUNC, CURRENT_DATE and interval arithmetic (across the spring-forward change, + INTERVAL '1 day' adds 23 hours).
• ...

v0.0.7

Highlights

CHECK constraints are enforced (#76)

• Column constraints parse: CREATE TABLE and ADD COLUMN accept NOT NULL, NULL, CHECK (expr), DEFAULT, PRIMARY KEY, UNIQUE and REFERENCES t [(col)] (with MATCH, ON DELETE and ON UPDATE). Each may be named with CONSTRAINT name, and a column may have any number of them, in any order. On v0.0.6, amount integer check (amount >= 0) was a syntax error.
• Where CHECK is enforced:
  • INSERT (VALUES, SELECT, DEFAULT VALUES), UPDATE (also … FROM), MERGE and bulk ingest.
  • A statement with a failing row writes nothing. A row fails only when the expression is FALSE; NULL passes.
  • The error is Postgres's: new row for relation "t" violates check constraint "t_amount_check", an IntegrityError in Python.
• Table-level CHECK is stored and enforced too. Earlier versions parsed it and skipped it.
• Names follow Postgres: <table>_<column>_check when the expression reads one column, otherwise <table>_check, numbered on a collision.
• New ALTER TABLE forms: ADD [CONSTRAINT n] CHECK (…) and DROP CONSTRAINT [IF EXISTS] n. Adding a constraint, or a column with one, checks the existing rows first.
• What keeps them: RENAME TO (also with adbc.redis.rename_rekey), RENAME COLUMN and TRUNCATE. DROP COLUMN drops the constraints that read the column.
• GetObjects: table_constraints, which was always empty, lists each CHECK.
• Not enforced: PRIMARY KEY, UNIQUE and FOREIGN KEY are accepted and ignored, as before.
• dbt: the adapter can now declare check as enforced and foreign_key as not enforced. It had to declare both unsupported, so dbt dropped them.

Queries that can't return rows read nothing (#77)

• What's detected:
  • WHERE false, WHERE 1 = 0, and any other condition that is FALSE or NULL whatever the rows hold.
  • LIMIT 0, FETCH FIRST 0 ROWS ONLY, and a HAVING or QUALIFY that is never true.
  • At any level: subqueries, CTEs, derived tables, views, join items and set-operation branches.
• Joins: an item that can't match is read only when it can change the result. An inner join with one is empty, and LEFT JOIN … ON false keeps the left rows without reading the right.
• Writes: INSERT … SELECT, UPDATE, DELETE and MERGE over such inputs read nothing.
• ExecuteSchema and CREATE VIEW plan the statement without running any of it.
• Same columns: the result has the full query's column names, types and nullability. So CREATE TABLE … AS over dbt's empty form creates the full query's columns.
• dbt: contract checks (select * from (…) as __dbt_sbq where false limit 0), snapshots, unit tests and dbt run --empty no longer cost a full build. In a benchmark on a 152,686-row table, a model's contract check went from about 1 s to about 1 ms, and the model's query under --empty from 2.7 s to 1 ms.
• New syntax: FETCH {FIRST | NEXT} [n] {ROW | ROWS} ONLY and OFFSET n ROW[S]. LIMIT 0 OFFSET n now works.

Function calls are checked while planning (#79)

• Unknown functions and wrong argument counts are errors whether or not rows are read. That covers empty tables, WHERE false, ExecuteSchema, CREATE VIEW, and CHECK and DEFAULT definitions.
• Messages:
  • Unknown functions get Postgres's function nosuchfunc(integer) does not exist.
  • Counts have one format: UPPER expects 1 argument, ROUND expects 1 or 2 arguments, GREATEST expects at least 1 argument.
  • When a statement has several problems, the one reported is the one Postgres reports.
• Value errors stay on rows: 1/0, failed casts and LOG(0) still fail only on rows that are read.
• New information_schema.routines: one row per function name, aliases included, with its kind and argument counts.
• Crash fix: a call written with an internal name, such as __is_json(x), panicked, which takes down the host process. It's now an unknown function.

Redis Flex is refused on connect (#81)

• The check: connecting to a Flex (RAM + SSD) database fails at once with Redis Flex databases are not supported: …. The status is NotImplemented, a NotSupportedError in Python. Nothing is written first.
• Why: Search on Flex doesn't support the NUMERIC SORTABLE row-ID field every table's index has, or the FT.AGGREGATE cursors every query reads through. On v0.0.6, every CREATE TABLE and ingest on Flex failed with SEARCH_FLEX_UNSUPPORTED_FIELD, after the driver had written its metadata.
• How: the driver runs one FT.AGGREGATE on an index that doesn't exist, once per connection. It tests whether FT.AGGREGATE works, not whether the database is Flex, so a Flex release that supports it will connect.
• Tested on: Redis Software 8.2.0-78. Not tested on Redis Cloud Pro Flex: if its error wording differs, users get the raw SEARCH_FLEX_UNSUPPORTED_FIELD error at CREATE TABLE instead. The README's Server requirements has the details per deployment.
• Local Flex test: go/flex-setup.sh starts a single-node Redis Software cluster with a Flex database, and TestFlexRefused runs when REDIS_FLEX_URI is set. CI doesn't run it.

Upgrading from v0.0.6

• Upgrade every client of a database together. v0.0.7 stores CHECK constraints in an optional checks metadata field, and no migration is needed. v0.0.6 ignores the field, so it doesn't enforce the constraints. Any metadata change it makes removes them: an ALTER TABLE, a COMMENT ON, or some writes.

• Tables from earlier versions: they never stored a table-level CHECK, so theirs aren't enforced. Add them with ALTER TABLE … ADD CHECK (…).

• Stored objects that call an unknown function now fail when used:

  • a view, on every read (view "v" is no longer valid: …);
  • a CHECK, on every write to its table;
  • a DEFAULT, on the inserts that need it.

  CREATE OR REPLACE VIEW, DROP VIEW, DROP CONSTRAINT or ALTER COLUMN … DROP DEFAULT fixes them.

• Behaviour changes:

  • Planning errors: an unknown function or a wrong argument count fails even when no rows are read. ExecuteSchema of such a query used to return a null column.
  • NULL arguments no longer hide a wrong count. upper(NULL, NULL) used to return NULL.
  • DISTINCT and * on scalar functions are rejected. upper(DISTINCT s) used to ignore the DISTINCT.
  • CURRENT_DATE, NOW and the other current-time functions take at most one argument.
  • Internal names can't be called. __interval(2, 'day') used to work.
  • Unused named windows are checked, so WINDOW w AS (ORDER BY nosuchcol) fails, as in Postgres.
  • Messages: argument counts use the format above, unsupported function X is now function x(…) does not exist, and function … does not exist messages are in lower case.
  • Rows that aren't read raise nothing: WHERE x / 0 = 1 AND false no longer raises an error on a table with rows.
  • CREATE VIEW no longer runs its body's CTEs and derived tables. An error that only shows on rows now appears when the view is read.
  • FETCH is reserved after an expression, so an alias named fetch needs AS. A second LIMIT gives multiple LIMIT clauses not allowed; the last one used to win.
  • Constraint syntax: CONSTRAINT name with no constraint after it is a syntax error, and so is DEFERRABLE after a table-level CHECK. Both used to be skipped.
  • ALTER TABLE t ADD unique … / ADD check … without COLUMN adds a constraint, as in Postgres.
  • Flex databases are refused on connect (see above).

Tested with

Before the release, main passed the full ADBC validation suite (324 passed, 10 skipped, 1 expected failure) and all 196 Go tests. TestFlexRefused was skipped, since it needs a Flex database; #81 ran it against Redis Software 8.2.0-78.

Setup	Validation suite	Go tests
Redis 8.6.2, standalone	✓	✓
Redis 8.6.2, 3-shard cluster	✓	✓

The release workflow then tests the built libraries; see Assets. This release wasn't run against Redis Cloud.

Assets

The release workflow attaches these once its tests pass. Each tarball contains the shared library, its C header and the README; each .sha256 file has the tarball's checksum.

Asset	Platform
adbc_driver_redis-v0.0.7-darwin-arm64.tar.gz	macOS, Apple Silicon
adbc_driver_redis-v0.0.7-linux-amd64.tar.gz	Linux x86-64: RHEL 8+, Ubuntu 20.04+, Amazon Linux 2023
adbc_driver_redis-v0.0.7-linux-arm64.tar.gz	Linux arm64: same distributions

For other platforms:

cd go && make build