Repository navigation
QuackDB now targets DuckDB 2.0 and its stable Quack extension, which speaks Quack protocol version 3. DuckDB 1.5 servers are no longer supported: the two protocol versions are not wire-compatible, and connecting to an older server fails with :unsupported_quack_version.
Breaking changes
- Requires DuckDB 2.0. The managed binary is DuckDB 2.0.0, downloaded as the new
duckdb-cli-<target>.tar.gzarchives with checksums for all Linux and macOS targets, includingosx-universal. The DuckDB 1.5 checksums are gone, andQuackDB.Binaryrejects CLIs older than 2.0. QuackDB.Serverno longer sets the removedquack_fetch_batch_chunkssetting, which made DuckDB 2.0 exit during boot (#8). DuckDB 2.0 sizes result batches by bytes withquack_target_batch_bytes.- Native appends no longer use the removed
APPEND_REQUESTmessage. Each append is oneINSERT INTO table (columns) SELECT * FROM scan_data_from_quack_client(...)statement fed by numbered send-data batches. Omitted columns now get their defaults and sequence values, and append errors no longer carry theFailed to append:prefix. QuackDB.Profilefollows DuckDB 2.0's grouped profile JSON:query,parser,planner,physical_planner,optimizer,io,system, andoperators. Operators carrytype,timing,intermediate_rows, androws_scanned; the flat 1.5 fields such aslatencyandoperator_timingare gone.- Append type inference considers every row and value instead of the first. Rows with keys the first row lacks now append those columns too, and columns that mix shapes, such as maps and integers, fail with
:mixed_append_typesinstead of dropping or NULLing values. Declare:columns(or:variant) for such data. result_uuidis renamedquery_uuidinQuackDB.Query,QuackDB.Cursor, result metadata, and fetch telemetry, matching the protocol.- DuckDB 2.0 itself changes some output: function-call column names follow the new parser (
"version"()), integer division by zero raises,EXPLAINrenders boxes with readable operator names, and database files written by 2.0 cannot be opened by DuckDB 1.5.
Added
- Query cancellation.
QuackDB.cancel/3interrupts the query on a remote connection from another process, and the query fails with:query_cancelled. A transport timeout cancels its query on the server, and DBConnection disconnects cancel over a fresh HTTP connection. - Typed server errors.
QuackDB.Errormetadata carries DuckDB's:exception_type(also viaQuackDB.Error.exception_type/1) and transferable:extra_info. A server whose database died returns:connection_invalidatedand an expired heartbeat lease returns:connection_expired; both disconnect so the pool reconnects. - Heartbeat leases. Connections request a lease with
:heartbeat_timeout(60 seconds by default) and renew it every third of the lease over a separate HTTP connection, as DuckDB's own client does, so queries, streams, and transactions that outlast the lease keep their connection. DBConnection idle pings send heartbeat messages instead ofSELECT 1.:client_idnames the client in server logs. ON CONFLICTandRETURNINGfor native appends:insert_rows/4andinsert_columns/4accept:on_conflictand:returning, and Ectoinsert_all(..., insert_method: :append)runson_conflict: :nothingandreturningas one statement instead of staging rows in a temporary table. Schemaless appends can usereturning.VARIANTdecoding, unshredded and shredded, into Elixir terms.:variantand:jsonappend column types send JSON text that DuckDB casts, so mixed objects, arrays, and scalars keep their shapes.QuackDB.Ecto.Variantwithvariant_extract/2(keys, 1-based indexes, and paths),variant_extract_string/2,variant_type/1,2,variant_typeof/1,variant_keys/1,2,variant_array_length/1,2, andvariant_contains/2, imported byuse QuackDB.Ecto.TUPLEvalues decode to Elixir tuples, andTIMESTAMPTZ_NSvalues toQuackDB.NanosecondTimestamp.- Ecto
json_set/3,json_insert/3,json_replace/3, andjson_remove/2, andvariable/1for DuckDB 2.0's$namevariables, withQuackDB.SQL.set_variable/2andreset_variable/1. QuackDB.DDL.create_trigger/4anddrop_trigger/3for DuckDB 2.0 triggers, andcreate_schema/2anddrop_schema/2. Table names accept nested schema paths such as{[:finance, :reports], :ledger}, including the:schemaappend option.- Ecto migrations can
executeQuackDB.DDLbuilders directly (execute DDL.create_trigger(...)); the adapter accepts iodata DDL. - Reconnect support:
acknowledge_results: trueconfirms each full result to servers running withquack_enable_reconnects, and a fetch that lost its HTTP connection requests the same numbered batch again (:fetch_retries, default 1). - An
:inline_rowsquery option caps the rows of the first response; cursors default it to:max_rows. - A
:copy_binariesoption, per connection or per query, copies decoded strings and blobs off the HTTP response, so keeping a few values from a large result no longer keeps the whole response in memory. QuackDB.Serveraccepts:serve_options(allow_other_hostname,disable_ssl,ssl_cert_file,ssl_key_file) and derives anhttpsURI for non-local hosts asquack_servedoes. The CLI runs with-no-agentso DuckDB 2.0's agent output mode stays off under coding agents.
Changed
{:json, term}parameters encode with Elixir's built-inJSONmodule, so they no longer need Jason.- Faster result decoding. Fixed-width vectors decode with one binary comprehension per column instead of one call per value, temporal values are built from calendar fields, and LIST, ARRAY, STRUCT, and VARCHAR vectors decode in one pass. Per 2048-row vector this takes 2x fewer reductions for integers, lists, and arrays and 4x fewer for timestamps.
Fixed
- Connecting to DuckDB 2.0 failed with
unknown error response field 2(#8). - Dictionary, ARRAY, and VARIANT vectors decoded in quadratic time: a 2048-row dictionary vector took 1.2 ms instead of about 60 µs, and a 4000-element VARIANT array 29 ms instead of about 0.5 ms.
- Empty STRUCT values, which DuckDB 2.0 allows (
SELECT {}), failed to decode. GEOMETRYcolumns with a coordinate reference system, such asGEOMETRY('OGC:CRS84'), failed to decode with:invalid_string.- Append type inference looked only at the first non-nil value of each column. A later larger integer was silently truncated to 32 bits, struct keys and row keys missing from the first value were dropped, a non-map value in a struct column became NULL, and integers mixed with floats or strings crashed while encoding. Inference now considers every value: integers widen to
BIGINTorHUGEINT, integers join floats asDOUBLEwhile that stays exact, decimals widen to oneDECIMAL, and struct fields and row columns are the union of all keys. Any other mix fails with:mixed_append_types, naming the column and suggesting:columnsor:variant. DECIMAL(p, 0)values decoded as integers instead ofDecimal, because DuckDB omits a zero scale.- Result types QuackDB cannot decode, such as
UNION, and malformed UTF-8,BIGNUM, andBITvalues raised and killed the connection process. The query now fails with:unsupported_typeor the matching decode error. - A failed handshake (a wrong token, an older server) left its HTTP connection process running, one more on every reconnect attempt.
- Ecto reads,
update_all, anddelete_allignored schema prefixes and queried the unqualified table:from(... prefix: ...),@schema_prefix, andRepo.all(query, prefix: ...)now qualify the source, as inserts and migrations already did. A dotted prefix such as"finance.reports"names a nested schema orcatalog.schema, in reads and ininsert_method: :append. As in Ecto's planner, the query prefix does not apply to CTE names.