Repository navigation
Releases: bmsuisse/arrowbricks
Releases · bmsuisse/arrowbricks
Release list
arrowbricks v6.2.0
Why TSV: with the shipped renderer on ten sample result sets, TSV used 28% fewer
o200k_base tokens than 6.1's Markdown table (12% fewer than the new unpadded
Markdown), decoded back losslessly for all 14,810 cells, and could not be broken
by any Unicode character. In 12 blind reader runs (Haiku and Sonnet, 82
questions) the layouts scored the same within noise (TSV 97.6%, new Markdown
97.6%, 6.1 Markdown 98.8%), so TSV is cheaper without being less accurate; the
benchmark does not show it is more accurate. Method, numbers and caveats:
benchmarks/2026-10-12-agent-output-format.md.
- Add
output_formattoDatabricksClient.query_agentand--formatto the
CLI:"tsv"(the new default) or"markdown"(typed as the exported
arrowbricks.AgentFormatliteral). TSV prints aname:Type
header and one tab-separated line per row. Backslash, tab, newline and
carriage return in values become\\,\t,\nand\r, SQL NULL is\N
and the empty string is an empty field, so a cell can never contain a raw
delimiter. - Behavior change: the default output is now TSV, and the Markdown table no
longer pads cells with spaces (|a (Int64)|b (Utf8)|, separator|-|-|), 19% fewer tokens.
Passoutput_format="markdown"/--format markdownfor the table. Both
layouts end with the same completeness footer and share the byte budget. - The CLI recognises an error by parsing it as a JSON object, so a column named
like{x}no longer sends a successful TSV result to stderr.
arrowbricks v6.1.0
- Accept PEP 750 t-strings (Python 3.14+) as SQL on every method that takes it:
each interpolation is bound as a typed named parameter,{name:i}quotes an
identifier, nested t-strings compose, and explicit parameters still merge in.
sqlglot expressions are accepted too (arrowbricks[sqlglot]extra). - Type
sqlasLiteralString | Template | <sqlglot expression>(arrowbricks.Sql), so type checkers flag
SQL built from runtime strings. Typing only -- astrstill works at runtime;
arrowbricks.unsafe_sql(text)marks SQL you vouch for (#21).
arrowbricks v6.0.1
- Write CLI query output as UTF-8 bytes, so Unicode works under ASCII locales
and Windows newline conversion cannot exceed the output byte limit. - Lead the README with the dependency, performance, and agent-mode benefits;
move detailed usage and benchmark notes into linked documentation.
arrowbricks v6.0.0
- Add an explicit
DatabricksClient.query_agentinterface returning compact,
typed Markdown on success and actionable JSON diagnostics on failure.
Query execution, bounded Arrow rendering and cancellation stay in Rust;
no required Python dependencies are added. - Agent output has configurable row, UTF-8 byte and cell limits, with an
explicit completeness footer. It stops fetching a bounded prefix instead
of downloading the whole result to count it. Existing fetch APIs are unchanged. - Add the
arrowbricksCLI andpython -m arrowbricks, with compact Markdown
on stdout, JSON diagnostics on stderr, and SQL arguments, files and stdin. - Add agent-mode examples, automated coverage on both protocols, and
reproducible renderer and live SQL warehouse verification. - Ship
skills/arrowbricks/SKILL.mdwith agent guidance for the CLI and Python
API, result completeness, budgets and JSON error recovery.
arrowbricks v5.2.2
- Thrift: a pooled session the server no longer knows no longer fails the
next statement (issue #14). When Databricks rejectsExecuteStatementon
an idle pooled session withINVALID_STATE: Invalid SessionHandle: ...
(HTTP 400; the statement never ran), arrowbricks used to drop the session
and raise. It now also discards the key's other idle sessions (they go
stale together) and retries the statement once on a newly opened pooled
session. Only a session taken from the idle pool is retried, only for that
error, and only once; any other rejection (bad SQL, auth, ...) still fails
on the first attempt.
What was and was not shown: the error is real and the fix works against the
real warehouse (closing the pooled session server-side makes 5.2.1 fail and
5.2.2 recover). The suspected trigger, a warehouse auto-stop, did not
invalidate sessions on the serverless Pro warehouse we tested (409 s idle
until auto-stop, 60 s more stopped, then the old session worked), so this
is a defensive fix for whatever does invalidate a session (session expiry,
an administrator or another client closing it, other warehouse types),
not a confirmed cure for the auto-stop report.
arrowbricks v5.2.1
- Fix an async
token_providerfailing withAuthError: RuntimeError: no running event loopon a client that had not run any query yet.
stream_query_jsonwas the usual trigger: since 3.2.0 its statement is
submitted from a background task, and the provider used to look for its
event loop in whichever task first asked for a token. Once any
execute()had run on the client, the loop it found then was reused,
which is why it only hit cold clients. - The event loop (and
contextvars) is now captured when you call the
method, where your loop is running, and travels with the query into
every background task it starts: the submit, chunk downloads, the Thrift
fetch loop, and best-effort cancels. Each token request therefore runs on
the loop of the query that needs it, also when one client is used from
several loops at once (one per thread) -- before, a token for one loop's
query could be fetched on another loop. - Sync
token_providers are unchanged. An asyncon_eventcallback picks
its event loop the same way now, so it also fires for a cold client's
first query when that query times out or is cancelled (it used to need an
earlier query on the same client to have reported first). - Tests:
tests/test_token_provider_event_loop.py(both protocols; every
public entry point on a cold client, two successiveasyncio.run()s,
asyncio.run()in a thread, two threads at once, and a timed-out stream
whose cancel and asyncon_eventmust still run),
tests_py/test_token_provider.py, and call-context tests in
call_context.rs,wiremock_pipeline.rsandwiremock_thrift.rs. The
cold-client streaming tests fail on 5.2.0. - No API changes.
arrowbricks v5.1.2
- Stop the background fetch as soon as a result is abandoned. Cancelling
the task that awaits a fetch, closing astream_query_jsoniterator
(what a web framework does when the client disconnects), or dropping a
Cursorbefore its result is drained used to leave the chunk downloads
already in flight running to completion (up tochunk_fetch_concurrency
of them), and workers could start new ones after the consumer was gone.
In-flight downloads are now aborted right away, on both protocols. - Thrift: closing the operation after an abandoned fetch no longer hangs.
With more links left than the download workers and their queue could hold
(about2 * chunk_fetch_concurrency; 128 by default), the fetch loop
blocked forever, soCloseOperation(andCloseSessionfor a throwaway
session) never went out, and the task, its client handle and its
connection pool were never freed. The loop now also stops issuing
FetchResultsonce nobody reads the result. - Unchanged: server-side cancel (
CancelOperation/ SEAPOST .../cancel)
still fires when a still-running statement is abandoned, as since 3.2.0. - Tests:
wiremock_thrift.rs/wiremock_pipeline.rsdrop a partly read
result with a download in flight, andtest_thrift_pipeline.pystops
readingstream_query_jsonafter the first rows. All three fail on 5.1.1. - No API changes.
arrowbricks v5.1.1
- Fix a hole in 5.0.2's
read_body: the up-front buffer reservation came from
the untrustedContent-Length(capped at 1 GiB per download), and a failed
reservation aborted the process, which the new allocator can turn into a
crash underRLIMIT_ASor strict overcommit. It now reserves at most
256 MiB, and falls back toResponse::bytes()if even that fails. - The allocator is behind a default-on
large-block-alloccargo feature and
now requires Linux with glibc (it was only ever measured there; musl wheels
are unaffected). A Rust crate that depends onarrowbricks_corewith its
own global allocator can opt out withdefault-features = false. - Tests: a check that fails if the allocator is removed or mis-gated,
read_body
with and withoutContent-Lengthand with a short body, the Thrift inline
LZ4 path with a zero-content frame, andimport arrowbricksnot loading
asyncio(plus deferred submodules on 3.15). AGENTS.mdandrust/.../alloc.rs:// SAFETY:comments with a lint to keep
them, and design-invariant entries for the allocator,read_bodyand the
lazy-import rule; the LZ4-loop entry now points atlz4_frame_decode_into.- Removed the claim in 5.0.0 that the 3.15 suite passes with
-W error: four
tests fail intermittently under it from unclosed mock-server sockets. - No API changes.
arrowbricks v5.1.0
- Linux: buffers of 1 MiB or more now come straight from
mmapand go back
to the OS when freed (src/alloc.rs, a scoped global allocator for this
extension only; the host process's malloc is untouched, other platforms
are unchanged). glibc raises its mmap threshold as large blocks are freed,
so repeated queries in one long-lived process kept the memory: idle RSS
grew from 200 MB to 1.4 GB over five queries of the same 500k-row table.
Measured on a real warehouse (5 workloads, Thrift and SEA, LZ4-compressed
results, interleaved against 5.0.3): peak RSS -41% to -64% on the larger
results, idle RSS after repeated queries about -80%, time unchanged within
noise. Tiny results are unaffected. No API changes. - Correction to 5.0.2: its memory and speed figures were measured against a
local mock warehouse serving uncompressed bodies. Against a real warehouse,
where results arrive LZ4-compressed, 5.0.2 and 5.0.3 behave like 4.0.0 on
fetch time and memory; the gain described here comes from 5.1.0.
arrowbricks v5.0.3
- Internal cleanup, no behavior change: the 21 hand-built permanent
ApiErrorliterals now useApiError::permanent; the multi-frame LZ4
decode loop (with its past silent-truncation fix) lives in one helper
shared by the chunk-download and Thrift inline-blob paths;Cursor
shares one schema-refresh helper and no longer copies the row list when
nothing is buffered infetchall()/fetchmany().