Skip to content

Releases: bmsuisse/arrowbricks

arrowbricks v6.2.0

Choose a tag to compare

@github-actions github-actions released this 12 Oct 19:24
3741a93

Why TSV: with the shipped renderer on ten sample result sets, TSV used 28% fewer
o200k_base tokens than 6.1's Markdown table (12% fewer than the new unpadded
Markdown), decoded back losslessly for all 14,810 cells, and could not be broken
by any Unicode character. In 12 blind reader runs (Haiku and Sonnet, 82
questions) the layouts scored the same within noise (TSV 97.6%, new Markdown
97.6%, 6.1 Markdown 98.8%), so TSV is cheaper without being less accurate; the
benchmark does not show it is more accurate. Method, numbers and caveats:
benchmarks/2026-10-12-agent-output-format.md.

  • Add output_format to DatabricksClient.query_agent and --format to the
    CLI: "tsv" (the new default) or "markdown" (typed as the exported
    arrowbricks.AgentFormat literal). TSV prints a name:Type
    header and one tab-separated line per row. Backslash, tab, newline and
    carriage return in values become \\, \t, \n and \r, SQL NULL is \N
    and the empty string is an empty field, so a cell can never contain a raw
    delimiter.
  • Behavior change: the default output is now TSV, and the Markdown table no
    longer pads cells with spaces (|a (Int64)|b (Utf8)|, separator |-|-|), 19% fewer tokens.
    Pass output_format="markdown" / --format markdown for the table. Both
    layouts end with the same completeness footer and share the byte budget.
  • The CLI recognises an error by parsing it as a JSON object, so a column named
    like {x} no longer sends a successful TSV result to stderr.

arrowbricks v6.1.0

Choose a tag to compare

@github-actions github-actions released this 12 Oct 06:58
5243cf6
  • Accept PEP 750 t-strings (Python 3.14+) as SQL on every method that takes it:
    each interpolation is bound as a typed named parameter, {name:i} quotes an
    identifier, nested t-strings compose, and explicit parameters still merge in.
    sqlglot expressions are accepted too (arrowbricks[sqlglot] extra).
  • Type sql as LiteralString | Template | <sqlglot expression> (arrowbricks.Sql), so type checkers flag
    SQL built from runtime strings. Typing only -- a str still works at runtime;
    arrowbricks.unsafe_sql(text) marks SQL you vouch for (#21).

arrowbricks v6.0.1

Choose a tag to compare

@github-actions github-actions released this 11 Oct 12:22
99bc5b2
  • Write CLI query output as UTF-8 bytes, so Unicode works under ASCII locales
    and Windows newline conversion cannot exceed the output byte limit.
  • Lead the README with the dependency, performance, and agent-mode benefits;
    move detailed usage and benchmark notes into linked documentation.

arrowbricks v6.0.0

Choose a tag to compare

@github-actions github-actions released this 11 Oct 11:58
ce28505
  • Add an explicit DatabricksClient.query_agent interface returning compact,
    typed Markdown on success and actionable JSON diagnostics on failure.
    Query execution, bounded Arrow rendering and cancellation stay in Rust;
    no required Python dependencies are added.
  • Agent output has configurable row, UTF-8 byte and cell limits, with an
    explicit completeness footer. It stops fetching a bounded prefix instead
    of downloading the whole result to count it. Existing fetch APIs are unchanged.
  • Add the arrowbricks CLI and python -m arrowbricks, with compact Markdown
    on stdout, JSON diagnostics on stderr, and SQL arguments, files and stdin.
  • Add agent-mode examples, automated coverage on both protocols, and
    reproducible renderer and live SQL warehouse verification.
  • Ship skills/arrowbricks/SKILL.md with agent guidance for the CLI and Python
    API, result completeness, budgets and JSON error recovery.

arrowbricks v5.2.2

Choose a tag to compare

@github-actions github-actions released this 10 Oct 12:01
7835f09
  • Thrift: a pooled session the server no longer knows no longer fails the
    next statement (issue #14). When Databricks rejects ExecuteStatement on
    an idle pooled session with INVALID_STATE: Invalid SessionHandle: ...
    (HTTP 400; the statement never ran), arrowbricks used to drop the session
    and raise. It now also discards the key's other idle sessions (they go
    stale together) and retries the statement once on a newly opened pooled
    session. Only a session taken from the idle pool is retried, only for that
    error, and only once; any other rejection (bad SQL, auth, ...) still fails
    on the first attempt.
    What was and was not shown: the error is real and the fix works against the
    real warehouse (closing the pooled session server-side makes 5.2.1 fail and
    5.2.2 recover). The suspected trigger, a warehouse auto-stop, did not
    invalidate sessions on the serverless Pro warehouse we tested (409 s idle
    until auto-stop, 60 s more stopped, then the old session worked), so this
    is a defensive fix for whatever does invalidate a session (session expiry,
    an administrator or another client closing it, other warehouse types),
    not a confirmed cure for the auto-stop report.

arrowbricks v5.2.1

Choose a tag to compare

@github-actions github-actions released this 10 Oct 09:41
6a103d2
  • Fix an async token_provider failing with AuthError: RuntimeError: no running event loop on a client that had not run any query yet.
    stream_query_json was the usual trigger: since 3.2.0 its statement is
    submitted from a background task, and the provider used to look for its
    event loop in whichever task first asked for a token. Once any
    execute() had run on the client, the loop it found then was reused,
    which is why it only hit cold clients.
  • The event loop (and contextvars) is now captured when you call the
    method, where your loop is running, and travels with the query into
    every background task it starts: the submit, chunk downloads, the Thrift
    fetch loop, and best-effort cancels. Each token request therefore runs on
    the loop of the query that needs it, also when one client is used from
    several loops at once (one per thread) -- before, a token for one loop's
    query could be fetched on another loop.
  • Sync token_providers are unchanged. An async on_event callback picks
    its event loop the same way now, so it also fires for a cold client's
    first query when that query times out or is cancelled (it used to need an
    earlier query on the same client to have reported first).
  • Tests: tests/test_token_provider_event_loop.py (both protocols; every
    public entry point on a cold client, two successive asyncio.run()s,
    asyncio.run() in a thread, two threads at once, and a timed-out stream
    whose cancel and async on_event must still run),
    tests_py/test_token_provider.py, and call-context tests in
    call_context.rs, wiremock_pipeline.rs and wiremock_thrift.rs. The
    cold-client streaming tests fail on 5.2.0.
  • No API changes.

arrowbricks v5.1.2

Choose a tag to compare

@github-actions github-actions released this 10 Oct 07:44
92f6adb
  • Stop the background fetch as soon as a result is abandoned. Cancelling
    the task that awaits a fetch, closing a stream_query_json iterator
    (what a web framework does when the client disconnects), or dropping a
    Cursor before its result is drained used to leave the chunk downloads
    already in flight running to completion (up to chunk_fetch_concurrency
    of them), and workers could start new ones after the consumer was gone.
    In-flight downloads are now aborted right away, on both protocols.
  • Thrift: closing the operation after an abandoned fetch no longer hangs.
    With more links left than the download workers and their queue could hold
    (about 2 * chunk_fetch_concurrency; 128 by default), the fetch loop
    blocked forever, so CloseOperation (and CloseSession for a throwaway
    session) never went out, and the task, its client handle and its
    connection pool were never freed. The loop now also stops issuing
    FetchResults once nobody reads the result.
  • Unchanged: server-side cancel (CancelOperation / SEA POST .../cancel)
    still fires when a still-running statement is abandoned, as since 3.2.0.
  • Tests: wiremock_thrift.rs/wiremock_pipeline.rs drop a partly read
    result with a download in flight, and test_thrift_pipeline.py stops
    reading stream_query_json after the first rows. All three fail on 5.1.1.
  • No API changes.

arrowbricks v5.1.1

Choose a tag to compare

@github-actions github-actions released this 07 Oct 14:32
f5c5e1f
  • Fix a hole in 5.0.2's read_body: the up-front buffer reservation came from
    the untrusted Content-Length (capped at 1 GiB per download), and a failed
    reservation aborted the process, which the new allocator can turn into a
    crash under RLIMIT_AS or strict overcommit. It now reserves at most
    256 MiB, and falls back to Response::bytes() if even that fails.
  • The allocator is behind a default-on large-block-alloc cargo feature and
    now requires Linux with glibc (it was only ever measured there; musl wheels
    are unaffected). A Rust crate that depends on arrowbricks_core with its
    own global allocator can opt out with default-features = false.
  • Tests: a check that fails if the allocator is removed or mis-gated, read_body
    with and without Content-Length and with a short body, the Thrift inline
    LZ4 path with a zero-content frame, and import arrowbricks not loading
    asyncio (plus deferred submodules on 3.15).
  • AGENTS.md and rust/.../alloc.rs: // SAFETY: comments with a lint to keep
    them, and design-invariant entries for the allocator, read_body and the
    lazy-import rule; the LZ4-loop entry now points at lz4_frame_decode_into.
  • Removed the claim in 5.0.0 that the 3.15 suite passes with -W error: four
    tests fail intermittently under it from unclosed mock-server sockets.
  • No API changes.

arrowbricks v5.1.0

Choose a tag to compare

@github-actions github-actions released this 07 Oct 09:20
a7e0830
  • Linux: buffers of 1 MiB or more now come straight from mmap and go back
    to the OS when freed (src/alloc.rs, a scoped global allocator for this
    extension only; the host process's malloc is untouched, other platforms
    are unchanged). glibc raises its mmap threshold as large blocks are freed,
    so repeated queries in one long-lived process kept the memory: idle RSS
    grew from 200 MB to 1.4 GB over five queries of the same 500k-row table.
    Measured on a real warehouse (5 workloads, Thrift and SEA, LZ4-compressed
    results, interleaved against 5.0.3): peak RSS -41% to -64% on the larger
    results, idle RSS after repeated queries about -80%, time unchanged within
    noise. Tiny results are unaffected. No API changes.
  • Correction to 5.0.2: its memory and speed figures were measured against a
    local mock warehouse serving uncompressed bodies. Against a real warehouse,
    where results arrive LZ4-compressed, 5.0.2 and 5.0.3 behave like 4.0.0 on
    fetch time and memory; the gain described here comes from 5.1.0.

arrowbricks v5.0.3

Choose a tag to compare

@github-actions github-actions released this 06 Oct 21:38
8b54a69
  • Internal cleanup, no behavior change: the 21 hand-built permanent
    ApiError literals now use ApiError::permanent; the multi-frame LZ4
    decode loop (with its past silent-truncation fix) lives in one helper
    shared by the chunk-download and Thrift inline-blob paths; Cursor
    shares one schema-refresh helper and no longer copies the row list when
    nothing is buffered in fetchall()/fetchmany().