SQLDesk 0.7.2 — Over the Wire

Nothing compressed a response, every driver was behind, and a DuckDB upgrade uncovered a fault older than the upgrade.

Nothing in SQLDesk compressed a response

• Not the app, not the chart, not the ingress templates. A /ping asked with Accept-Encoding: gzip came back with no Content-Encoding at all — which is how it survived, because there was nothing to notice.
• A 20,000-row result sent its full 3,715,944 bytes, where 420 KB would do.
• And 6,747,167 bytes of JavaScript and CSS went out byte for byte on every cold page load. They gzip to 1,630,082 — a factor of 4.1.
• The codecs are chosen from measurement, not habit: brotli q4 at 27 ms and gzip 6 at 31 ms, both 8.8×. Level 9 is deliberately absent — 157 ms for ten percent more. For scale, encoding that result as JSON already costs 112 ms.
• The saving that matters is not bandwidth. A sync Gunicorn worker, of which there are four, holds its socket for a fraction as long, and the web tier is the first ceiling an install meets.
• Only text. A PNG from the renderer, an .xlsx (a zip) and a Parquet upload would each get bigger for CPU spent. SQLDESK_COMPRESS_RESPONSES=false turns the whole thing off.

Every number and timestamp from DuckDB was declared a string

• TYPES_MAP has always been keyed by DuckDB's own type names, and up to 1.3 the driver reported DB-API ones instead: NUMBER, STRING, DATETIME, bool.
• Of twelve ordinary column types, five matched. The seven that did not included every numeric type and every timestamp — so every number and every timestamp from a DuckDB source, and therefore from every uploaded CSV and Parquet file, arrived typed as text.
• It was invisible because the browser sniffs the type of any column declared string and recovers most of it. Charts have been quietly getting the right answer by guessing.
• 1.5 reports INTEGER, TIMESTAMP, BOOLEAN and the rest, which the table already knew — so the upgrade fixes it. Eleven of twelve now map with nothing but a str(); the twelfth, DECIMAL(10,2), is parameterised exactly like Trino's and handled the same way.
• A test now asks DuckDB for a column of each of seventeen types, so a rename fails loudly instead of quietly typing everything as text.
• The upgrade also broke the runner outright before that was found: cursor.description's type code became a DuckDBPyType, and .upper() on one forwards to its child-type lookup rather than raising an ordinary attribute error — so the failure read "Tried to get child type by the name of 'upper'", which says nothing about the cause.

Drivers

Eight upgrades, each checked rather than bumped.

	from	to
trino	0.330.0	0.340.0
duckdb	1.3.2	1.5.6
mysqlclient	2.1.1	2.3.0
pymongo	4.6.3	4.18.2
snowflake-connector-python	4.5.0	4.8.0
pyhive	0.6.1	0.7.0
impyla	0.22.0	0.24.0
psycopg2-binary	2.9.11	2.9.13

• DuckDB persists real database files — a stream window is one, and a DuckDB data source points at one — so the storage format was checked in both directions. 1.5.6 reads and writes a file written by 1.3.2, and 1.3.2 reads one written by 1.5.6, so the upgrade and the rollback are both safe.
• MySQL's ssl_mode is what 0.7.1 fixed, so 2.3.0 had to still take it as its own connect argument. It does — and it still ignores an unknown key in the ssl mapping without complaint, which is the behaviour that made the original fault invisible.
• pymongo loses [srv,tls], which turns out never to have been real: neither extra has existed since pymongo 3.x, so the pin has been asking for nothing since the Redash days. SRV connection strings are unaffected — dnspython is a hard dependency.
• pandas, oracledb and pyathena are deliberately held, with the reason written beside each pin. pandas 3 changes copy-on-write and the default string dtype, and both utils/pandas.py and the Python runner depend on today's behaviour. oracledb is an eight-major-version jump. pyathena 3 renames its cursor classes, and its ArrowCursor belongs with the columnar work rather than a patch release.

Trino can use the spooled protocol

• Under the standard protocol every row of every result is paged through the coordinator's heap as JSON, which is Trino's own ceiling on large results.
• Spooling has the workers write segments to object storage for the client to fetch directly, compressed with lz4 or zstd, leaving the coordinator to pass only metadata.
• It is a field on the data source and off unless chosen, because it needs a server that offers it — Trino 466 and later, with spooling configured — and there is no way to tell from here whether one does.
• There is no ADBC driver for Trino, which is what prompted looking: it is not on PyPI, and Trino does not speak Flight SQL natively either, so that driver does not reach it.
• Worth knowing what this is not: the Python client fetches segments sequentially. Its only thread pool backgrounds the segment acknowledgements. Parallel fetch is a Java-client feature.

Images

ghcr.io/bot-netizen/sqldesk:0.7.2
ghcr.io/bot-netizen/sqldesk-screenshots:0.7.2

linux/amd64 and linux/arm64.

Upgrading from 0.7.1

Nothing to do beyond the image. No migration, no configuration change, no default moved.

Docker Compose

# .env
SQLDESK_IMAGE=ghcr.io/bot-netizen/sqldesk:0.7.2

docker compose -f compose.prod.yaml pull
docker compose -f compose.prod.yaml up -d

Helm — the image tag has no v; the git tag does.

helm upgrade sqldesk ./charts/sqldesk --reuse-values

Leaving image.tag unset is both correct and the safer habit: the chart defaults it to its own appVersion.

One thing to expect. A DuckDB column that rendered as text may now render as a number, a boolean or a date — because it always was one and is finally declared as such. Charts that worked will keep working, since the browser had been guessing the same answer, but a column can change its alignment or its formatting.

If people upload files, --set uploads.enabled=true still applies. That default changed in 0.7.0 and has not moved since.

Verified before tagging

• CI green on every job, including the end-to-end suite.
• 2418 backend tests and 697 frontend tests pass. black 23.1.0 and ruff 0.0.287 — the versions CI pins — clean across 645 files.
• Every new test was mutation-tested. Putting each fault back makes its own test fail: three mutants for the compression config, three for the DuckDB type mapping, one for the Trino encoding.
• Checked against the published image rather than the source, through compose.prod.yaml with the schema created before the services started:
  • /setup answers Content-Encoding: br, and the largest bundle comes down 1,103,381 → 305,161 bytes, a factor of 3.6, through the real server. /ping's five-byte body is correctly left alone.
  • DuckDB types a published-image query as integer, float, datetime, boolean, string and string — the thing that was text until this release.
  • The Trino encoding field offers json, json+lz4 and json+zstd, and both codecs are installed.
  • /ping and /setup at 200, 14 periodic jobs registered, refresh_queries and cleanup_query_results clean on the worker, and 75 data source types still offered.