Skip to content

v0.17.0

Choose a tag to compare

View all tags
@kenshaw kenshaw released this 10 Oct 21:44
· 11 commits to main since this release
v0.17.0
348c9eb

This release adds five drivers. Each one speaks the HTTP API of a hosted service with JSON, uses the standard library, and needs no binary encoding.

New drivers

  • databricks reads the SQL Statement Execution API with the format JSON_ARRAY and the disposition INLINE. The DSN is databricks://token:<pat>@host/<warehouse-id>?catalog=&schema=. A result that the server cuts, or that has more than one chunk, is an error. The server gives timestamps with 3 fraction digits.
  • bigquery reads the REST API of BigQuery. The DSN is bigquery://project/dataset?credential_file=/path/key.json. It supports the hosted service and the community emulator bigquery-0.8.1. A JSON null and a SQL NULL are both nil, and a STRUCT is a map[string]any.
  • athena reads the JSON API of Athena and signs each request with Signature Version 4. The DSN is athena://key:secret@athena.<region>.amazonaws.com/<database>?workgroup=&output=s3://.... It runs one statement for each request, and it has no transactions.
  • cosmos reads documents from one container of Azure Cosmos DB over its REST API. The DSN is cosmos://x:<key>@host/<db>/<container>, and the master key is the password. It reads only: Exec returns an error. A query that the gateway cannot serve across partitions returns its HTTP 400, so a caller names the partition key. It also answers read-only catalog statements, such as SELECT * FROM "$containers" WHERE database = 'db', which dbmeta uses.
  • spanner reads and writes through the REST API of Cloud Spanner, and it never speaks gRPC. The DSN is spanner:///project/instance/database?credential_file=/path/key.json, and an emulator form is spanner://localhost:9020/p/i/d (the driver reads the emulator of Google, and the first live run on the dbmeta emulator comes next). It reads every result with executeStreamingSql, and it runs DDL as a long-running operation.

What a consumer must do

dburl points the schemes databricks, bigquery, athena, cosmos and spanner at these packages. usql can replace the heavier clients that it imports today. dbmeta reads the catalog of Cosmos DB through the statements of the cosmos driver.

Other changes

  • The recorder gains "auth": "none", "auth": "cosmos" and the flag -insecure. A run for one release no longer deletes the files of another release that has the same prefix.
  • Rule 4 of AGENTS.md names the BigQuery query, the Cosmos DB query, the Athena query and the Spanner transaction.
  • Decisions D184 to D195 record the targets and the answers for each driver.

Tests

CI passed all 45 jobs on the tagged commit. The tests of large results now measure the growth of the heap with dbimptest.HeapGauge, so they no longer fail when many packages run at once.

The integration tests of all five drivers passed on the live services: a free Databricks workspace, a BigQuery project and its emulator, an Athena account with a Lambda connector, a Cosmos DB account and its emulator, and a Spanner free instance. The hosted drivers run no integration test in CI, because CI holds no secret for them. CI runs BigQuery and Cosmos DB on their emulators.

Assets 2
Loading