Releases: xo/dbimp
Release list
v0.6.1
This release fixes two faults of the driver databend of v0.6.0, which usql found and measured. No other driver changes.
What changes for a caller
- Databend. A connection keeps its session for as long as it lives, so
USEandSETstay in effect whendatabase/sqlhands the connection to the next statement, as a pooled connection of MySQL does. Inv0.6.0, the driver reset the session each time the connection went back to the pool, so a caller that runs each statement through*sql.DB, asusqldoes, lost aUSEat once. The driver no longer implementsdriver.SessionResetter(D126). A caller who wants the session of the DSN again opens a new connection. - Databend. After a transaction fails on the server,
CommitandRollbacknow send theROLLBACKthat ends it. Inv0.6.0, the driver refused its ownROLLBACK, and the server kept the transaction aborted until the connection closed.
Other changes
- TDengine gets no driver here. Its REST interface ends a result that failed as valid JSON with
code0, so a driver cannot tell a cut result from a whole one (D127).docs/TDENGINE.mdholds the measurements. Apache Pinot is the next target.
v0.6.0
This release adds the driver databend, for Databend and its SQL. It also gives every driver the same options for one statement, and it adds the key auth to the drivers influxdb and neo4j. A caller of couchbase.Option needs no change.
The Databend driver
Import github.com/xo/dbimp/databend. It registers the name databend, and it takes a DSN of this form:
databend://user:password@host:port/database?key=value
The path is the database, and default without one. The port is 8000 without one, with tls=true too. The keys are tls, auth=basic|bearer, cancel=kill|none and timezone, and the driver refuses any other key, the keys sslmode, tenant and warehouse of databend-go too (D117).
These are the rules that a caller sees:
- The server binds each argument. A positional argument fills each
?in order, andsql.Named("k", v)fills:k. One statement cannot take both kinds. A decimal goes as a string, which keeps every digit, and a[]byteis refused (D120 and D124). - Every value arrives as text, and the driver decodes it by the type of its column:
int64,float64,*apd.Decimal,bool,[]byte,time.Timeandstring. AnArray, aMapand aTuplearrive as text of SQL, and the driver decodes them into[]anyand maps. AGeometryand aGeographyare WKT (D118 and D119). - A
Bitmapcannot be read, because the server writes none of its bytes in JSON. Reading one fails withdbimp.ErrNotSupported(D119). - Each result arrives one page at a time. When the context ends, or the rows close early, the driver kills the query on the server, because the server runs it on when the client leaves (D123).
BeginTxbegins a transaction, which the session of the connection carries. A DDL statement or any error ends it, andCommitthen returns why.USEandSETstay on the connection untildatabase/sqlhands it to another caller (D121 and D122).RowsAffectedis the count that the result names, such asnumber of rows inserted. A statement whose result names none, such asREPLACE INTO, returnsdbimp.ErrNotSupported(D125).
The tests passed against 1.2.881 and 1.2.948 as the administrator and as the ordinary user. docs/DATABEND.md holds what the driver knows about the server, and D117 to D125 in docs/decisions/ hold the reasons.
What changes for a caller
- All drivers. Each driver takes
WithTimeout,WithReadonly,WithParameterandWithDatabase, throughWithOptionsor as an argument of a statement, and an option for each key of its DSN that can change for one statement. An option that the server cannot honor fails the statement withdbimp.ErrNotSupported, and a value that the DSN would refuse fails it withdbimp.ErrInvalidValue.WithParameterreplaces a key that the driver sets itself (D109). - Couchbase.
couchbase.Optionis nowdbimp.Option[...], which stays source compatible.WithDatabasesetsquery_context, asWithQueryContextdoes. - Neo4j.
WithTimeoutsendsmaxExecutionTimein whole seconds, and fails withdbimp.ErrNotSupportedon a release before 2026.04, which ignores it.WithDatabaseandWithCancelare new. The keyauth=bearersends the password as a Bearer token (D116). - InfluxDB.
WithDatabase,WithRetentionPolicy,WithChunkedandWithDescribeare new.WithTimeoutandWithReadonlyfail withdbimp.ErrNotSupported, because no release has either for one request. The keyauth=bearersends the password as a token:Tokento InfluxDB 2, andBearerto 1 and 3 (D116). - ArangoDB.
WithTimeoutsendsmaxRuntime, andWithDatabase,WithBatchandWithCancelare new.WithReadonlyholds only in a read-only transaction. - SurrealDB.
WithDatabaseandWithNamespaceset the database and the namespace of one statement.WithTimeoutandWithReadonlyfail withdbimp.ErrNotSupported.
Other changes
- The root package holds the options of every driver:
Option,WithOptions,Resolve,UnsupportedandMarshalParams. It also holdsSetAuth, which sends the secret of a DSN as the keyauthsays. - The recorder of step 6 follows each page of a query, and escapes a captured value inside a JSON body.
TestEveryDriverTakesTheCommonOptionsholds the options of each driver.
v0.5.0
This release adds the driver arangodb, for ArangoDB 3.12 and its language AQL. It also changes the drivers couchbase, surrealdb, neo4j and influxdb of v0.4.0, and a caller of one of them must read "What changes for a caller" below.
The ArangoDB driver
Import github.com/xo/dbimp/arangodb. It registers the name arangodb, and it takes a DSN of this form:
arangodb://user:password@host:port/database?key=value
The path is the database, and _system without one. The port is 8529 without one. The keys are tls, cancel=tag|none, batch, the size of each batch, and auth=basic|bearer. The secret is always the password of the URL: basic sends it with the user, and bearer sends it as a Bearer token, such as a JWT.
These are the rules that a caller sees:
- A stored document or edge is one column that holds a map. Any other object gives its keys as the columns, and a scalar is one column (D89).
sql.Named("k", v)fills@k, and a positional argument n fills@n. A named argument that the query uses as@@knames a collection. A string or a comment that holds@@kdoes not count (D104).- Each query streams, one batch at a time. When the context ends, or the rows close early, the driver deletes the cursor, or kills the query by a comment that it adds to the query (D90 and D99).
BeginTxbegins a stream transaction that names every collection of the database for write, except the system collections, because ArangoDB needs them before the transaction begins (D91).- The driver takes
CREATEandDROPof collections and indexes, which AQL does not have (D92). A geo index on one field reads GeoJSON (D106).
The tests passed against 3.12.12 as the administrator and as the ordinary user. docs/ARANGODB.md holds what the driver knows about the server, and D89 to D108 in docs/decisions/ hold the reasons.
What changes for a caller
- Neo4j. The comment of
cancel=tagnow ends each statement, on a line of its own, so the position of an error is where the caller wrote it (D95). A rollback now reaches the server after the context ofBeginTxends, so the server frees the locks of the transaction at once (D100). When the rows close before their end, the driver stops a statement that still runs on the server, and in a transaction it reads the rest of the answer, so that the transaction goes on (D105). - InfluxDB. The first column of an InfluxQL result set is
measurement, and notname, soSHOW DATABASESgivesmeasurement | name(D96). - All drivers.
dbimp.ErrIncompletenow means that at least one row of a result set reached the caller before it failed. A failure before the first row, such as a syntax error of Couchbase, no longer wraps it (D107). - SurrealDB.
BeginTxreturnsdbimp.ErrNotSupportedfor every option, and not only for the default ones (D54).
Other changes
- Step 17a of
docs/DRIVER.mdcompares each driver withcouchbasebefore its commit (D97). Two reviews of the whole repository fixed the documents, the comments and the tests that no longer agreed with the code. - Each wire format meets its driver at the row, with a concrete decoder for each format (D108).
- The parser of placeholders in the root package knows the
//comment of AQL,@@name, and a name that starts with a digit.
v0.4.0
This release adds the driver influxdb, for InfluxDB 1, InfluxDB 2, and InfluxDB 3 and later. The drivers couchbase, surrealdb and neo4j do not change.
The InfluxDB driver
Import github.com/xo/dbimp/influxdb. It registers the name influxdb, and it takes a DSN of this form:
influxdb://user:password@host:port/database?key=value
The path is the database. A token is the password, sent with basic authentication. Without a port, the driver uses 8086 for version=1 or version=2, and 8181 otherwise.
The driver speaks two dialects:
influxdbis SQL, on InfluxDB 3 and later.influxqlis InfluxQL, on every release.
The key sqlmode chooses the dialect, as sslmode does for PostgreSQL. Its default, prefer, asks GET /ping for the release, and speaks SQL to InfluxDB 3 and InfluxQL to InfluxDB 1 and 2. With disable or allow, the driver sends no ping, and the key version names the release. Inside sql.Conn.Raw, influxdb.Version returns the release, and influxdb.Dialect returns the dialect of the connection.
These are the rules that a caller sees:
- SQL learns the columns and their types from
DESCRIBE, because the JSON of the server leaves out every NULL. The keydescribe=disableturns that off. A NaN or an infinity from an expression reads asmath.NaN(), because the JSON writes all three alike. - InfluxQL gives each series its own result set. Its first columns are
nameand the tags of the series. CallRows.NextResultSetto read the next one. A statement with no series is a result set with no columns. - The column
timeof InfluxQL is atime.Time. A number in InfluxQL is anint64, auint64above the range ofint64, or afloat64, by its text, so a whole float reads as anint64. INSERT [INTO <database>[.<retention-policy>]] <line protocol>writes line protocol, as theinfluxshell does, in both dialects. Each$nameor$1argument becomes a literal of line protocol. A NULL leaves its tag, its field or its timestamp out of the line.- DELETE goes to the server. InfluxDB 1 and 2 run it. InfluxDB 3 Core has no delete of points, and refuses it.
- The driver asks InfluxDB 1 for chunks, so that a large result streams. The key
chunked=disableturns that off. - InfluxDB has no transactions, so
BeginTxreturnsdbimp.ErrNotSupported.
The tests passed against 1.11.8, 1.13.1, 2.8.0, 2.9.1, 3.9.13, 3.10.6 and 3.11.5, as the administrator, and as the ordinary user on InfluxDB 1 and 2. docs/INFLUXDB.md holds what the driver knows about each release, and decisions D78 to D87 in docs/decisions/ hold the reasons.
For the authors of a driver
dbimptest.RoundTripCasehas three new fields:Column, the column of the value in the select;SkipUpdate, the reason why an update cannot run; and an emptyDelete, for a database that cannot delete one row (D86). Each keeps the old behavior when it is not set.- The recorder in
dbimptest/cmd/recordkeeps a body that the server closed before its end, andReplaycloses it at the same place. A request can name the releases that it runs on, and-ordinarycan be empty for a release that has no ordinary user.
Other changes
- The list of targets drops Gel, Blazegraph and Stargate, because each one is retired (D84). Milvus and PostgREST move to P2 (D87).
- A SurrealDB
RecordIDwrites its text form in JSON (D70). The code was inv0.3.0, and this release accepts the decision.
v0.1.0
The first release of dbimp. It holds one driver, for Couchbase, and the code that every later driver shares.
The Couchbase driver
github.com/xo/dbimp/couchbase is a database/sql driver for the Couchbase query service and SQL++. It replaces github.com/couchbase/go_n1ql and xo/n1ql. It uses the Go standard library and github.com/cockroachdb/apd/v3, and it needs no cgo.
import (
"database/sql"
_ "github.com/xo/dbimp/couchbase"
)
db, err := sql.Open("couchbase", "couchbase://user:pass@localhost:8093/")- It registers one name,
couchbase, and takes only acouchbase://URL. The keys of the URL aretls,query_context,scan_consistency,timeout,durability_levelandtxtimeout. An unknown key or a repeated key is an error. - The server binds each argument.
?,$1andsql.Namedall work. - A row keeps the order of the columns that the server sends, and each value is a Go value:
int64,float64,string,bool,[]any,map[string]any, ornilfor NULL and MISSING. An integer too large for anint64is a*apd.Decimal. - A string scans into a
[]byteas base64, and as its own bytes when it is not base64. - It reads a result one token at a time as it arrives, and it never holds the whole result in memory.
database/sqltransactions map onto the transactions of the query service. On a server with one node, setdurability_level=none, because the default durability cannot commit there. The server ends a transaction after 15 seconds unlesstxtimeoutis longer.WithOptionsand theOptionfunctions set the options of one statement or one transaction.WithParametersets any parameter of the request by its name.- A failed response is a
*couchbase.ResponseError, anderrors.Asfinds eachcouchbase.Errorin it, with its code. - It stops a query when its context ends.
It is tested on Couchbase 7.2.9, 7.6.12 and 8.0.3, as an administrator and as an ordinary user. The tests cover CRUD, each native type, and each feature in testdata/couchbase/features.json. docs/COUCHBASE.md holds every fact that was measured.
What a consumer changes
- The scheme is
couchbase, notn1ql.dburlkeepsn1qlas an alias. - A value is a decoded Go value, not JSON text, so code that called
strconv.Unquoteon a value no longer needs it. - The columns keep the order that the server sends. Couchbase 7.2 sends them in the order of their names.
LastInsertIdreturns an error, andRowsAffectedreturns the count of mutations.
For the authors of drivers
- The root package
dbimpholds the HTTP client, the readers of rows, the decoders of values and the parser of placeholders. dbimptestholds the helpers for the tests of a driver: recorded exchanges, the contract suite, the round trip of a type, and the survey of features.docs/DRIVER.mdholds every step to add a driver, and the gates ingates_test.gohold the steps.