Databricks SQL Driver for Go

A database/sql driver for Databricks SQL. It connects to Databricks SQL Warehouses and clusters and runs queries through Go's standard database/sql interface.

• Quick start
• Cloning the repository
• Choosing a backend (Thrift vs SEA/kernel)
• Building
• Connecting
  • DSN (Data Source Name)
  • Connector object
• Connection properties
• Authentication
• Cloud Fetch
• TLS
• Proxy
• Data types
• Telemetry
• Examples
• Develop

import (
  "context"
  "database/sql"
  _ "github.com/databricks/databricks-sql-go"
)

db, err := sql.Open("databricks", "token:********@********.databricks.com:443/sql/1.0/endpoints/********")
if err != nil {
  panic(err)
}
defer db.Close()

rows, err := db.QueryContext(context.Background(), "SELECT 1")
defer rows.Close()

See doc.go for full package documentation or the Databricks documentation for the SQL Driver for Go.

Using the driver in your own project? Add it with go get github.com/databricks/databricks-sql-go and go build — you never clone this repo. A default Thrift build pulls no kernel binaries. The "Cloning" section below is only for contributors working in this repo directly.

This repo is small: for the SEA/kernel backend it commits only the C header (internal/backend/kernel/include/databricks_kernel.h). The prebuilt kernel archives (per-platform libdatabricks_sql_kernel.a, ~60–95 MB each) live in a separate repo, databricks-sql-kernel-bindings, one Go module per platform. The driver requires them, so go get pulls only your platform's archive — no build step (see Building).

A plain git clone of this repo is cheap. Sparse-clone the bindings repo instead if its committed archives make a full clone heavy:

git clone --filter=blob:none --sparse https://github.com/databricks/databricks-sql-kernel-bindings
cd databricks-sql-kernel-bindings
git sparse-checkout set --no-cone '/*' '!/lib' 'lib/darwin_arm64'   # keep only your platform

The driver has two execution backends, selected once per connection:

Thrift is the default and needs no special setup. Select the SEA/kernel backend per connection either way:

• Connector option: dbsql.WithUseKernel(true)
• DSN parameter: useKernel=true

If the binary was not built with the databricks_kernel tag, selecting the kernel backend returns an error wrapping dbsqlerr.ErrKernelNotCompiled at connect — it never silently falls back to Thrift.

Parameter parity. Parameters are intended to behave identically on both backends. Where a backend can't honor an option it is rejected at connect or execute (wrapping dbsqlerr.ErrNotSupportedByKernel), not silently ignored. The Connection properties Protocol column records, per parameter, whether it applies to Both, Thrift-only, or SEA-only.

The two backends differ at build time, not just at connect.

Pure Go, CGO_ENABLED=0, go get-able, cross-compilable to any GOOS/GOARCH. No C, no Rust, no linked native library.

go build ./...
go test  ./...

# Repo Makefile equivalents (both CGO_ENABLED=0):
make build      # multi-arch pure-Go binaries (linux + darwin)
make test       # pure-Go unit tests

Cross-compiling is free, e.g. GOOS=windows GOARCH=amd64 go build ./....

The kernel backend compiles in only under the databricks_kernel build tag with CGO_ENABLED=1. The prebuilt libdatabricks_sql_kernel.a for your platform is pulled automatically as a Go module dependency and linked — no Rust toolchain, no build step. You need only a C toolchain (cgo) on a supported platform: linux amd64/arm64/arm, darwin amd64/arm64, windows amd64/arm64.

CGO_ENABLED=1 go build -tags databricks_kernel ./...

Carry both flags — dropping either produces a pure-Go binary where WithUseKernel(true) fails at connect. Then select the backend per connection with WithUseKernel(true) (or useKernel=true in the DSN).

To build the archive from the pinned kernel revision instead of using the prebuilt module — for kernel development or an unsupported platform — you need a Rust toolchain (cargo, pinned in rust-toolchain.toml) and network access to clone databricks-sql-kernel at KERNEL_REV:

make kernel-lib     # clone + cargo-build the pinned archive into the cgo link dir
make build-kernel   # == CGO_ENABLED=1 go build -tags databricks_kernel ./...
make test-kernel    # kernel-tagged unit tests

Stage a prebuilt archive without Rust via make kernel-lib KERNEL_LOCAL_A=/path/to/libdatabricks_sql_kernel.a.

token:[your token]@[Workspace hostname]:[Port number][Endpoint HTTP Path]?param=value&param=value

The token:[your token]@ prefix authenticates with a personal access token (PAT). For other authentication types, omit the prefix and use the authType, clientID/clientSecret, or accessToken parameters — see Authentication.

db, err := sql.Open("databricks",
  "token:<pat>@<host>:443/sql/1.0/warehouses/<id>?timeout=1000&maxRows=1000")

To use the SEA/kernel backend, append useKernel=true (and, optionally, warehouseId=<id>):

token:<pat>@<host>:443/sql/1.0/warehouses/<id>?useKernel=true

You can also connect with a connector object built from functional options:

import (
  "database/sql"
  dbsql "github.com/databricks/databricks-sql-go"
)

connector, err := dbsql.NewConnector(
  dbsql.WithServerHostname(<host>),
  dbsql.WithPort(<port>),
  dbsql.WithHTTPPath(<http path>),
  dbsql.WithAccessToken(<your token>),
  // dbsql.WithUseKernel(true), // opt into the SEA/kernel backend
)
if err != nil {
  log.Fatal(err)
}
db := sql.OpenDB(connector)
defer db.Close()

See doc.go or connector.go for the full set of functional options.

See CONNECTION_PARAMETERS.md for every connection, session, and per-statement parameter the driver accepts, and whether each one applies to the Thrift backend (default), the SEA/kernel backend, or both. The sections below cover the same parameters inline.

Optional DSN parameters are appended as ?param=value&param=value; the equivalent connector options are listed alongside. The Protocol column shows applicability:

• Both — honored identically on Thrift and SEA/kernel.
• Thrift only — honored on Thrift; rejected at connect on the kernel path (wraps ErrNotSupportedByKernel) unless noted "inert" (accepted, no effect).
• SEA only — kernel path only. The experimental WithKernel* options are rejected on Thrift when set without WithUseKernel(true) (wraps ErrRequiresKernelBackend); warehouseId is the exception — Thrift silently ignores it (see its row).

Any parameter not recognized below (e.g. ansi_mode, timezone) is passed through as a session parameter on both backends.

See Cloud Fetch, TLS, and Proxy for the remaining groups. Telemetry parameters are covered under Telemetry.

PAT (default): supply token:<pat>@… in the DSN, or WithAccessToken.

OAuth M2M: leave the token:...@ prefix off and pass the service principal's clientID and clientSecret:

[host]:443[path]?authType=OauthM2M&clientID=<id>&clientSecret=<secret>

authType=OauthM2M is optional — supplying clientID + clientSecret selects M2M.

OAuth U2M (interactive browser login):

[host]:443[path]?authType=OauthU2M

Notes for the SEA/kernel backend:

• The kernel snapshots one WithFederatedTokenProvider* token during setup; AndClientID also forwards the SP-wide client ID. Expired tokens require a new connection.
• Custom OAuth M2M scopes are rejected on the kernel path (the kernel applies its own default scopes). Default scopes work on both.
• U2M is interactive: on a cache miss, connecting launches the browser and a connect-context deadline is not honored during the login window. U2M scopes are at parity with Thrift. Use PAT or M2M for headless/deadline-bound connects.
• Custom token-provider / external / static authenticators are Thrift only.
• OAuth token caching/refresh is owned by the kernel on the kernel path (no driver config).

Cloud Fetch increases performance of extracting large results by fetching data in parallel via cloud storage (more info).

On the Thrift backend:

token:<pat>@<host>:443[path]?useCloudFetch=true&maxDownloadThreads=3
# or disable it entirely:
token:<pat>@<host>:443[path]?useCloudFetch=false

The Thrift and kernel backends both honor the standard HTTP_PROXY / HTTPS_PROXY / NO_PROXY environment variables.

Results render byte-for-byte identically on both backends. Scalars, DECIMAL (exact string), TIMESTAMP / TIMESTAMP_NTZ (shifted into the session time zone), INTERVAL, nested ARRAY / MAP / STRUCT and VARIANT (as JSON), and GEOMETRY / GEOGRAPHY (as WKT) are all supported. BINARY is returned as sql.RawBytes.

Metadata is reached through SQL (SHOW, DESCRIBE, information_schema) on both backends — the driver exposes no GetCatalogs/GetSchemas/GetTables/GetColumns API (a database/sql limitation, not backend-specific).

The driver includes optional telemetry to help improve performance and reliability. Go wrapper telemetry applies to the default Thrift backend. When enableTelemetry is left unset (the default), a server-side feature flag decides whether wrapper telemetry is active — so it may be enabled without an explicit opt-in. Setting enableTelemetry explicitly overrides the flag. On the kernel backend, the Go wrapper skips its telemetry interceptor entirely so it does not duplicate kernel-owned telemetry; enableTelemetry and telemetry_batch_size are forwarded into the kernel telemetry config instead.

# force on (regardless of the server flag):
token:<pat>@<host>:443[path]?enableTelemetry=true
# force off:
token:<pat>@<host>:443[path]?enableTelemetry=false

Collected: query latency/performance, error codes (not messages), feature usage, driver version/environment. Not collected: SQL text, query results/values, table/column names, user identities or credentials. Telemetry has < 1% overhead and is protected by a circuit breaker. The kernel path additionally emits a connection-config telemetry event at connect (mode, auth mechanism/flow, proxy, arrow, query tags, metric-view); the Thrift path's telemetry is unchanged. See telemetry/DESIGN.md.

Runnable examples live in examples/. Notable ones:

• examples/workflow — end-to-end connector setup, logging, query.
• examples/oauth — OAuth U2M and M2M.
• examples/parameters — bound query parameters.
• examples/cloudfetch — large results via Cloud Fetch.
• examples/kernel — the SEA/kernel backend (requires a -tags databricks_kernel, CGO_ENABLED=1 build; see Building).

We use golangci-lint. In VS Code:

{
  "go.lintTool": "golangci-lint",
  "go.lintFlags": ["--fast"]
}

go test           # default (Thrift) backend, pure Go
make test-kernel  # kernel-tagged unit tests (links the prebuilt bindings; no Rust)

If you find any issues, feel free to create an issue or send a pull request directly.

See CONTRIBUTING.md.

Apache 2.0