Skip to content

v1.7.7

Choose a tag to compare

@github-actions github-actions released this 15 Sep 12:49
· 15 commits to main since this release
50ea1ec

Expands API emulation coverage from 13.3% to 17.8% of the endpoints the
databricks-sdk actually calls (155 → 208), and adds the tooling that measures it.
Command groups of the databricks CLI that now work end to end: 10 → 21.

No behaviour changes for existing SQL, Unity Catalog or Jobs workloads.

pip install minilake==1.7.7
docker pull ghcr.io/dmux/minilake:1.7.7

Coverage is now measured, not asserted

scripts/cli_coverage.py points the real databricks CLI at a disposable minilake and
probes every (method, path) pair the installed SDK calls, recording which fall through
to the catchall 501. The unit is the method+path pair rather than the path: probing
everything with GET scores POST-only endpoints as missing, because FastAPI falls through
when no method matches — that distinction alone moved the measured baseline from 10.0% to
13.3%.

It produces docs/CLI_COVERAGE.md
(generated) and docs/EMULATION_ROADMAP.md
(curated — what is worth building next, with measured gap counts). The CLI is pinned into
Dockerfile.test so numbers stay comparable between runs.

Things that already had an engine and only lacked an endpoint

  • sql_task.query now executes for real. It was SKIPPED. Zero new endpoints were
    needed — the Queries API had existed all along, and the "no Queries API" note it was
    skipped for was simply stale.
  • POST /api/2.2/jobs/runs/submit — one-shot runs that carry their tasks inline and
    belong to no job, the path databricks bundle run and most CI code take. Reuses the
    existing DAG scheduler, and honours idempotency_token so a repeated submit returns
    the original run rather than starting a second one.
  • Alerts (/api/2.0/sql/alerts) and sql_task.alert, which evaluates the
    condition against real query results rather than reporting a stored state.

Compatibility walls removed

  • Unity Catalog metastore reads — current-metastore-assignment, metastore_summary,
    and metastores list/get. Small, but load-bearing: several clients call the assignment
    during setup and previously gave up at the 501 before ever reaching a catalog that
    worked perfectly well.
  • SCIM Users, Groups and Service Principals (18 endpoints) — CRUD, the PATCH forms
    Terraform sends for group membership (including members[value eq "id"]), and
    attribute eq "value" filtering. The current user is seeded so Me and Users never
    disagree about who is signed in.
  • Personal access tokens, secret scope ACLs, and clusters update/pin/unpin.
    update is a partial edit honouring update_mask, so changing one setting no longer
    blanks the rest of the spec the way a full edit would.

New functionality

  • Unity Catalog functions. A SQL function is created as a real DuckDB MACRO, so a
    function registered through the UC API is then callable from the Statement Execution
    API by its three-part name — SELECT cat.sch.add_tax(100) actually computes. Not
    metadata-only.
  • Cluster policies and instance pools. Neither is enforced — there is no compute here
    for a policy to constrain, and pool stats are permanently zero — but both resolve, so a
    bundle or Terraform config naming one deploys instead of hitting a 501.

Behaviour change worth knowing

Clusters now retain and validate policy_id and instance_pool_id.
CreateClusterRequest previously dropped both via extra="ignore". Now that policies and
pools are real resources, a cluster that forgot the policy it was created with would read
as permanent drift in Terraform. A cluster naming a policy or pool that does not exist is
rejected, and one in use cannot be deleted.

What this does not give you

Identities, tokens and ACLs are records, not credentials. minilake accepts any bearer
token by design, so creating a user makes no way to sign in, and revoking a token locks
nobody out. They exist because tooling creates them as a setup step and stops when that
step fails — not because the values mean anything.

Likewise, secret ACLs and cluster policies are stored and read back faithfully but never
enforced. A test that passes here still says nothing about grants in a real workspace.

Also in this release

  • Docs corrected: README.md and the published site had both described Identity as a
    static current-user endpoint and listed secret ACLs as unimplemented. Neither was true
    any more.
  • next bumped to 16.3.3 in the web UI (#16).

Testing

338 tests passing, up from 267 — 71 new, all driven through the real databricks-sdk
against the Dockerized stack, per the project's rule that the official client is the
source of truth.

Per-release detail lives in CHANGELOG.md.


What's Changed

  • Expand Databricks API emulation: 13.3% -> 17.8% coverage by @dmux in #17
  • build(deps): bump next from 16.3.2 to 16.3.3 in /ui by @dependabot[bot] in #16
  • chore(release): 1.7.7 — docs sync and version bump by @dmux in #18

New Contributors

Full Changelog: v1.7.6...v1.7.7