Repository navigation
v1.7.8
Makes the Terraform claim testable, then fixes what the test found.
pip install minilake==1.7.8
docker pull ghcr.io/dmux/minilake:1.7.8Coverage: 208 → 267 of the 1168 endpoints the databricks-sdk calls. CLI command
groups working end to end: 21 → 30.
The measure changed, and it matters
That raw figure is 22.9%, and the roadmap used to quote it without qualification. It is
the wrong number. 604 of the missing endpoints are in blocks this project has
deliberately decided not to emulate — MLflow, the Account API, identity v2,
marketplace, Genie, Lakebase, vector search, serving, clean rooms, apps.
Against the reachable surface, coverage is 267 of 564 — about 47%. That is what
docs/EMULATION_ROADMAP.md
now leads with. Counting deliberate scope cuts as gaps makes progress look permanently
hopeless and hides which gaps are real.
Terraform is now tested, not asserted
docs/terraform.md promised the Databricks provider worked against minilake. Nothing
verified it — there was no Terraform test in the repo at all, and the services added in
1.7.7 (SCIM, tokens, cluster policies, instance pools) were specifically
Terraform-facing resources with no way to check them.
tests/terraform/ now drives the real provider through apply → plan → destroy
over a 15-resource config and asserts the second plan is empty. terraform and a pinned
provider mirror are baked into the test image, so the suite still needs no network.
Its first run found three defects that every SDK-driven test in the suite had missed,
because they live in behaviour only the provider exercises:
| Defect | Why no SDK test caught it |
|---|---|
The provider posts "entitlements": [{}] for a group with none; minilake echoed the empty entry back and broke the provider on its own input (Invalid address to set: []string{""}) |
The SDK never sends [{}]. Real Databricks drops valueless entries from multi-valued SCIM attributes; now so does minilake |
SQL warehouses never reported auto_stop_mins, enable_photon, max_num_clusters or odbc_params, so every terraform plan showed a permanent in-place update |
The SDK reads back only what it asked about; Terraform diffs the whole object |
SCIM ignored the attributes projection the provider requests |
The SDK does not send it |
Permanent drift is worse than a 501: it looks like it works until someone runs plan
twice. The empty-plan assertion is what catches it.
New endpoints
- Unity Catalog grants —
w.grants.*and Terraform'sdatabricks_grants, with real
inheritance: a grant on a catalog is effective on its schemas and their tables, and
each effective privilege is tagged with the ancestor it came from. Without that,
get_effectivewould just begetunder another name. - The workspace-admin APIs — git credentials, IP access lists, global init scripts,
notification destinations, instance profiles andworkspace-conf. Each is trivial
alone; together they are what a realisticterraform applytouches before it reaches
a catalog, where a single 501 fails the whole plan. - Legacy
/api/2.0/preview/sql/*— queries, alerts, dashboards, widgets,
visualizations and data sources, as adapters over the modern stores rather than a
second copy. An object created through either surface is visible and editable from the
other.
Three details that are load-bearing
The legacy list endpoints must paginate. The SDK walks them by incrementing page
until a response comes back empty, so a handler that ignores the parameter never
terminates. The symptom is a hung client, not an error.
DELETE trashes and POST .../trash/{id} restores, not the other way round.
Implemented backwards, a client asking to restore would destroy.
securable_type arrives as SecurableType.CATALOG, not CATALOG. The SDK's enum is
not a str subclass, so its f-string interpolation puts the repr on the wire. Both forms
are accepted; without that, grants stored under an unrecognised key and inheritance
silently found nothing.
What this does not give you
Nothing added here is enforced. Grants, secret ACLs, IP access lists and cluster policies
are stored and read back faithfully and checked nowhere; no IP is blocked, no init script
runs, no notification is sent. A terraform plan that applies cleanly against minilake
says the shape of your config is right, not that its access control is. docs/terraform.md
now states this plainly.
Also fixed
scripts/cli_coverage.py was scoring all 8 Files API endpoints as missing — they are
implemented and tested. The SDK writes /api/2.0/fs/files{path} with the parameter
carrying its own leading slash, and the normalizer produced a path matching no route.
Those two paths are the only ones in the SDK shaped that way.
Testing
378 passing, up from 338. Running pytest on a host without terraform installed
skips the conformance tests rather than failing.
Per-release detail lives in CHANGELOG.md.
What's Changed
- Terraform conformance test, UC grants, workspace-admin belt, legacy SQL by @dmux in #19
- chore(release): 1.7.8 — Terraform conformance, grants, admin belt by @dmux in #20
Full Changelog: v1.7.7...v1.7.8
- PyPI: https://pypi.org/project/minilake/1.7.8/
- Container image:
ghcr.io/dmux/minilake:1.7.8