Skip to content

feat(gcp): Dataplex lakes + zones + assets - #1254

Merged
NitinKumar004 merged 1 commit into
developmentfrom
feat/gcp-dataplex
Sep 7, 2026
Merged

feat(gcp): Dataplex lakes + zones + assets#1254
NitinKumar004 merged 1 commit into
developmentfrom
feat/gcp-dataplex

Conversation

@NitinKumar004

Copy link
Copy Markdown
Collaborator

Summary

New greenfield GCP control-plane service: Dataplex (dataplex.googleapis.com/v1), Track-1 control-plane emulation of the lakes → zones → assets hierarchy. Unmodified google.golang.org/api/dataplex clients, gcloud, and the Terraform google_dataplex_lake / google_dataplex_zone / google_dataplex_asset resources hit the handler unchanged.

Provider tier: all three resources are GA in the stable hashicorp/google provider (verified live against v5.45.2). The e2e drove them at the provider's default /v1/ base path — no /v1beta1/ needed.

What's covered

  • Full CRUD + List across the three levels, every mutation LRO-wrapped and resolved inline (done:true with the resource as a typed Any, exact @type google.cloud.dataplex.v1.{Lake,Zone,Asset}), so a Terraform/SDK LRO wait terminates on the first poll.
  • Hierarchy: a zone create validates its parent lake exists (404 otherwise); an asset create validates its parent lake + zone. Deleting a lake cascades to its zones and their assets; deleting a zone cascades to its assets.
  • Trailing-slash-bounded prefix scans for the cascade, so deleting lake l1 never touches l10's descendants (guard test + live curl proof).
  • Client-named create (?lakeId=/?zoneId=/?assetId=, body-name fallback), 409 on dup, 404 on missing parent, 400 on bad enums (zone type, resourceSpec.locationType, asset resourceSpec.type).
  • updateMask top-level replace; unmasked fields survive.
  • Computed-field stability (the drift point): name, deterministic uid, state=ACTIVE, lake serviceAccount, assetStatus/metastoreStatus/resourceStatus/securityStatus/discoveryStatus, createTime/updateTime — all derived from stable stored inputs and byte-identical across reads. Rich spec blocks (discoverySpec/resourceSpec/metastore) round-trip verbatim as raw passthrough.
  • Shared lro.Registry wiring; Matches yields operations paths to the shared poller (no sibling operations-404 regression). COW memstore with deep clone on read; Snapshottable auto-discovered by the persist completeness test.

Live e2e (Terraform, stable hashicorp/google v5.45.2, default /v1/ path)

  • apply a lake + zone + asset (labels/description/specs) → all created, LRO did not hang, states ACTIVE.
  • plan -detailed-exitcodeexit 0, no drift (on uid/state/service_account/*_status/create_time + the spec blocks).
  • update description + labels → applies in-place; re-plan → exit 0, clean.
  • destroy → all gone (cascade verified).
  • Adversarial l1/l10 prefix-collision cascade (curl): delete l1 → its children 404, l10's children survive 200.

Files

services/dataplex/driver/, providers/gcp/dataplex/, server/gcp/dataplex/; wired into providers/gcp/gcp.go, server/gcp/gcp.go (SetOperationRegistry at the shared LRO registry), server/gcp/from_provider.go; regenerated docs/coverage/ (dataplex-only diff).

Deferrals (tasks, entities, entryGroups, dataScans, environments, IAM verbs, referenced-resource data plane) are recorded out of scope.

@NitinKumar004
NitinKumar004 merged commit a9da977 into development Sep 7, 2026
23 checks passed
@NitinKumar004
NitinKumar004 deleted the feat/gcp-dataplex branch September 7, 2026 05:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant