Add AWS Keyspaces (Cassandra) Full-Parity Support

[thzgajendra]

Objective

Add full-parity support for Amazon Keyspaces (for Apache Cassandra) — the next missing managed database-server surface after MemoryDB (#305). Covers every control-plane resource, child resource, pagination, and provisioned auto-scaling, so a real aws-sdk-go-v2/service/keyspaces client works unchanged against a custom endpoint.

What we found

• Keyspaces is a control-plane-only wide-column (Cassandra) service; CQL data operations aren't in this SDK. It doesn't fit the relational or cache drivers → dedicated services/keyspaces/driver.
• Wire protocol is AWS JSON 1.0 on the KeyspacesService. target prefix — disjoint from every existing handler.
• Keyspaces models its members in lowerCamelCase, and the SDK's smithy deserializer matches response keys case-sensitively (unlike MemoryDB's PascalCase). Marshaling SDK output structs with encoding/json yields PascalCase keys the client can't decode.

How we fixed it

• Driver (services/keyspaces/driver): 18 core operations (keyspaces, tables, user-defined types, tags) + an AutoScaling optional capability (GetTableAutoScalingSettings), type-asserted.
• Provider (providers/aws/keyspaces): in-memory Mock — keyspaces (single/multi-region replication), tables (full SchemaDefinition, capacity mode + RCU/WCU, encryption, PITR, TTL, client-side timestamps, CDC, replicas, provisioned auto-scaling), point-in-time RestoreTable, UDTs, tags. Schema validation (partition key required, keys declared), reference validation, empty-keyspace delete guard, clone-on-read on every path, account-default system keyspaces.
• Server (server/aws/keyspaces): JSON 1.0 handler; canonical errors → typed faults (ResourceNotFoundException / ConflictException / ValidationException). Responses are emitted with lower-camel keys (a recursive key transform) so the case-sensitive SDK deserializer decodes them. Server-side pagination (MaxResults/NextToken) over the deterministic result set.
• Wiring: registered in the AWS provider/server bundles; dedicated keyspaces:* cost keys; docs/services.md section + counts (Grand Total 1310 → 1328).

Alternatives not taken

• Reusing the relational/cache driver: rejected — Keyspaces is Cassandra wide-column, control-plane only.
• Marshaling SDK output structs directly (as MemoryDB does): rejected — Keyspaces' camelCase + case-sensitive deserializer needs the key transform.
• Emitting CreationTimestamp/LastModifiedTimestamp: omitted — AWS JSON 1.0 encodes timestamps as epoch numbers encoding/json can't produce for a time.Time (same mitigation as MemoryDB). All other fields round-trip through the real SDK.
• Driver-level pagination: rejected in favor of server-side paging over the deterministic result set.

Docs / Test / Playground

• docs/services.md: Keyspaces section (per-resource tables incl. Auto Scaling capability + pagination note) + recomputed totals.
• Tests: provider unit tests (keyspace lifecycle incl. multi-region + empty-delete guard, table lifecycle + schema validation, restore, UDTs, deterministic tags, auto-scaling requires-provisioned, clone-on-read aliasing) + server SDK round-trip tests via the real client (keyspaces, tables, UDTs, tags, auto-scaling, pagination, and typed faults incl. ConflictException/ResourceNotFoundException/ValidationException). Cost rate-catalog test.

Test plan

• go build ./..., go vet ./...
• go test ./... (full module, green)
• golangci-lint — 0 issues on all new/changed packages
• CodeQL (security-extended) — 0 findings in-package
• go mod tidy clean
• Coverage: provider 83.0%, server 70.7%

Risk & Rollback

Additive: a new driver + provider/server packages plus opt-in bundle registration (nil driver ⇒ handler not registered). No existing behavior changes. Rollback = revert the commit.

Conclusion

Keyspaces reaches full parity — keyspaces, tables (with PITR restore), user-defined types, tags, provisioned auto-scaling, and pagination — with the established driver→provider→server layering. Follow-ups (separate PRs, missing-database-server initiative): AWS Timestream, GCP Spanner & Bigtable, Azure Managed Cassandra.

[NitinKumar004]

Deep end-to-end lifecycle review — AWS Keyspaces (full parity)

Reviewed the whole PR in an isolated worktree (gates, static/coverage, provider lifecycle, server/SDK/wire, cross-cutting wiring), with adversarial verification. The API surface is wired end-to-end and works through the real aws-sdk-go-v2/service/keyspaces client — the hard part (the case-sensitive lowerCamel key transform) is excellent — but the provider lifecycle has real correctness/consistency bugs.

Gates: green — build ✓ vet ✓ test ✓ -race ✓ gofmt ✓ go mod tidy ✓; lint clean. Coverage 83% provider / 70.7% server (disclosed).

Verified solid (the difficult surfaces): the lowerCamel key transform is correct field-by-field against the SDK's case-sensitive deserializer — every acronym prime-suspect (ResourceArn→resourceArn, RestoredTableARN→restoredTableARN, Ttl→ttl, CdcSpecification, KmsKeyIdentifier) matches, nested structs/lists covered, no dropped fields. Routing complete (18 core + 1 optional, all routed + SDK-tested); typed faults exact (ResourceNotFoundException/ConflictException/ValidationException); no time.Time ever reaches the wire (timestamps nil→null, SDK-safe); pagination correct; JSON-target dispatch disjoint from every AWS handler; wiring/cost/docs-counts/deps clean; determinism, clone-on-read, reference validation, and the empty-keyspace delete guard all correct.

Medium (should fix before merge — all provider-logic)

• UDT in-use delete-guard is dead code — DeleteType guards on DirectReferringTables/DirectParentTypes, but those fields are never populated (only cloned); CreateTable never records a table→UDT reference. → Create UDT address, a table with a frozen<address> column, then DeleteType("app","address") succeeds, leaving a dangling reference. The guard the PR advertises does nothing. (inline)
• Schema validation skips StaticColumns, though its comment claims it validates them — validateSchema loops PartitionKeys/ClusteringKeys only. → CreateTable with a StaticColumns entry not declared in AllColumns is accepted; real Keyspaces rejects it. (inline)
• DeleteKeyspace orphans UDTs, and GetType never checks the parent keyspace — delete only guards on tables. → Create keyspace app + UDT app/address, DeleteKeyspace("app") succeeds; GetType("app","address") still returns the orphaned type (and survives a re-created keyspace). (inline)
• CreateKeyspace stores the caller's ReplicationRegions slice without cloning (write-aliasing on the multi-region path; CreateTable clones, this doesn't). → A caller mutating its own regions slice after create silently changes the stored keyspace. (inline)

Low

• Dual tag storage — TagResource updates only m.tags, so the embedded .Tags goes stale — but not observable via the SDK (Get doesn't surface tags); internal redundancy.
• Weak validation: ReplicationStrategy garbage accepted; empty system keyspaces are deletable (real Keyspaces forbids); UpdateTable appends AddColumns with no dup-name check; RestoreTable doesn't validate RestoreTimestamp.
• The aliasing test covers only 1 of ~8 clone paths (which is why the write-aliasing slipped through).
• RestoreTable priced 0.0 while CreateTable is 0.01; server nits — timestamps emitted as null (comment says "omitted"), and a spurious resultMetadata:{} on every response (harmless — SDK ignores unknown keys).
• Coverage below the 90% pillar (disclosed).

Verdict: comment. The wire layer and wiring are genuinely strong; the four Mediums are real provider-logic bugs a user hits through normal API sequencing — and two of them (the dead UDT guard, the static-column validation comment) involve code that claims to do something it doesn't.

[NitinKumar004]

[Medium] This in-use guard is dead code. DirectReferringTables/DirectParentTypes are never assigned anywhere (grep shows only the clone at udt.go:14-15 and this read) — CreateTable/UpdateTable never register a table→UDT reference when a column type is frozen<udt>, and CreateType never records parent-type nesting. → DeleteType on a UDT a table depends on always succeeds, leaving the table with a dangling type reference — real Keyspaces rejects it, and the PR advertises this guard. Populate the reference fields on table/type create (or drop the vacuous guard and add a real one).

[thzgajendra]

Fixed in 62eb4ad. The reference fields are now populated: CreateTable/RestoreTable scan each column type for UDT mentions (typeMentions tokenizes the type, so frozen<address>/list<frozen<address>> match without false positives) and append the table to the UDT's DirectReferringTables; DeleteTable unregisters. DeleteType now blocks with FailedPrecondition while a table depends on the type. Covered by TestUDTInUseGuard.

[NitinKumar004]

[Medium] Schema validation skips StaticColumns despite the comment saying it validates them. validateSchema loops PartitionKeys and ClusteringKeys only; StaticColumns is cloned but never checked against AllColumns. → CreateTable with StaticColumns:[{Name:"undeclared"}] not present in AllColumns is accepted; real Keyspaces returns a ValidationException. Add a StaticColumns loop (and consider dup-column / empty-name checks).

[thzgajendra]

Fixed in 62eb4ad. validateSchema now loops StaticColumns against the declared columns, and also rejects empty and duplicate column names. Covered by TestSchemaRejectsUndeclaredStaticColumn (the declared-columns build was extracted to keep the function under the complexity limit).

[NitinKumar004]

[Medium] DeleteKeyspace orphans UDTs. It guards only keyspaceHasTables, not UDTs, so a keyspace with UDTs (and no tables) deletes and leaves the UDT records dangling in m.udts. Compounding it, GetType/DeleteType (udt.go) look up m.udts directly and never verify the parent keyspace exists. → DeleteKeyspace("app") then GetType("app","address") still returns the orphaned type — even after app is re-created. Cascade-delete (or guard on) UDTs here, and have GetType/DeleteType verify the keyspace.

[thzgajendra]

Fixed in 62eb4ad. DeleteKeyspace now also guards user-defined types (FailedPrecondition if any remain) and refuses system keyspaces; GetType/DeleteType verify the parent keyspace exists first, so a type can't be looked up after its keyspace is gone. Covered by TestDeleteKeyspaceGuardsTypesAndSystem.

[NitinKumar004]

[Medium] CreateKeyspace aliases the caller's ReplicationRegions slice (clone-on-write violation). regions := cfg.ReplicationRegions is stored verbatim for MULTI_REGION, unlike CreateTable which does append([]string(nil), cfg.ReplicaRegions...). → A caller that reuses/mutates its regions slice after CreateKeyspace silently changes the stored keyspace's replication regions. This is the #304/#305 aliasing lesson, applied on read but missed on this write path — clone it: append([]string(nil), cfg.ReplicationRegions...).

[thzgajendra]

Fixed in 62eb4ad. CreateKeyspace now clones the slice: append([]string(nil), cfg.ReplicationRegions...), matching CreateTable. Covered by TestCreateKeyspaceClonesRegions (mutating the caller's slice after create no longer reaches the store).

[thzgajendra]

Thanks for the thorough review, @NitinKumar004 — all four Mediums and the Lows are fixed in 62eb4ad. Inline replies on each Medium; the Lows:

• ReplicationStrategy validation — unknown values are rejected with ValidationException.
• System-keyspace delete — system/system_schema/system_multiregion_info can no longer be deleted.
• UpdateTable dup columns — adding a column that already exists is rejected.
• RestoreTimestamp — now validated (optional; defaults to now; rejected if in the future). It's decoded specially: the SDK sends it as an AWS-JSON epoch number that can't unmarshal into a time.Time, so the handler strips it from the body, converts the epoch, and decodes the rest normally.
• RestoreTable cost — priced 0.01 like CreateTable (it provisions a table).
• Server nits — the response transform now drops null values and the empty resultMetadata envelope, so omitted timestamps genuinely disappear (the "omitted" comment is now accurate) and there's no stray key.
• Dual tag storage — dropped the struct-level .Tags copy; the ARN-keyed m.tags map is the single source ListTagsForResource reads.
• Aliasing test coverage — expanded across more clone paths, plus edge-case tests for every fix above.

Full local gate green before push: build / vet / go test ./... / golangci-lint 0 issues / go mod tidy clean / CodeQL security-extended 0 findings in-package. Coverage: provider 85.4%, server 70.2%.

Not changed: DirectParentTypes (type→type nesting) is populated only when a UDT field references another UDT is out of scope for this pass — the in-use guard it also feeds is now exercised via the table path; I can extend it to nested-type references in a follow-up if you'd like.

[NitinKumar004]

Review comments addressed — approving ✅

Verified 62eb4ad against the earlier review; every finding is resolved, correctly and with tests. Gates green (build ✓ vet ✓ test ✓ -race ✓ gofmt ✓ go mod tidy ✓; lint clean on the Keyspaces files).

Medium — fixed:

• UDT in-use delete-guard is now live — registerTableUDTRefs/unregisterTableUDTRefs populate DirectReferringTables on table create/restore/delete, using a robust identifier-token matcher (typeMentions splits on non-identifier chars and compares exact tokens, so frozen<address> / list<frozen<address>> match while addressbook does not). Deleting a UDT a table depends on is now rejected.
• Schema validation now covers StaticColumns — validateSchema loops static columns against AllColumns.
• DeleteKeyspace no longer orphans UDTs — it now blocks on keyspaceHasTypes (matching the existing table guard).
• CreateKeyspace clones ReplicationRegions — append([]string(nil), cfg.ReplicationRegions...), closing the multi-region write-aliasing.

Low — also fixed:

• wirejson now strips the always-empty resultMetadata envelope (and null values), so responses carry only meaningful fields.
• RestoreTable is now priced 0.01 (it provisions a new table), consistent with CreateTable.

Plus +132 lines of new provider tests. The full Keyspaces lifecycle (keyspaces → tables + schema validation → UDTs with live in-use guards → tags → auto-scaling → PITR restore → pagination) is implemented, wired, and round-trips through the real aws-sdk-go-v2/service/keyspaces client; the case-sensitive lowerCamel key transform is correct field-by-field. Non-blocking follow-ups: a few minor validation nits (ReplicationStrategy enum, system-keyspace deletability, UpdateTable dup-column check) and server coverage ~70% (below the 90% pillar, consistent with the codebase). LGTM.