Skip to content

v2.0.1

Choose a tag to compare

@NitinKumar004 NitinKumar004 released this 19 Jul 16:02
· 1020 commits to master since this release
ca5832d

cloudemu Release Notes — v2.0.1

🔧 Enhancements

Pagination correctness

All three NoSQL drivers paged over a randomized map snapshot with offset tokens, so multi-page Query/Scan duplicated and dropped items. Results now get a stable type-aware ordering (numbers numerically — no more "10" < "9"), the invariant is enforced centrally by pagination.PaginateSorted, and a malformed page token returns InvalidArgument on every driver instead of an empty 200 page that clients read as end-of-results.

DynamoDB key-based continuation

ExclusiveStartKey / LastEvaluatedKey were not implemented — SDK clients could never page past page 1. Continuation now lives in the driver: per-page cost is O(page) instead of O(table), ScanIndexForward:false pages descending, GSI queries order by the index keys and return the real LastEvaluatedKey shape (index + base keys), and a stale or unknown start key is a ValidationException rather than a silent restart that re-serves consumed items.

Conditional writes & document merges

DynamoDB PutItem honors ConditionExpression attribute_exists / attribute_not_exists — including #alias names via ExpressionAttributeNames — with ConditionalCheckFailedException. Firestore :commit enforces currentDocument exists-preconditions (409/404) and honors updateMask on both write paths: masked paths are written or deleted-when-absent, unmentioned fields survive.

Wire fidelity

S3 PutObject returns the ETag header. Azure Put Blob honors x-ms-blob-content-type. GCS multipart uploads parse byte-exact via mime/multipart (trailing -/CRLF bytes are no longer stripped from payloads) and metadata.contentType takes precedence like real GCS. Firestore batchGet returns one JSON array (clients previously decoded only the first document) and runQuery honors structuredQuery.where and limit. Cosmos routes continuation-page queries correctly (application/query+json was previously misrouted to document create → 400) and both the query and GET-list paths page via x-ms-max-item-count / x-ms-continuation.

Test coverage

~110 lifecycle suites across 12 cells (storage + database × 3 providers × 2 surfaces): full lifecycles, typed errors, pagination continuation, multipart, TTL under the fake clock, streams/change feeds, GSIs, conditional writes, unicode keys, batch ops. Every fix is locked by a test that fails on pre-fix code.

Technical Details

Database drivers & pagination core
  • fix(driver): type-aware ordering for query/scan resultsdriver.CompareValues (numbers numerically, strings lexically, bools, fixed type rank for mixed) + SortByFields replace the Sprintf-keyed sort.
  • fix(dynamodb): key-based continuation in the driver, drop full materializationQueryInput.ExclusiveStartKey / SortDescending, ScanInput.ExclusiveStartKey, QueryResult.LastEvaluatedKey; one shared driver.PageOrdered paging path (stable sort → optional reverse → key or token slicing); the wire handler's fetchAll + paginateWire are deleted, so ConsumedReadCapacityUnits / ScannedCount reflect the returned page.
  • fix(driver): GSI queries order by index keys; LastEvaluatedKey carries index + base keys.
  • fix(pagination): enforce the stable-ordering invariant centrallyPaginate documents its contract, PaginateSorted enforces it (shuffled-input regression test), and providers/{aws/s3,azure/blobstorage,gcp/gcs} stop swallowing token decode errors.
  • review fixescheckCondition resolves #alias names and rejects compound expressions; drivers surface Paginate errors; unknown ExclusiveStartKeyValidationException; paginateWire key separator aligned with the driver (later removed entirely).
Storage wire handlers
  • campaign(storage)server/aws/s3: PutObject response carries the quoted sha256 ETag; server/azure/blob: x-ms-blob-content-type honored with request Content-Type as fallback; server/gcp/gcs: multipart parsing rewritten on mime/multipart.NextRawPart (byte-exact per RFC 2046, deletes splitHeaderBody / lookupHeader and the corrupting TrimRight), plus metadata.contentType precedence.
DynamoDB, Cosmos & Firestore wire handlers
  • campaign(database) — DynamoDB wire pagination (ExclusiveStartKey / LastEvaluatedKey) and ConditionExpression support; Cosmos recognizes continuation-page queries by media type and honors x-ms-max-item-count / x-ms-continuation; Firestore batchGet single-array framing and currentDocument precondition enforcement.
  • fix(firestore): runQuery honors structuredQuery where and limit — fieldFilter + AND composites mapped onto driver scan filters, operators accepted as enum names or protobuf numbers (what the REST client sends); unsupported shapes return INVALID_ARGUMENT; the 100-doc cap is gone.
  • fix(firestore): honor updateMask on document updates — merge semantics on the :commit write op (updateMask.fieldPaths, the Go client's Update path) and the PATCH query params.
  • fix(cosmos): GET document list pages via x-ms-max-item-count and x-ms-continuation.
Test suites
  • campaign(storage) / campaign(database) — portable suites in providers/{aws,azure,gcp}/{s3,blobstorage,gcs,dynamodb,cosmosdb,firestore} and real-SDK suites in server/{aws,azure,gcp}/…, named <service>_lifecycle_test.go per repo convention; SDK suites disable retries so error paths are observed on one attempt.
  • test: move server lifecycle suites into their service packages — external _test packages under server/<provider>/<service>/ with self-contained bootstraps.
  • Lifecycle tests that had pinned old divergences (filters-ignored runQuery, full-replace update) now assert real-cloud semantics.
  • chore: go mod tidygoogle.golang.org/grpc promoted to a direct test dependency.