Skip to content

v0.1.15

Choose a tag to compare

@robinnsc robinnsc released this 18 Nov 11:09
· 6 commits to main since this release
db81369

What's Changed

  • feat(migrations): one list of catalog migrations with a ledger in every store, for SQLite and PostgreSQL by @LeeroyHannigan in #666
  • feat(migrations): catalog migrations with a ledger in every store, for Cassandra and MongoDB by @LeeroyHannigan in #681
  • fix: limit failed logins per source address and bound bcrypt verification by @LeeroyHannigan in #718
  • fix: serve the console account pages to an IAM user for its own account only by @LeeroyHannigan in #734
  • feat(container): publish a Cassandra-backed ExtendDB image by @robinnsc in #725
  • ci: sign container images keyless via GitHub OIDC (Sigstore/Fulcio) by @LeeroyHannigan in #754
  • feat(backup): [backup.store], the store identity file, backup purge [backup/restore 7] by @robinnsc in #672
  • fix(console): end console sessions 8 hours after login by @hitesh96db in #875
  • feat(backup): CreateBackup job, worker and reconciler duties [backup/restore 8] by @robinnsc in #700
  • feat(backup): RestoreTableFromBackup job and cross-backend matrix [backup/restore 9] by @robinnsc in #740
  • feat(partiql): implement ExecuteStatement, BatchExecuteStatement, ExecuteTransaction by @LeeroyHannigan in #411
  • feat: store and report WarmThroughput on tables and global secondary indexes by @LeeroyHannigan in #474
  • fix(management): refuse access key import unless allow_credential_import is true by @LeeroyHannigan in #840
  • fix(server): check table names, index names and ARNs before authorization by @LeeroyHannigan in #830
  • feat(backup): backup sync, --adopt-store and --prune-missing [backup/restore 10] by @robinnsc in #825
  • fix(auth): check the scope and session token, canonicalize repeated headers, and time out stalled requests by @LeeroyHannigan in #355
  • ci: publish the workspace crates to crates.io by @LeeroyHannigan in #930
  • fix(iam): delete what a principal or account owns with it by @hitesh96db in #923
  • fix: create export files 0600 and the directories made for them 0700 by @LeeroyHannigan in #682
  • fix(deps): update aws-smithy-json to 0.62.7 and raise the minimum Rust version to 1.91.1 by @LeeroyHannigan in #857
  • feat(storage-cassandra): init sets replication factor and datacenter by @jcshepherd in #909
  • fix(deps): update tokio, webpki-roots and ctutils for three RustSec advisories by @LeeroyHannigan in #1088
  • chore(release): bump version to 0.1.15 and freeze migrations 1-12 by @robinnsc in #1069

These are the most notable of the 128 pull requests in this release. The Full Changelog below lists every one.

New Contributors

Full Changelog: v0.1.14...v0.1.15


Upgrading from 0.1.14

Run extenddb migrate --yes once before starting 0.1.15 on an existing catalog. 0.1.15 versions every backend's catalog with a migration ledger: a table in each store that records the catalog migrations applied to it. A production build of extenddb serve refuses a catalog with pending migrations (exit 3) and prints the command to run. This release carries migrations 1 to 12; extenddb migrate --list prints them. A new catalog needs no migrate step.

To upgrade:

  1. Stop every server.
  2. Take a backup of the catalog. On PostgreSQL, dump both the catalog database and the data database. On Cassandra, take a snapshot of the catalog keyspace and every account keyspace on every node.
  3. Run extenddb migrate --yes with the 0.1.15 binary.
  4. Start 0.1.15.
  • Dry-run first. extenddb migrate without --yes lists what it would apply and writes nothing. On PostgreSQL, pass --pg-user and set EXTENDDB_PG_PASSWORD, as for init.
  • Published images. The backend images run serve by default, so on an existing catalog they exit at startup until the migration has run. Run it once as a one-off container or Job against the same state before rolling out the new image. See the Upgrade section of docker/README.md.
  • Compose stacks. The bundled PostgreSQL, MongoDB and Cassandra compose files run migrate --yes on every start after the first.
  • Development builds. The extenddb-dev image and the @extenddb/dev npm launcher migrate the catalog at startup.
  • Rollback. Every migration in this release is breaking. A rollback to 0.1.14 restores the catalog backup taken before the upgrade; the 0.1.14 binary alone refuses the migrated catalog.

Behavior changes. Each is described, with what to check before upgrading, under "Behavior Changes by Release" in docs/manuals/07-upgrade-manual.md (the v0.1.15: entries). The ones most likely to need action:

  • MongoDB and PostgreSQL TLS no longer trust Atos TrustedRoot 2011, SecureSign Root CA12 and emSign ECC Root CA - G3 (#1088). On MongoDB this applies when the configuration names no certificate authority (CA) file. On PostgreSQL it applies with sslmode set to verify-ca or verify-full. Under prefer or require, nothing changes. A connection still succeeds when its configuration names the root as its CA file.
  • Access key import (import-access-key and the management API) is refused unless the allow_credential_import setting is true (#840).
  • server.region is part of every backup ARN.
  • CreateBackup with a configured [backup.store] is asynchronous on PostgreSQL, SQLite and MongoDB: it answers CREATING, and the backup becomes AVAILABLE when its upload ends (#700).
  • IAM evaluation changes: policy variables in Resource, NotResource and condition values (#535, #541, #926); ArnEquals/ArnNotEquals wildcards and Arn condition values (#471, #692); ? and * count UTF-16 code units (#581); tag condition keys on TagResource, UntagResource and CreateTable (#522); dynamodb:Attributes on every item request (#732); a LeadingKeys-fenced Query with a non-equality partition key condition gets AccessDeniedException (#511).
  • Console: an IAM user sees only its own account, user page and access keys, and the metrics dashboard is for admins (#727, #734, #844).
  • Export files are created 0600, and the import and export directories the server creates 0700 (#682).
  • Deleting a principal or account deletes what it owns (#923); disabling a stream closes its shards and enabling one creates a new stream (#515); metrics are kept per account (#955); point-in-time recovery is reported as unsupported (#335, #950).
  • Requests are validated as DynamoDB validates them: table names, index names and ARNs before authorization (#830), numbers on every key and item path (#619, #620), and the nesting depth of update expressions (#1000).

Highlights

Off-host backup and restore. With [backup.store] configured, CreateBackup writes a portable backup to a filesystem directory or an S3 bucket, and RestoreTableFromBackup restores it on any backend, including a different one from the source (#393, #394, #395, #671, #672, #700, #740). extenddb backup sync adopts, moves and forgets the backups in a store (#825, #880), and extenddb backup migrate-legacy carries in-catalog backups into it (#900). The new Backup and Restore manual covers choosing a store, recovery, and the restore drill (#855).

Catalog migrations. Every backend's catalog is versioned with a migration ledger, and extenddb migrate upgrades it (#666, #681). Migrations 1 to 12 are frozen in this release: later releases keep their bodies unchanged, so a catalog migrated by 0.1.15 upgrades without a re-run (#1068, #1069).

PartiQL and new API surface. ExecuteStatement, BatchExecuteStatement and ExecuteTransaction (#411). WarmThroughput is stored and reported on tables and global secondary indexes (#474).

Cassandra container image. extenddb-cassandra is published for the first time (#725). init takes the keyspaces' replication factor and datacenter, and serve warns of keyspaces at another replication (#909).

Security. Failed logins are rate-limited per source address and bcrypt verification is bounded (#718, #847); console sessions end 8 hours after login, or when the principal is deleted or its password changes, and the session cookie is Secure under TLS (#704, #773, #875); TLS private keys are created and kept 0600 (#766, #921); settings list and get mask secrets (#469); SigV4 checks the credential scope and session token and times out stalled requests (#355). Dependencies are updated for open RustSec advisories (#1088).

Other changes. The minimum Rust version for building from source is 1.91.1 (#857). The workspace crates are published to crates.io (#927, #930). The development image runs on distroless cc-debian13 (#1089).

npm packages

Package Version
@extenddb/dev 0.1.15 (latest)
@extenddb/linux-x64, @extenddb/linux-arm64, @extenddb/darwin-x64, @extenddb/darwin-arm64, @extenddb/win32-x64 0.1.15 (latest)

All six packages are published with npm provenance attestation through OIDC trusted publishing; verify with npm audit signatures.

crates.io

The 14 workspace crates are published at 0.1.15, for building ExtendDB from source; their APIs are not covered by the versioning policy.

Container images

All four images ship at 0.1.15, multi-arch (linux/amd64, linux/arm64), published and signed on all three registries.

extenddb-postgres: the production image

Registry Reference
Docker Hub docker.io/extenddb/extenddb-postgres:0.1.15
ECR Public public.ecr.aws/extenddb/extenddb-postgres:0.1.15 (gallery)
GHCR ghcr.io/extenddb/extenddb-postgres:0.1.15

Index digest (pin this in production):

sha256:244897a2653b6141544e8cc71beac7c99a1d956521ab97b2ffa8e5f8336e1902

extenddb-mongodb

The same DynamoDB-compatible API backed by an external MongoDB 7+ replica set. MongoDB itself is not bundled.

Registry Reference
Docker Hub docker.io/extenddb/extenddb-mongodb:0.1.15
ECR Public public.ecr.aws/extenddb/extenddb-mongodb:0.1.15 (gallery)
GHCR ghcr.io/extenddb/extenddb-mongodb:0.1.15

Index digest (pin this in production):

sha256:02c74eb9c7bafaf247cc9c1f74d2d09c989c23115e9a4cbc74f160fa33f9015c

extenddb-cassandra

New in this release. The same API backed by an external Cassandra 4.1+ cluster. Cassandra itself is not bundled.

Registry Reference
Docker Hub docker.io/extenddb/extenddb-cassandra:0.1.15
ECR Public public.ecr.aws/extenddb/extenddb-cassandra:0.1.15 (gallery)
GHCR ghcr.io/extenddb/extenddb-cassandra:0.1.15

Index digest (pin this in production):

sha256:44dec405707c49d2ae046000dac9036cbc5d716b637bd1492b6fc1eb11736278

extenddb-dev: the zero-config development image

One container, DynamoDB-compatible endpoint over SQLite, plain HTTP, no init step: for local development and CI, never production. Same three registries under extenddb/extenddb-dev. Index digest:

sha256:e5173e75f253ceeb49e56019843404ec206445de9f05b6d98385b0cc9e19d668

A latest tag is published for all four images; pin the version tag or digest for anything beyond experimentation.

Verifying signatures

0.1.15 is the first release signed keyless. Each image is signed by the GitHub Actions workflow that released it, through Sigstore, with no long-lived key; signatures are recorded in the public Sigstore transparency log. Images of 0.1.14 and earlier keep their key signatures and verify with extenddb-signing.pub.pem as before. With cosign, a backend image:

cosign verify \
  --certificate-identity-regexp '^https://github\.com/ExtendDB/extenddb/\.github/workflows/(release-image|sign-image)\.yml@refs/heads/main$' \
  --certificate-oidc-issuer 'https://token.actions.githubusercontent.com' \
  docker.io/extenddb/extenddb-postgres:0.1.15

and the development image, whose signer is release-dev-image.yml:

cosign verify \
  --certificate-identity-regexp '^https://github\.com/ExtendDB/extenddb/\.github/workflows/(release-dev-image|sign-image)\.yml@refs/heads/main$' \
  --certificate-oidc-issuer 'https://token.actions.githubusercontent.com' \
  docker.io/extenddb/extenddb-dev:0.1.15

The same commands work on all three registries. Successful verification prints the signed claims, including org.extenddb.release: v0.1.15.

The signer identity separates the production and development images. The first pattern rejects a development image, so an admission policy that matches it never admits the plain-HTTP development image where a backend image is expected. sign-image.yml is the break-glass signer that both patterns accept; drop it from the pattern to accept only the routine release path.