v0.6.14

What's Changed

• runtime-next: build per-collection state once per Target / Source by @jgraettinger in #3330
• runtime: remove shard recovery directory on Destroy by @williamhbaker in #3334
• docs: use :::deprecated admonition for deprecated connector notices by @jwhartley in #3321
• runtime-next: log an auto-split only when one is started by @dgreer-dev in #3346
• runtime-next: absorb a shard Stop which outruns the session it addressed by @jacobmarble in #3339
• Docs: Update dashboard instructions for alert configs by @aeluce in #3327
• mise: pin RocksDB's target ISA instead of inheriting the builder's by @Alex-Bair in #3348
• docs: correct the default materialization maxTxnDuration by @jwhartley in #3347
• sources: update snapshot for catalog schema by @danielnelson in #3360
• data-plane-controller: pass through aws_resource_tags stack config field by @SeanWhelan in #3355
• docs: fix broken anchor, enforce anchor checking, document macOS build gotcha by @jwhartley in #3272
• docs: reference gcloud storage instead of gsutil for GCS bucket IAM by @jwhartley in #3220
• docs: clarify sync schedule connector scope and backfill heuristic by @jwhartley in #3233
• connector-init: demote broken-pipe stdin writes to debug by @jacobmarble in #3358
• docs: document materialize-s3-iceberg variant_columns option by @jacobmarble in #3367
• control-plane-api: return stored storage mapping specs by @GregorShear in #3364
• docs: fix hubspot property name for /_meta/op field by @danielnelson in #3368
• docs: use :::deprecated on four deprecated connector pages by @jwhartley in #3362
• control-plane-api: add id and an id filter to the dataPlanes GraphQL query by @GregorShear in #3240
• dekaf: keep connection open on document validation error by @danielnelson in #3359
• docs: Shopify REST API tense, drop "Estuary Flow" from Smartsheet by @jwhartley in #3370
• control-plane, agent: discovers authorize live specs via Snapshot by @bbartman in #3373
• shuffle: refresh the disk-backlog gauge on the LogActor tick by @williamhbaker in #3361
• agent: discovers authorize data-planes via Snapshot by @bbartman in #3386
• runtime-next: log when a task applies a changed spec by @williamhbaker in #3369
• docs: Explain 'Additional Backfill Filter' usage by @willdonnelly in #3381
• go.mod: bump gazette to latest by @williamhbaker in #3390
• ci: mark site/ as the deprecated docs location by @jwhartley in #3384
• agent: discovers require SpecEdit to the capture by @bbartman in #3389
• flowctl: bound fixture log blocks so a large transaction cannot trip the block assert by @jacobmarble in #3391
• runtime: captures use the V2 runtime by default by @williamhbaker in #3400
• runtime-next: tolerate connector exit during Acknowledge; preview sessions stop at capture EOF by @jgraettinger in #3401
• runtime-next: bound status trailers so connector errors aren't masked by ResourceExhausted by @jacobmarble in #3356
• control-plane-api: Add public/private filter option to DataPlanes query by @adrian-estuary in #3403
• Docs: Point OpenMetrics API endpoint at api.estuary.dev by @skord in #3411
• runtime-next: yield from the materialize actor's scan loop by @dgreer-dev in #3412
• flowctl: bound preview-next fixture memory and disk by @jacobmarble in #3408
• flowctl: preview-next no longer hangs when its fixture feeder dies by @jacobmarble in #3410
• ci: label PRs that ship with the Deploy agent-api workflow by @GregorShear in #3374
• Labeler fixes by @GregorShear in #3431
• docs: promote backfill modes to headings, mark Shopify deprecated by @jwhartley in #3371
• docs: name the single MotherDuck metadata table, document splitting it by @jwhartley in #3382
• doc: bound allocator growth when draining a key across many segments by @williamhbaker in #3428
• one reconciler job per target by @GregorShear in #3432
• validation: stamp created_at on a task's first build by @dgreer-dev in #3447
• Runtime "Sync Now" by @dgreer-dev in #3326
• Housekeeping and prep for the catalog-test harness by @jgraettinger in #3415
• flowctl: don't underflow the fixture disk backlog when compression grows a segment by @mdibaiee in #3445
• go/bindings: restamp TestBuildCatalog snapshots with created_at by @jgraettinger in #3448
• flowctl: preview runs on the V2 runtime, remove V1 preview by @jgraettinger in #3442
• go/flow/keyhash: split the packed-key hash into its own package by @jacobmarble in #3441
• runtime-local: extract the local task-drive layer from flowctl by @jgraettinger in #3416
• shuffle: never adopt an expanding compression of a sealed segment by @jacobmarble in #3429
• catalog-tests: the catalog test harness, and flowctl raw test by @jgraettinger in #3417
• control-plane-api: Add tenant filter to dataPlanes query by @adrian-estuary in #3437
• materialize: consistency suite verifying exactly-once against a real runtime by @mdibaiee in #3294
• agent, control-plane-api: publications authorize via Snapshot by @bbartman in #3387
• control-plane-api: run publication tests on catalog-tests by @jgraettinger in #3418
• shuffle: remap materialization key routing to couple shard read rates by @dgreer-dev in #3439
• network: add HSTS header to frontend HTTP responses by @skord in #2262
• ci,local: one package set for local-stack binaries, and warm CI caches by @jgraettinger in #3454
• Remove the V1 catalog-test machinery by @jgraettinger in #3419
• shuffle, runtime-next: send panic statuses to peers by @williamhbaker in #3460
• control-plane-api: publications check spec-to-spec authorization via Snapshot by @bbartman in #3393
• First-class secrets: the model, control plane, and config-encryption foundations by @jgraettinger in #3423
• shuffle: gate the startup heap drain on a Barrier from the Session by @williamhbaker in #3456
• control-plane-api: create_data_plane gates ops admin via Snapshot by @bbartman in #3435
• control-plane-api: update_l2_reporting gates ops admin via Snapshot by @bbartman in #3436
• Derivation connector cleanups, user-defined environment variables, and shared gRPC helpers by @jgraettinger in #3465
• runtime-next: make per-shard tokio worker threads configurable by @jgraettinger in #3470
• proto-flow: gather connector Request/Response fields into a oneof kind by @jgraettinger in #3469
• shuffle: discharge causal hints made unreachable by a byte gap by @williamhbaker in #3430
• go.mod: bump gazette to latest by @williamhbaker in #3478
• update_l2_reporting: run the old L2 stats rollup on runtime-v2 by @williamhbaker in #3487
• data-plane-controller: new task type for testing separate builds by @qaoj in #3481
• agent: remove the storageMappings applied directive by @GregorShear in #3489
• Allow negative read priority on derivation transforms and materialization bindings by @jgraettinger in #3494
• network: pin auth-redirect orig_url to the request host by @skord in #3499
• workflows: declare minimal GITHUB_TOKEN permissions by @skord in #3500
• Introduce models::authz::Subject to encapsulate authorization context beyond just user identity by @jshearer in #3502
• network: drop the bearer token from authorization errors by @skord in #3503
• product name: use "Estuary" instead of "Estuary Flow" by @jwhartley in #3463
• connector: a V2 connector service, served in-process and over the network by @jgraettinger in #3490
• unseal: strip encrypted_suffix in Rust rather than shelling out to jq by @dgreer-dev in #3510
• deps: upgrade sqlx to 0.9 and rusqlite to 0.39 (SQLite 3.51) by @jgraettinger in #3512
• supabase: oauth edge function validation by @Alex-Bair in #3515
• oauth: require a user when callers supply a connector_config by @Alex-Bair in #3517
• runtime-next: stop a session whose shuffle remainders pin too much disk by @williamhbaker in #3518
• graphql: add draft management API by @jshearer in #3504
• oauth: require a user for encrypt-config by @skord in #3523
• CapabilityMask Support by @bbartman in #3509
• billing: check payment method belongs to tenant before updating it by @skord in #3526
• 3376 Capability-Masked Tokens: No refresh tokens by @bbartman in #3527
• control-plane-api: 2973 - Add catalog stats GQL resolver by @adrian-estuary in #3519
• graphql: compute draft isUnchanged from live_specs.md5 by @jshearer in #3525
• Control plane: cut over to the V2 connector service by @jgraettinger in #3514
• Fix schema inference for enum fields omitted from sourced schemas by @dgreer-dev in #3532
• flowctl: send backfill begin/complete signals from preview-next fixtures by @jacobmarble in #3529
• mise: Minor upgrade Supabase CLI by @GregorShear in #3549
• data-plane-controller: plumb runtime_worker_threads by @qaoj in #3544
• control-plane-api: allow GraphQL request bodies up to 16 MiB by @jshearer in #3538
• Clean up live specs, fix pagination bugs by @GregorShear in #3546
• flowctl: manage drafts through the GraphQL drafts API by @jshearer in #3524
• graphql: look up specification revisions by publication ID by @jshearer in #3539
• docs: Snowflake high-performance Snowpipe Streaming write path by @jacobmarble in #3275
• control-plane: Add sensitive flag to tenants table and expose in GQL by @adrian-estuary in #3557
• Tenant create mutation by @GregorShear in #3542
• Revert Supabase CLI upgrade to 2.54.11 by @GregorShear in #3561
• Remove the deprecated site/ docs tree by @jwhartley in #3556
• First-class secrets: control-plane handling by @jgraettinger in #3533
• Add optional prefix_scope claim by @GregorShear in #3543
• Fix Rust 1.99 deprecation and lint errors by @jgraettinger in #3564
• Label merged PRs pending:migration if applicable by @GregorShear in #3566
• Say where the docs moved, and fail PRs that add files under site/ by @jwhartley in #3563
• 3376 Capability Tokens: Token minting by @bbartman in #3530
• shuffle: lower PRODUCER_STALENESS_HORIZON to 24h by @jgraettinger in #3570
• First Class Secrets: data-plane resolution, task update, and flowctl secret by @jgraettinger in #3568
• Add unauthenticated legalTerms gql query by @GregorShear in #3567
• Sandbox API by @GregorShear in #3520
• billing: add --dry-run mode and always send manual invoices by @jshearer in #3099
• control-plane: 3395 Deprecate storageMapping.spec graphql field in favor of new structured fields by @adrian-estuary in #3572
• shuffle: use blocking reads for gapped-producer replays by @jgraettinger in #3576
• shuffle: send Appends to Logs in rounds, merged under Slice constraints by @jgraettinger in #3574
• runtime-next: hot-path optimizations by @jgraettinger in #3575
• Put CreateSandbox bit into its own bundle (and remove from Admin) by @GregorShear in #3580
• Add dataPlane arg to tenantCreate mutation by @GregorShear in #3578
• Move live spec authz to capability bits by @GregorShear in #3547
• gazette, shuffle: avoid exhausting h2's local-reset budget by @jgraettinger in #3581
• tokens, runtime-next, publisher: bound and reduce derivation token requests by @williamhbaker in #3582
• Revert "shuffle: send Appends to Logs in rounds, merged under Slice constraints" by @jgraettinger in #3590
• unseal: retry sops when it cannot retrieve its data key by @jacobmarble in #3587
• dekaf, flow-client: tighten authentication checks by @skord in #3591
• runtime-lab: a single-VM reproduction lab for runtime-next and shuffle by @jgraettinger in #3573
• graphql: add discovery APIs by @jshearer in #3545
• internal.log_lines: preserve microsecond timestamp order across writer batches by @jshearer in #3550
• Reject Estuary name variants in tenantCreate by @GregorShear in #3601
• go: bump x/crypto, x/net, x/text and grpc for HIGH CVEs by @skord in #3603
• graphql: filter prefixes by catalog prefix by @jshearer in #3595
• deps: fix HIGH and MEDIUM Trivy findings in Rust and Python by @skord in #3604
• proto-grpc, runtime-next: pass connector errors through unwrapped by @williamhbaker in #3613
• Reject Unicode confusables of estuary in tenantCreate by @GregorShear in #3616
• assemble, go/shuffle: keep sub-second precision of notBefore / notAfter by @jwhartley in #3615
• flowctl: draft live and historical specifications through GraphQL by @jshearer in #3540
• shuffle: precise Log merges through positioned round Barriers by @jgraettinger in #3611
• doc: re-use a spilled MemTable's allocator within a transaction by @jgraettinger in #3612
• flowctl: map the GraphQL AlertType to a string in alerts subscriptions by @jshearer in #3617
• Retire clickToAccept and betaOnboard directive processing by @GregorShear in #3602
• Retire demo tenant directive and directive execution by @GregorShear in #3624
• Try GHCR first with ECR and Docker Hub fallback for Supabase by @GregorShear in #3632
• graphql: enforce CatalogRead on publicationForId by @jshearer in #3619
• control-plane-api: GraphQL createPublication and publication(id) by @jshearer in #3629
• GraphQL: report required bundles on authorization failures by @GregorShear in #3631
• go.mod: bump gazette to latest by @williamhbaker in #3636
• ci: fail pull requests on HIGH or CRITICAL dependency vulnerabilities by @skord in #3620

New Contributors

• @adrian-estuary made their first contribution in #3403

Full Changelog: v0.6.13...v0.6.14