Skip to content

Sink v0.13.0

Choose a tag to compare

View all tags
@liran liran released this 14 Sep 18:06
v0.13.0
65007e7
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Sink v0.13.0 hardens batching, Lua execution, backend boundaries, and asynchronous processing, and adds per-store operational metrics.

Upgrade requirements

  • Remove service.batching.enabled from both server and worker configurations before upgrading. Batching is always enabled, and the strict configuration parser rejects the removed field for either value.
  • Search Kafka partition keys now use store, dataset, and record key without the logical namespace. When upgrading an existing search Kafka deployment, stop publishing, drain the old topic, and switch publishers and workers together. Mixing key schemes can route the same record to different partitions. Apply the same coordination when rolling back.
  • Merge now has one create-or-merge behavior. The former missing_document_mode field is reserved and ignored, including for older clients and queued records. Review callers relying on MissingDocumentFail and Lua scripts handling a missing current document. Upgrade callers to sink-go v0.6.0, whose wire format preserves create-or-merge compatibility with older servers.
  • Operational metrics now include a bounded store label. Review dashboard aggregations and recording rules when adopting per-store metrics.

Changes

  • Make bounded request batching mandatory and improve read batching, caller isolation, cancellation, and returned-document budget accounting.
  • Bound Lua allocation, Unicode conversion, pattern matching, table operations, and failure responses; preserve JSON/BSON types and validate UTF-8.
  • Correct MongoDB numeric record identities, conditional writes, duplicate/conflict classification, Count/Scan behavior, and query validation.
  • Harden Elasticsearch and OpenSearch query bounds, pagination sizing, response identities, endpoint recovery, and write retry behavior.
  • Split worker batches rejected by admission capacity without replaying successfully applied work, and preserve physical-record ordering across search namespaces.
  • Attribute request, write, batching, publisher, and worker metrics to configured stores.
  • Update the immutable public qualification suite with regression and per-store metric assertions.

Validation

The release candidate passed the full server CI, including race tests, lint, mutation-envelope fuzzing, MongoDB ReplicaSet integration, Elasticsearch/OpenSearch integration, public API conformance on OpenSearch 2.17 and 3.8, client protocol checks, Docker Compose quickstart, and release artifact builds. Local full race tests, vet, and staticcheck also passed.

Release binaries and container images are gated by the pinned public production qualification suite.

Full changes: v0.12.1...v0.13.0

Installation artifacts

Docker image:

ghcr.io/liran/sink:0.13.0

docker pull ghcr.io/liran/sink:0.13.0

Immutable image: ghcr.io/liran/sink@sha256:c0ac1379313ff64da773f4186cc1411f76421a0edfa07a0b2852fb6e0369ab9b

Standalone binaries for Linux and macOS on amd64 and arm64 are attached to this release.
Verify an archive with checksums.txt before installation.

Assets 7
Loading