Skip to content

v1.2.0: worker transport reliability

Choose a tag to compare

@github-actions github-actions released this 09 Sep 17:07
· 8 commits to main since this release

v1.2.0: worker transport reliability

This series follows the maintenance backports at
e69d246f575e899700528027d927fbcf41f7b6b1.

  • Preserve task resolutions that arrive before the corresponding task is
    registered during replay, without dispatching already resolved work again.
  • Issue a fresh completion token for each gRPC dispatch attempt and validate it
    before forwarding a worker response to the backend. The Go worker echoes the
    token on both success and failure responses.
  • Recover attempts owned by a disconnected stream through backend abandonment
    and reacquisition. Preserve the registry reservation until any in-flight
    backend completion finishes, so stale cleanup cannot affect a successor.
  • Clean up canceled local waiters conditionally and make executor shutdown
    idempotent without closing a queue while producers may still send.

The public synchronous Backend and Executor interfaces, generated protobufs,
and database schemas are unchanged. No schema migration is required.

Histories with unresolved legacy timer numbering retain the ambiguous work until
a scheduling event confirms the task IDs. If no such evidence arrives, replay
remains waiting with a warning; the engine does not guess which task owns a
response or publish work with provisional IDs.

Worker rollout

Empty-token responses remain accepted by default for compatibility. That mode
cannot fence stale responses from legacy workers. Upgrade every worker before
requiring completion tokens on the server:

executor, register := backend.NewGrpcExecutor(
    be, logger, backend.WithRequireCompletionTokens(),
)

In strict mode, a worker that does not echo tokens cannot complete its work.
Completion-token validation does not provide exactly-once external side effects;
activities still require external idempotency when effects can be retried.

See the transport design and proof boundary
for upstream commit provenance and compatibility details.

Validation (2026-09-09)

  • The complete suite passed with PostgreSQL enabled:
    go test -p 1 -count=1 -timeout=3m ./....
  • Race-detector checks passed for the task replay tests, executor transport
    regressions, local waiters, client, and SQLite/gRPC recovery test.
  • All five PostgreSQL crash/restart scenarios passed in an isolated Docker
    instance; go vet ./... and git diff --check passed.
  • Independent cross-review finished with no remaining findings after correcting
    ambiguous legacy timer IDs, detached-workflow dispatch, and shutdown blocking
    behind a slow backend completion.

These local checks used Go 1.27.0-X:nodwarf5 and PostgreSQL 17. The transport
recovery test uses SQLite on disk and gRPC bufconn; it proves abandonment,
reacquisition, stale-token rejection, and terminal completion locally, not a
deployment or a real network partition. PostgreSQL crash tests are a separate
backend durability proof.

The runtime baseline faccb3cbb050ba69e72569f664c617154df559a4 also passed
PR Validation
and OpenTelemetry integration.
Release preparation changes documentation and adds the tag-triggered publication
workflow only; the validated runtime is unchanged.