v2.0.4

v2.0.4 - Supervised Worker Shutdown

Development prerelease. This release is published from dev, may receive
rapid follow-up changes, and is not selected by the default latest image.

Summary

v2.0.4 gives every worker owned by the backend worker manager a common
supervisor and a coordinated, bounded shutdown path. A worker that panics or
returns unexpectedly is restarted with exponential backoff, while an intended
drain stops new claims, completes bounded cleanup, and never starts replacement
work.

Shutdown now drains request-producing HTTP traffic before worker consumers.
The process first becomes unready, then allows active HTTP requests to finish,
drains workers, applies a hard cancellation when required, and finally closes
Redis and MySQL resources. Listener failures enter the same lifecycle as
SIGINT and SIGTERM instead of terminating through log.Fatalf.

This release does not add a database schema version, change an HTTP or upload
API, or change an image-processing recipe.

Changes

Unified worker supervision

• Runs heartbeat, outbox, V2 publish, V2 cleanup, view flush, cleanup, and user
  deletion workers under one named supervisor.
• Recovers worker panics and restarts unexpected returns with bounded
  exponential delays of 1, 2, 4, 8, 16, and 30 seconds. Further failures remain
  capped at 30 seconds.
• Logs the failed worker name and original panic value so repeated failures can
  be attributed without terminating the entire process.
• Keeps Manager.Start(ctx) blocking until all supervised workers stop and
  exposes an idempotent, concurrency-safe BeginDrain() signal.
• Interrupts restart waits immediately when either drain or hard cancellation
  begins, and never restarts a worker after either stop signal.

Producer-before-consumer shutdown

• Changes /ready to return HTTP 503 with status: draining as soon as
  shutdown begins. /health behavior is unchanged.
• Drains the HTTP server before asking workers to stop claiming work. This lets
  in-flight uploads finish creating their durable publish and outbox records
  before the consumers perform their final drain.
• Waits for workers during a soft-drain phase, then cancels their root context
  and waits through a separate hard-stop phase when needed.
• Closes Redis and the SQL pool only after worker completion or expiry of the
  worker stop budget.
• Routes unexpected listener exit through the same shutdown coordinator and
  reports shutdown failures without using a fatal logger in the lifecycle path.

Bounded lifecycle budget

• Enforces a 30-second application shutdown budget: up to 18 seconds for HTTP
  drain, 5 seconds for worker soft drain, 5 seconds for worker hard stop, and
  2 seconds for runtime resource closure.
• Increases the Compose backend stop_grace_period to 40 seconds, leaving time
  for the 30-second application budget and container runtime scheduling.
• Bounds final view flushing and lease cleanup with fresh contexts so cleanup
  can still run after the normal worker context has been canceled.

Durable publish and outbox leases

• Stops V2 publish and outbox workers from claiming new rows after drain begins
  while allowing the currently claimed operation to finish during soft drain.
• Releases interrupted publish and outbox claims with a fresh three-second
  cleanup context.
• Fences every release by lease owner and lease token. Cleanup from an old
  worker cannot clear a completed row or a lease acquired by a replacement.
• Refunds the claim attempt for orderly cancellation so shutdown alone does not
  consume the retry budget.
• Preserves the attempt for a publish panic, then rethrows the panic to the
  supervisor so deterministic failures can still reach the attempt limit.
• On an outbox delivery panic, marks only the active event failed, refunds
  untouched claimed events, and rethrows the original panic to the supervisor.

Final view-count flush

• Performs one final view-count flush with a fresh context bounded at three
  seconds on either soft drain or hard cancellation.
• When an explicit MySQL update error follows Redis GETDEL, restores the
  removed count with a fresh one-second Redis INCRBY context instead of
  silently dropping it.
• Keeps the restore operation inside the five-second worker hard-stop budget.

Continuous integration

• Adds a dedicated lifecycle schema to the pinned MySQL 8.4.10 CI service.
• Adds required real-MySQL coverage for publish lease owner/token fencing,
  publish panic release and propagation, and outbox lease owner/token fencing.
• Adds supervisor, restart-backoff, drain ordering, lifecycle deadline,
  readiness-gate, view-flush, and Compose grace-period unit tests.

Verification

• Real MySQL 8.4 lifecycle schema bootstrap and three lease-cleanup integration
  scenarios: passed.
• Backend go test ./... -count=1: passed.
• Backend race tests for server lifecycle, workers, and publish service: passed.
• Backend go vet ./...: passed.
• Backend go build ./...: passed.
• Frontend ESLint, Vitest, TypeScript project, and Vite production build: passed.
• Python requirements lock and release-image policy tests: passed.
• GitBook documentation and translation verification: passed after refreshing
  translation source hashes for this release.
• The existing wasm-vips direct-eval dependency warning remains unchanged and
  does not fail the build.

Installation

Development images must be pulled explicitly:

docker pull jaykserks/summerain:dev-v2.0.4

Equivalent exact development tag:

jaykserks/summerain:dev-2.0.4

The moving dev tag also points to the newest successful development build.
latest, main, and stable semantic-version aliases are not changed by this
release.

No database migration or API migration is required when upgrading from
v2.0.3. A rolling upgrade is supported, but production operators should pin an
exact image tag or digest rather than the moving dev tag.

Known Limitations

• View counts still cross Redis and MySQL without a transactional or idempotent
  ledger. If MySQL commits an update but returns an uncertain error, restoring
  the Redis count can apply that count a second time during a later flush. The
  new restore path prevents loss for explicit failures; it does not provide
  cross-store exactly-once delivery.
• Heartbeat and user-deletion workers stop between batches, but every database
  operation inside an already running batch is not yet fully context-aware. An
  in-flight database call can therefore remain until its driver operation
  returns or the process stop deadline is enforced.
• Strict configuration parsing and validation remain planned for v2.0.5.
• The supervisor covers only background workers owned by Manager. Resource-
  intensive paths executed inside requests, including legacy V1 AVIF
  conversion, are not managed by this supervisor. Their resource admission and
  limiting controls remain planned for v2.0.6.
• Complete readiness checks for runtime dependencies and worker health remain
  planned for v2.0.7. v2.0.4 only makes readiness fail immediately during
  shutdown drain.
• Animated image support remains planned for a later release.