Skip to content

Transactional job completion: commit handler output atomically with the completion (Rust in-tx API + SQL contract for Python et al) #401

Description

@hardbyte

Summary

Let handlers commit their business output atomically with the job's completion in the same Postgres transaction — upgrading at-least-once execution to exactly-once state effects within the application's database.

Requested-shape example: a handler writes rows to the app's tables and the job's completed transition commits with them, or neither happens; a crash before commit means a clean retry, and the existing run_lease stale-completion guard already refuses a rescued attempt's late commit — no double-apply.

Why the pieces are already here

Design constraints (for the ADR)

  1. Python (and any non-Rust consumer) must be inverted. Don't pass a live sqlx transaction across FFI — expose completion as a SQL contract (awa.complete_job_compat(job_id, run_lease, …) family mirroring insert_job_compat) that the user calls inside their own transaction with their own driver (asyncpg/SQLAlchemy/psycopg). The handler then returns a new JobResult::AlreadyFinalized so the executor skips its completion. This belongs inside Stabilize the SQL enqueue contract as a public producer API #342's SQL contract — completion, not just enqueue.
  2. Opt-in bypass of the completion batcher. Queue-storage batches completions (Compact receipt-backed completions #352); a user tx can't join a batch carrying other jobs' completions. Atomic completions write their own single terminal evidence in the user's tx — the pre-batching shape, bounded to opted-in jobs. Throughput-sensitive queues don't opt in.
  3. Rust gets the ergonomic in-process variant: store.complete_in_tx(&mut tx, job_id, run_lease, …) + the same result variant. ADR-029 follow-ups join the same tx via existing _tx inserts.
  4. Failure paths compose: user-tx conflict → handler error → normal retry; fail/DLQ variants already have _in_tx shapes.

Deliverables

ADR (touches ADR-026/029), Rust API + queue-storage complete_in_tx path, the SQL completion contract + cross-driver examples and conformance coverage (with #342), tests including a rescue-races-user-commit witness (TLA+ candidate: completion-vs-rescue with a user-held transaction), docs positioning ("exactly-once state effects in your Postgres").

Sequencing: M4, alongside #342. From the roadmap-gap review; requested in discussion of worker-side atomicity for Rust/Python clients.

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew functionality

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions