Skip to content

taquba-v0.7.0

Choose a tag to compare

@micllam micllam released this 28 May 12:56
· 74 commits to master since this release
908744f

Added

  • EnqueueOptions::id_override lets callers supply the job id instead
    of receiving a generated ULID. Useful when the id must be known before
    the enqueue returns. Ids are validated at the API boundary (1-128 bytes
    of [A-Za-z0-9_-]) and bad inputs return the new
    Error::InvalidId { id, reason } variant. Callers should prefer
    ULID-shaped ids when FIFO-within-priority claim order matters:
    pending/scheduled keys end with the id, so claim order follows
    id sort.
  • Queue::clock() accessor returns the Arc<dyn Clock> the queue
    was opened with (or the default SystemClock). Lets downstream
    crates share the queue's time source for their own timestamp work
    so virtualising time with MockClock advances the whole stack
    in lockstep.
  • OpenOptions::flush_interval: Option<Duration> exposes SlateDB's
    WAL flush interval. None keeps SlateDB's own default (100ms).
    Every taquba state transition (enqueue, claim, ack, nack,
    dead_letter) blocks on txn.commit() which waits for the next
    flush tick, so this value is the lower bound on per-operation
    latency.

Changed

  • Breaking on-disk layout: the done: keyspace is reordered from
    done:{queue}:{id} to done:{completed_at:020}:{queue}:{id},
    mirroring the existing time-first layout of claimed: and
    scheduled:. The retention sweep can now early-exit on the first
    unexpired record instead of walking the full prefix. Public API is
    unchanged; in-flight runs from prior versions must be drained
    before upgrading because the old keys will not be observed by the
    reaper.
  • Queue::claim (and therefore claim_next / claim_with_wait)
    serialises same-queue claim attempts through an in-process
    tokio::sync::Mutex. Same-queue attempts no longer rely on
    SlateDB's transaction-conflict retry to resolve which worker
    takes the head of pending:. The lock is per-queue, so different
    queues' claims still run in parallel. Per-claim wall-clock latency
    under high single-queue concurrency drops from seconds to roughly
    one commit interval. Public API unchanged.
  • Queue::claim now maintains an in-memory per-queue cursor that
    records the most recently claimed pending: key, and starts the
    next claim's scan from immediately after it. This skips the
    tombstone band left by previously claimed (and deleted) pending:
    entries that the SlateDB iterator would otherwise walk. The
    cursor is invalidated whenever a pending: key is written at or
    before it (nack-requeue, dead-job requeue, reaper-requeue,
    scheduler promotion, and any enqueue at a lower-numbered
    priority); when this happens the next claim falls back to a full
    prefix scan. The cursor is not persisted: on process restart the
    first claim falls back to the prefix scan and re-warms naturally.
    Public API unchanged.
  • Bumped minimum slatedb version from 0.13 to 0.13.1.

Fixed

  • enqueue_with's non-dedup path (write_new) now retries on
    transaction conflict, matching the dedup path (write_unique),
    enqueue_with_kv, ack, dead_letter, and every other write path
    in the crate. Previously a conflict during a non-dedup enqueue would
    surface as Error::Storage to the caller; under normal contention
    this would have manifested as spurious enqueue failures that a retry
    could resolve.