Skip to content

Releases

v0.3.6

Choose a tag to compare

@danish-a1 danish-a1 released this 10 Nov 17:44

Added

  • goqueue.Permanent(err) (also job.Permanent) marks a job error as non-retryable, and goqueue.IsPermanent(err) detects the mark. When a handler returns a permanent error (also when wrapped again with %w), the worker skips the remaining retries and dead-letters the job at once instead of retrying a request that cannot succeed, such as one an API rejected with a 4xx. The DLQ entry keeps the original error chain. Permanent(nil) is nil.

v0.3.5

Choose a tag to compare

@danish-a1 danish-a1 released this 10 Nov 17:25

Fixed

  • Jobs lost when they failed during shutdown. A job that failed on its last attempt after the worker context was canceled was dead-lettered with that canceled context, so network DLQs (Redis, Postgres, MySQL) refused the push; the job was then acked and gone. DLQ pushes now detach from the worker context's cancellation (context.WithoutCancel) with their own 5s timeout (dlqPushTimeout). Regression test: TestDLQPushSurvivesWorkerCtxCancel.

v0.3.4

Choose a tag to compare

@danish-a1 danish-a1 released this 10 Nov 16:54

Fixed

  • Graceful shutdown abandoned in-flight jobs. Apps cancel the worker context (their signal context) before calling Shutdown, and processJobSafely stopped waiting for the handler as soon as that context was done: Shutdown returned while the job was still running, and the job was counted as failed (retried or dead-lettered with context canceled). The worker now waits for the handler to return when the worker context is canceled; the handler still sees the cancel, and Shutdown's own deadline bounds the wait. Per-job timeouts still abandon the handler as before. Regression test: TestShutdownWaitsForInFlightJobAfterCtxCancel.

v0.3.3

Choose a tag to compare

@danish-a1 danish-a1 released this 10 Nov 16:27

Fixed

  • Busy spin in the metrics worker. Its select had a default: branch that called DequeueMetrics back to back, so whenever the call returned at once (SQS and database stores when empty, the memory store before its first metric, any store on error) the goroutine burned a full CPU core and, on errors, flooded the log. It now polls on a timer: after an empty or failed call it waits 1s (metricsIdleWait), after a delivered metric it polls again immediately to drain. Shutdown and new metrics still interrupt the wait. Regression tests: TestMetricsWorkerDoesNotSpinWhenIdle, TestMetricsWorkerDeliversMetric.
  • TestVersion no longer hard-codes a version string, which had it failing since 0.3.2.

v0.3.2

Choose a tag to compare

@danish-a1 danish-a1 released this 10 Nov 16:06

Fixed

  • Data race between Worker.Start and Worker.Shutdown. Start launched startMetricsWorker with go, so its wg.Add(1) could run after Shutdown had entered wg.Wait() (reported by -race; an Add racing a Wait on a zero counter is WaitGroup misuse). startMetricsWorker is now called synchronously: it still runs in its own goroutine, but its Add happens before Start returns. Regression test: TestWorkerShutdownImmediatelyAfterStart.

Docs

  • README: Features list and comparison table now mention the web dashboard; version badge corrected (it was stuck at 0.0.1, so scripts/release.sh could not bump it).

v0.3.1

Choose a tag to compare

@danish-a1 danish-a1 released this 08 Nov 22:08

Fixed

  • Redis ACL username is now sent to Redis. NewRedisConfig has accepted a username since 0.2.3 and stored it since 0.2.5, but the Redis client was built without it, so connections authenticated as the default user and failed on servers that require an ACL user (WRONGPASS invalid username-password pair).

Full Changelog: v0.3.0...v0.3.1

v0.3.0

Choose a tag to compare

@danish-a1 danish-a1 released this 08 Nov 22:00

Added

  • dashboard package: a password-protected web UI served as a plain http.Handler (Redis driver only, stdlib only, no new dependencies). Shows totals, per-day history, per-queue counts, job IDs, busy workers, retries, dead and deleted jobs, with Retry / Kill / Delete / Restore (single and batch, atomic Lua moves, nothing is hard-deleted). Mount it with dashboard.New(dashboard.Options{...}). An empty access token disables it (404 on every path); tokens must be at least 32 bytes. See docs/dashboard.md.
  • Tracking keys (Redis): goqueue:queues registry, goqueue:stat:* processed/failed counters (global, daily, per queue), goqueue:process:* worker heartbeats and busy-job hashes, goqueue:deleted:<queue> deleted-job lists. Key names live in adapter/redis/keys.go. NewQueue registers its queue with HSETNX (a producer never overwrites an existing dead key); Worker.Start registers the consumer's real dead key with HSET, so a producer on the default DLQ cannot hide a consumer's custom DLQ list.
  • Dashboard actions return {"Done","NotFound","Skipped"}. Skipped holds entries still at the source that cannot be moved (missing job name or unreadable data); the UI keeps their rows.
  • examples/dashboard: runnable demo on miniredis.

Changed

  • DLQ entries (Redis) now share one shape (dlq.Entry): field queue_name is renamed queue, job_name is added, and the error is capped at 256 bytes. dlq.DecodeEntry still reads legacy queue_name.
  • RedisDLQ.Push now uses RPUSH (newest entry at the tail instead of the head), keeps at most 1000 entries per queue and expires the list after 30 days.
  • Retry envelopes carry error_message and failed_at (last failed attempt); Pop exposes it as JobContext.LastError. Old envelopes without these fields still decode.

Breaking

  • RedisDLQ ordering: the newest dead job is now at the tail of the list, not the head. Anything that reads the list by index (LINDEX 0, LRANGE 0 n) now sees the oldest entries first.
  • RedisDLQ field rename: queue_name is now queue. Readers must use dlq.DecodeEntry or accept both names. Entries already stored with queue_name stay readable through DecodeEntry.

Upgrade notes

Legacy RedisDLQ lists (0.2.9 and older) were written with LPUSH (newest entry at the head) and never capped. 0.3.0 pushes with RPUSH and then LTRIM -1000 -1, which keeps the tail. On the first dead job after the upgrade:

  • a legacy list longer than 1000 entries loses its newest entries (they sit at the head), and
  • the remaining legacy entries show in reverse order in the dashboard (oldest as newest).

Before upgrading, with the old workers stopped:

  1. Find long lists: redis-cli --scan --pattern 'dlq:*' | while read k; do echo "$(redis-cli LLEN "$k") $k"; done | sort -rn
  2. Archive or drain every dlq:* list longer than 1000 entries, for example redis-cli COPY dlq:<queue> dlq-archive:<queue> (Redis 6.2+) and process or drop the archive yourself.
  3. Reverse each legacy list in place once, so the newest entry is at the tail. Run it exactly once per list (a second run reverses it back):
redis-cli EVAL "local v = redis.call('LRANGE', KEYS[1], 0, -1); redis.call('DEL', KEYS[1]); for i = #v, 1, -1 do redis.call('RPUSH', KEYS[1], v[i]) end; return #v" 1 dlq:<queue>

Then start the 0.3.0 workers. Custom DLQ adapters that already RPUSH with a cap (for example fleetlery's) are unaffected.

Fixed

  • <queue>:metrics:ack no longer grows without bound: DequeueMetrics uses BRPOP instead of BRPOPLPUSH and never wrote an ack for metrics. MetricsAckQueueSuffix is deprecated. Clean up existing lists with: redis-cli --scan --pattern '*:metrics:ack' | xargs redis-cli del.

Full Changelog: v0.2.9...v0.3.0

v0.2.9

Choose a tag to compare

@danish-a1 danish-a1 released this 08 Nov 22:00

Fixed

  • Idle workers now stop promptly on shutdown. The Redis Pop blocked forever on an empty queue (BRPOPLPUSH with timeout 0), so Shutdown always waited the full timeout and returned context.DeadlineExceeded. Pop now blocks for at most 1 second and returns an empty job, letting the worker loop re-check for shutdown.
  • The metrics goroutine now also exits on Shutdown, not only when the Start context is canceled, so Shutdown no longer times out when the caller keeps that context alive.

Full Changelog: v0.2.8...v0.2.9

v0.2.8

Choose a tag to compare

@danish-a1 danish-a1 released this 22 Oct 15:45

Added

  • Reclaim jobs left behind by a crashed process. A Redis job sits in processing:<queue> while a worker runs it. If the process is killed (SIGKILL, OOM, host loss), the job used to stay there and never run again. Config.WithReclaimProcessingOnStart(true) now moves those jobs back onto the queue when StartWorkers runs.
    • Opt-in: only enable it when a single consumer instance serves the queue. With several instances it would also move jobs another instance is still running.
    • Stores without a processing list are skipped; a reclaim failure is logged and does not stop the workers.

v0.2.7

Choose a tag to compare

@danish-a1 danish-a1 released this 22 Oct 10:33

Fixed

  • Redis DLQ never received jobs. RedisDLQ.Push redeclared err while marshalling the job, so err.Error() panicked on nil for every job that exhausted its retries.
  • Caller DLQ adapter ignored. NewQueue replaced a configured DLQAdapter with the driver default for the Redis and database drivers. The default now applies only when no adapter is set.
  • Database driver init. NewQueue used the database store before checking its init error, so a failed connection panicked.
  • Panicking jobs crashed the process. The job handler runs on its own goroutine, out of reach of the worker's recover. A panic now becomes a job error and follows the normal retry / DLQ path.

Also includes v0.2.6: the Redis health check no longer closes the shared client after a transient ping failure.