go-task-queue

A distributed task queue built with Go and Redis. Tasks are enqueued by a client, processed by concurrent workers with retry logic and dead-letter handling, and monitored through Prometheus and Grafana.

How It Works

Client ──LPUSH──> Redis Queue ──BLMove──> Worker (N goroutines)
                                              │
                                      ┌───────┴───────┐
                                   Success          Failure
                                      │               │
                               Remove from       Retry < Max?
                              processing queue     ┌──┴──┐
                                      │          Yes     No
                               Set idempotency   Re-enqueue  Move to
                                key (24h TTL)    with retry++ dead-letter queue

1. The client pushes tasks onto a Redis list via LPUSH
2. Workers atomically move tasks from the main queue to a :processing queue using BLMove (blocking pop + push)
3. The registered handler for the task type runs
4. On success, the task is removed from :processing and its idempotency key is recorded
5. On failure, the task is re-enqueued with an incremented retry count, or moved to a :dead queue if max retries are exceeded

Project Structure

cmd/
  client/main.go        # Enqueues sample tasks
  worker/main.go        # Starts worker process with registered handlers
internal/
  config/config.go      # Env-based configuration with defaults
  queue/broker.go       # Redis broker, task struct, enqueue logic
  worker/worker.go      # Concurrent processor, retry/DLQ, metrics recording
  metrics/metrics.go    # Prometheus counter + histogram
deploy/
  Dockerfile.worker     # Multi-stage build (Go compile -> Alpine runtime)
  prometheus.yaml       # Scrape config targeting worker:2112
docker-compose.yaml     # Redis, Worker, Prometheus, Grafana

Configuration

All config is loaded from environment variables (or .env file). See .env.example:

Variable	Default	Description
REDIS_ADDR	localhost:6379	Redis connection address
REDIS_PASSWORD	(empty)	Redis auth password
REDIS_DB	0	Redis database number
QUEUE_NAME	gopher_tasks	Name of the Redis list used as the queue
WORKER_CONCURRENCY	5	Number of goroutines processing tasks
METRICS_PORT	:2112	Port for the Prometheus /metrics endpoint

Prerequisites

• Go 1.25+
• Docker and Docker Compose

Installation & Running

# 1. Clone the repository
cd go-task-queue

# 2. Copy the example env file and adjust if needed
cp .env.example .env

# 3. Start the full stack (Redis, Worker, Prometheus, Grafana)
docker compose up --build

# 4. In a separate terminal, enqueue sample tasks
go run cmd/client/main.go

To stop everything:

docker compose down

Service URLs

Service	URL	Credentials
Prometheus	http://localhost:9090	--
Grafana	http://localhost:3000	admin / admin
Worker metrics	http://localhost:2112/metrics	--

Metrics

Metric	Type	Labels	Description
go_task_queue_tasks_processed_total	Counter	type, status	Total tasks processed, labeled by task type and result (success/fail)
go_task_queue_task_duration_seconds	Histogram	type	Processing duration per task type

Features

• Concurrent processing -- configurable number of worker goroutines
• Atomic task movement -- BLMove prevents tasks from being lost between pop and processing
• Retry with backoff -- failed tasks are re-enqueued up to MaxRetries (default 3)
• Dead-letter queue -- tasks that exceed max retries land in a :dead queue for inspection
• Idempotency -- optional key per task prevents duplicate processing within a 24-hour window
• Observability -- Prometheus metrics with Grafana dashboards

Known Limitations

• No backoff delay on retries -- failed tasks are immediately re-enqueued and can be retried in a tight loop. There is no exponential or fixed delay between attempts.
• No task priority -- all tasks share a single FIFO queue. There is no mechanism for high-priority tasks to skip ahead.
• No scheduled/delayed tasks -- tasks are processed as soon as they are enqueued. There is no support for "run at time X" or "run after delay Y".
• Single queue only -- all task types go through one queue. A slow handler blocks capacity for all other task types.
• No persistence guarantees -- if the worker crashes mid-processing, tasks in the :processing queue are orphaned. There is no recovery or reaper process.
• No task status tracking -- there is no way to query whether a specific task succeeded, failed, or is still pending.
• Client connects to localhost only -- the client binary hardcodes a localhost Redis connection, making it Docker-unaware.
• Errors during JSON marshal/unmarshal are silently ignored -- malformed tasks will cause undefined behavior.
• No graceful shutdown -- workers run in an infinite loop with select {}. There is no signal handling or drain mechanism.

Possible Improvements

• Add exponential backoff for retries using Redis sorted sets (ZADD with a future timestamp)
• Implement a stale task reaper that moves orphaned items from :processing back to the main queue
• Add multiple named queues with per-type routing and priority levels
• Support delayed/scheduled task execution
• Add a task status API (e.g. GET /tasks/:id) backed by Redis hashes
• Implement graceful shutdown with os.Signal handling and in-flight task draining
• Add structured logging (e.g. slog or zerolog) instead of fmt.Printf
• Use connection pooling and configure Redis timeouts/retry options
• Add authentication/TLS for Redis connections in production
• Build a small web dashboard or CLI tool for inspecting the dead-letter queue