queue-lab

A queue benchmark harness: the same notification fan-out workload run through multiple queue backends behind one explicit messaging contract, compared on latency, throughput, loss, backlog, retry, ordering, and failure behavior from real runs.

See FINDINGS.md for the measured results (the headline: under a slow consumer, Redis Pub/Sub drops ~93% of deliveries while Kafka loses none, holding them as backlog), and docs/diagrams/architecture.md for diagrams of the contract as coded and the target fan-out pipeline.

What it is

queue-lab models a notification fan-out pipeline — ingest an event, expand it into one delivery per subscriber, deliver each — and lets you run that identical pipeline on top of different queue technologies without changing the application code. Only the messaging adapter changes per backend, so the differences you observe are the queues' own, not the app's.

The deliverable is not a large codebase; it is a defensible set of measured findings: comparison tables, failure write-ups, and design notes derived from real runs. It answers two questions with evidence rather than folklore:

• Which queue for which workload — when Kafka, Redis Pub/Sub, RabbitMQ, SQS, Kinesis, or Google Pub/Sub is the right tool, based on behavior under the same load.
• How a pipeline behaves under stress — slow consumers, burst traffic, celebrity fan-out, retries, duplicate delivery, dead-lettering, and backend-specific throttling.

Backends compared

Backend	Main lesson
Kafka	Durable log, partitioning, replay, consumer-group lag, transactional consume-transform-produce
Redis Pub/Sub	Best-effort push with no durable backlog — where messages are lost under overload
RabbitMQ	AMQP queues, prefetch backpressure, redelivery, routing
SQS	Pull-based queue, visibility timeout, redelivery, approximate backlog, request cost
Kinesis	Managed stream, shard limits, iterator age, throttling
Google Pub/Sub	Ack deadlines, redelivery, managed pull subscription behavior

How it works

A few load-bearing decisions keep the comparison honest:

• One workload, one contract. The application path (ingestion → fan-out → delivery) is identical for every backend; only the MessageBus implementation is swapped. No backend gets a different workload.
• Two intentional bus crossings. An events topic decouples ingestion from fan-out, and a deliveries topic decouples fan-out from delivery. This makes celebrity fan-out (one event → thousands of deliveries) visible without blocking ingestion.
• Measure loss from the edge. Every event and delivery carries a stable id, so loss and duplicates are computed from expected-versus-observed delivery ids in a result ledger — not from a backend's own counters.
• At-least-once with idempotent delivery. A deterministic idempotency key lets workers suppress duplicate side effects without hiding genuine loss. The harness never claims end-to-end exactly-once; native transactions help inside a backend, external effects still need idempotency.
• Promises versus measurements. What a backend claims (ordering, durability, replay, dedup) and what it does under load are recorded as two separate artifacts.

Load scenarios

Scenario	Question it answers
Baseline	Does the path work and stay stable at a steady rate?
Celebrity fan-out	Where does fan-out or delivery saturate when one event explodes into many?
Burst	Does the backend buffer, throttle, or drop a sudden spike?
Slow consumer	Does backlog grow, does redelivery kick in, or does loss appear?

Scope

This is a measurement harness, not a product: no application frontend (Grafana and command-line reports are enough), no production hardening (auth, multi-tenancy, SLAs are out of scope), and multi-region behavior — if built — is simulated with logical regions and fault injection.

Status

The fan-out pipeline (ingestion → fan-out → delivery) runs on one MessageBus contract across an in-memory bus, Redis Pub/Sub, and Kafka (raw producer/consumer), with loss measured from the edge and a capability-gated contract test each backend must pass. A load suite drives baseline and slow-consumer scenarios and records the measured comparison in FINDINGS.md — the headline: under a slow consumer Redis drops ~93% of deliveries while Kafka loses none, holding them as backlog.

Prerequisites

• Java 17
• Docker (for the Kafka backend)

Common commands

Run the fast unit tests (no Docker needed):

./mvnw test

Run the full suite including the Testcontainers integration tests against real Redis and Kafka (requires Docker):

./mvnw verify

Run the application (exposes Actuator health and the Prometheus scrape endpoint):

./mvnw spring-boot:run
# http://localhost:8080/actuator/health
# http://localhost:8080/actuator/prometheus

Start the local Kafka broker (reachable on localhost:9092):

docker compose up -d

Reproducing the findings

The measured scenarios behind FINDINGS.md are Testcontainers integration tests that print each result as a Markdown table row. Docker must be running.

Apples-to-apples comparison — identical slow-consumer load on Redis and Kafka, measured in the same fixed time window:

./mvnw verify -Dit.test=FixedWindowComparisonIT 2>&1 | grep -E '^\| '

All load-suite rows — baseline and slow-consumer across in-memory, Redis, and Kafka:

./mvnw verify 2>&1 | grep -E '^\| (in-memory|redis-pubsub|kafka) \|'

A single backend's load suite:

./mvnw verify -Dit.test=KafkaLoadSuiteIT   # or RedisLoadSuiteIT

The printed rows are headerless; the columns are:

| backend | scenario | expected | delivered | permanentLoss | recoverableBacklog | deadLettered | p50ms | p99ms |