Reliable delivery for events committed to Postgres.
An open-source transactional outbox relay written in Go.
Ferry carries events committed alongside application data to workers, internal services, and webhooks, with durable delivery tracking, retries, dead-letter handling, and replay.
Ferry is currently pre-alpha. The repository contains the first Phase 1 slice: a versioned PostgreSQL schema, infrastructure-independent domain vocabulary, schema verification, and atomic outbox materialization. There is no server, delivery claiming or execution, retry runtime, or operational API. Proposed contracts may still change. Ferry is not production-ready.
An application often needs to commit business data to PostgreSQL and publish an event. Those operations are not atomic when the database and delivery system are separate.
For example, an order may commit successfully before the application crashes while publishing order.created. The order exists, but payment, email, analytics, or fulfilment never starts. Publishing first has the opposite risk: downstream work may begin for data that never commits.
Ferry uses the transactional outbox pattern. The application writes its business data and an outbox event in the same PostgreSQL transaction. After commit, Ferry discovers the event and creates durable delivery work for configured destinations.
Application transaction
|
| writes business data and an outbox event
v
PostgreSQL
|
| committed outbox rows
v
Ferry server
|
| creates and schedules deliveries
|
+----> Webhook destinations
+----> Internal HTTP services
+----> Pull-based workers (planned later)
PostgreSQL is the source of truth in version one. Ferry will use PostgreSQL-backed leases or equivalent durable coordination instead of introducing a separate replicated log. LISTEN/NOTIFY may later reduce discovery latency, but notifications will never replace recovery from durable PostgreSQL state.
The application inserts the outbox event in the same transaction as its business change:
BEGIN;
INSERT INTO orders (id, customer_id, total)
VALUES ('order_123', 'customer_456', 4999);
INSERT INTO ferry.outbox_events (
event_id,
event_type,
aggregate_id,
destination_ids,
occurred_at,
payload
)
VALUES (
'evt_123',
'order.created',
'order_123',
ARRAY['payments', 'notifications'],
transaction_timestamp(),
'{"order_id":"order_123","total":4999}'::jsonb
);
COMMIT;Migration 1 implements this insert contract. The broader proposed architecture may still change pending owner review.
The planned version-one flow is:
- Discover a committed outbox event idempotently.
- Atomically create one durable delivery for each destination ID committed with the event.
- Claim an eligible delivery with a PostgreSQL lease and fencing token and create its delivery attempt.
- Send the event to its destination outside the database transaction.
- Record success, or retry transient failures with bounded exponential backoff and jitter.
- Move a permanently failing delivery to a dead-letter state.
- Allow an operator to inspect and replay the dead-letter delivery as a new linked delivery.
Ferry targets at-least-once delivery semantics, not end-to-end exactly-once processing. A destination can process a request even if its acknowledgement is lost, causing Ferry to retry. Destinations must be idempotent and should use stable event and delivery IDs to deduplicate requests. Because retries are bounded, Ferry promises durable attempts and a visible success or dead-letter outcome—not that every destination will eventually accept every event.
Only the Phase 1 foundation and atomic materialization work unit are implemented. The planned version-one scope is:
- Outbox and delivery: PostgreSQL transactional outbox ingestion, stable event IDs, idempotent discovery, HTTP webhook destinations, durable deliveries and delivery attempts, request timeouts, bounded retry policies, exponential backoff with jitter, dead-letter handling, replay, and HMAC webhook signatures.
- Runtime and operations: per-destination concurrency limits, bounded internal queues, graceful shutdown, process-termination recovery, health and readiness endpoints, Prometheus-compatible metrics, structured logging, and configuration through a file and environment variables.
- Packaging and testing: a Go CLI and server binary named
ferry, Docker support, integration tests against a real PostgreSQL instance, and failure tests for crashes, retries, timeouts, and duplicate processing.
- Delivery uses at-least-once semantics with bounded attempts and a durable terminal outcome; exactly-once processing is not promised.
- Durable event and delivery state lives in PostgreSQL.
- Outbox discovery must be idempotent, and replay must be explicit and traceable.
- Destinations remain responsible for idempotent processing.
- Version one is directed at a single Ferry server process. PostgreSQL coordinates durable state; multi-process coordination comes later.
- There is no global ordering guarantee. Scoped ordering by destination or aggregate key may be considered later and would require an explicit contract.
- Missed notifications, restarts, and expired leases must be recoverable from PostgreSQL.
See the architecture overview for lifecycle and failure-boundary details.
Ferry version one is not intended to be:
- a Kafka-compatible broker or general-purpose streaming platform;
- a workflow orchestration engine;
- a database or an exactly-once processing system;
- a hosted control plane or multi-region event platform;
- a frontend application or dashboard;
- dependent on Kafka, Redis, NATS, Kubernetes, or a workflow DSL.
Operational access will come through HTTP endpoints, metrics, structured logs, a CLI, and PostgreSQL inspection where appropriate. A small read-only dashboard may be considered later, but it is not part of the core product.
Ferry is planned as a backend-only Go service. Go fits a long-running daemon with concurrent delivery workers, PostgreSQL coordination, HTTP APIs and webhook delivery, straightforward operations, a single deployable binary, and readable integration and failure tests.
The project establishes these boundaries:
cmd/ferry/ future server and CLI entry point
internal/app/ composition, supervision, and shutdown
internal/cli/ command parsing and service calls
internal/config/ file and environment configuration
internal/domain/ shared infrastructure-independent values
internal/postgres/ durable state and coordination
internal/outbox/ atomic event materialization
internal/delivery/ claiming, attempts, and delivery state
internal/destination/ destination adapters
internal/retry/ retry classification and timing
internal/telemetry/ logs and metrics
internal/api/ operational HTTP endpoints
migrations/ versioned PostgreSQL schema changes
tests/integration/ real PostgreSQL and HTTP integration tests
tests/failure/ crash and delivery-boundary failure tests
The domain, outbox, and PostgreSQL packages implement the first Phase 1 slice; the other directories remain package documentation. Package boundaries should evolve only when implementation and tests demonstrate a need.
- Phase 1 — PostgreSQL schema, atomic outbox materialization, and fenced delivery claims.
- Phase 2 — Webhook delivery plus attempts, retries, signatures, bounded concurrency, health, shutdown, and crash recovery.
- Phase 3 — Dead-letter handling, authenticated replay, metrics, and CLI operations.
- Phase 4 — Scheduling fairness, retention, secret rotation, operator security, and operational hardening.
- Phase 5 — Optional pull consumers and horizontal worker coordination.
The first usable release is expected after the single-node webhook path in Phase 2. See the detailed roadmap.
The Go module uses path github.com/op-q/ferry, targets Go 1.26, and uses
pgx/v5 for native PostgreSQL access. The current ferry command intentionally
exits with an error explaining that runtime commands are not implemented.
With Go 1.26 or later installed:
go test ./...
go build -o ./ferry ./cmd/ferryUnit and PostgreSQL-backed repository tests are documented in
tests/integration. These commands do not start
a Ferry server.
The intended repository is op-q/ferry, and
the implementation language is Go.
- Product vision — users, promise, scope, and success criteria.
- Architecture overview — lifecycles, delivery semantics, recovery, and open decisions.
- Data model — durable records, state transitions, and invariants.
- Runtime architecture — Go dependencies, workers, health, and shutdown.
- HTTP delivery contract — webhook envelope, signatures, retries, and limits.
- Architecture decisions — accepted constraints and proposed version-one choices.
- Roadmap — milestones toward a usable single-node service.
- Contributor and agent guidance — public-repository hygiene and project constraints.
- Security policy — responsible reporting without exposing sensitive material.
Ferry is available under the MIT License.