Persistence and Durable Delivery

Persistence and Durable Delivery

By default a message published while its subscriber is offline is lost. Durable delivery stores it until the subscriber returns.

Storage backends

new Server({
  storage: 'memory',               // default — in-process, no restart survival
  storage: 'sqlite',               // single file, zero infrastructure
  storage: 'redis',                // shared store; required for clustering
  storage: require('./my-store'),  // custom implementation
  storage_options: { /* backend specific */ }
})

Backend	Options	Notes
memory	default_ttl	survives reconnects, not restarts
sqlite	filename, default_ttl	persistent single-file store
redis	url, prefix, default_ttl, client (injectable)	shared across nodes
custom	module (path)	implement the store interface below

The backend can be hot-swapped at runtime with the set_persistence management command or the web UI's Persistence tab.

Store interface

store.enqueue(realm, event, message)    // → Promise<msgId>
store.dequeue(realm, event, consumer)   // → Promise<Message[]>  undelivered for this consumer
store.ack(msgId)                        // → Promise<void>
// reliability extensions:
store.deadLetter(msgId, reason)         // move to the DLQ
store.listDlq(realm)                    // → Promise<Entry[]>
store.discardDlq(msgId[, realm])        // drop from the DLQ

Opting in

Durability is per subscription:

consumer.subscribe('tasks', 'job', handler, {
    durable: true,
    consumer_id: 'worker-7'        // stable identity, not the transient socket id
});

Or per message from the producer, regardless of how subscribers subscribed:

producer.produce('job', data, { guaranteed: true });

Offline delivery on reconnect

1. The consumer reconnects and re-subscribes (the client library does this automatically) with the same consumer_id.
2. The server dequeues that consumer's undelivered messages for the subscribed events and replays them in order.
3. With ACK enabled, each replayed message is tracked; without ACK, replay uses the legacy immediate-ack behavior (delivered once, then removed).

Message TTL

Every enqueued message has a TTL; expired messages are purged undelivered.

producer.produce('job', data, { ttl: 3600 });            // per message, seconds
mq.createProducer('tasks', { default_ttl: 3600 });       // per producer
new Server({ storage_options: { default_ttl: 86400 } }); // backend default (24 h)

Topic subscriptions

Durable delivery composes with topic mode: a { mode: 'topic', durable: true } subscription queues matching messages while offline and replays them on reconnect. See Topics and Wildcards.

In a cluster

With storage: 'redis' shared by all nodes, a durable consumer can disconnect from node A and reconnect to node B — its queued messages replay from the shared store. See Clustering for the delivery-semantics details (at-least-once when switching nodes).