Skip to content

Latest commit

 

History

9 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Messenger Retry Bundle

Configurable RabbitMQ retry queues for Symfony Messenger.

The bundle provides stage-based delayed retries using RabbitMQ TTL and Dead Letter Exchanges.

Instead of keeping a failed message inside the original queue or relying on Symfony's built-in retry delay, the message is moved through configurable retry queues:

original queue
    ↓ failure
retry_5s
    ↓
original queue
    ↓ failure
retry_5s
    ↓
...
retry_30s
    ↓
original queue
    ↓
...
retry_5m

Retry stages, delays and attempt counts are fully configurable.

Features

  • Symfony Messenger integration
  • RabbitMQ TTL/DLX based retries
  • Fully configurable retry stages
  • Configurable number of attempts per stage
  • Unlimited number of retry stages
  • Multiple Messenger transports
  • Multiple RabbitMQ vhosts
  • Multiple queues per vhost
  • Multiple routing keys in the same retry queue
  • Original routing key preservation
  • Automatic retry queue/exchange creation
  • Global retry counter
  • Retry metadata available as AMQP headers
  • Symfony RedeliveryStamp compatibility
  • Optional infinite retry on the last stage
  • Symfony failure transport support
  • Non-retryable exception support

Requirements

  • PHP 8.2+
  • Symfony 6.4 / 7.x / 8.x
  • Symfony Messenger
  • Symfony AMQP Messenger
  • RabbitMQ
  • PHP AMQP extension

Installation

Install the package using Composer:

composer require tifed/messenger-retry-bundle

If Symfony Flex does not register the bundle automatically, add it manually:

// config/bundles.php

return [
    // ...

    Tifed\MessengerRetryBundle\MessengerRetryBundle::class => ['all' => true],
];

Messenger configuration

RabbitMQ transports continue to be configured normally through Symfony Messenger.

The bundle does not replace or duplicate Messenger transport configuration.

Example:

# config/packages/messenger.yaml

framework:
    messenger:
        failure_transport: failed

        transports:
            some_queue:
                dsn: '%env(MESSENGER_TRANSPORT_DSN)%'

                options:
                    vhost: test

                    exchange:
                        name: messages
                        type: topic

                    queues:
                        some_queue:
                            binding_keys:
                                - '#'

                retry_strategy:
                    max_retries: 0

            failed:
                dsn: '%env(MESSENGER_FAILED_TRANSPORT_DSN)%'

        routing:
            'App\Message\TestMessage': some_queue

Important

Symfony's native retry mechanism must be disabled for transports handled by this bundle:

retry_strategy:
    max_retries: 0

Otherwise Symfony's retry mechanism and this bundle may both attempt to retry the same failed message.

Retry configuration

Retry configuration lives separately from Messenger transport configuration:

# config/packages/messenger_retry.yaml

messenger_retry:
    after_exhausted: repeat_last

    stages:
        retry_5s:
            delay: 5s
            attempts: 5

        retry_30s:
            delay: 30s
            attempts: 5

        retry_5m:
            delay: 5m
            attempts: 5

        retry_15m:
            delay: 15m
            attempts: 5

        retry_30m:
            delay: 30m
            attempts: 5

        retry_1h:
            delay: 1h
            attempts: 5

The stage list is completely dynamic.

You can configure only one stage:

messenger_retry:
    stages:
        retry_5s:
            delay: 5s
            attempts: 3

or any number of stages:

messenger_retry:
    stages:
        retry_10s:
            delay: 10s
            attempts: 3

        retry_1m:
            delay: 1m
            attempts: 10

        retry_1h:
            delay: 1h
            attempts: 5

The order in YAML defines the retry order.

Stage names are not hardcoded by the bundle.

Delay format

Supported units:

ms  milliseconds
s   seconds
m   minutes
h   hours
d   days

Examples:

delay: 500ms
delay: 5s
delay: 30s
delay: 5m
delay: 1h
delay: 1d

How retries work

Assume the following configuration:

messenger_retry:
    stages:
        retry_5s:
            delay: 5s
            attempts: 3

        retry_30s:
            delay: 30s
            attempts: 2

        retry_5m:
            delay: 5m
            attempts: 1

When message processing fails:

failure
  ↓
retry_5s / attempt 1
  ↓ 5 seconds
original queue

failure
  ↓
retry_5s / attempt 2
  ↓ 5 seconds
original queue

failure
  ↓
retry_5s / attempt 3
  ↓ 5 seconds
original queue

failure
  ↓
retry_30s / attempt 1
  ↓ 30 seconds
original queue

failure
  ↓
retry_30s / attempt 2
  ↓ 30 seconds
original queue

failure
  ↓
retry_5m / attempt 1
  ↓ 5 minutes
original queue

What happens after the final stage depends on after_exhausted.

after_exhausted

Three modes are available.

failure_transport

messenger_retry:
    after_exhausted: failure_transport

After all configured retry stages are exhausted, normal Symfony failure transport handling takes over.

discard

messenger_retry:
    after_exhausted: discard

The message is discarded after all retry stages have been exhausted.

repeat_last

messenger_retry:
    after_exhausted: repeat_last

After the configured attempts of the last stage are exhausted, the message continues retrying using the last stage indefinitely.

For example:

messenger_retry:
    after_exhausted: repeat_last

    stages:
        retry_5s:
            delay: 5s
            attempts: 5

        retry_1h:
            delay: 1h
            attempts: 5

results in:

retry_5s × 5
    ↓
retry_1h × 5
    ↓
retry_1h
    ↓
retry_1h
    ↓
retry_1h
    ↓
...

The message remains in the retry cycle until it is successfully processed.

This is useful for messages that must eventually be delivered and must not disappear because a maximum retry count was reached.

RabbitMQ topology

Retry topology is created automatically.

For an original exchange:

messages

and stage:

retry_5s

the bundle creates:

queue:
messages.retry_5s

exchange:
messages.retry_5s.exchange

The retry queue contains:

x-message-ttl = 5000
x-dead-letter-exchange = messages

The bundle intentionally does not set:

x-dead-letter-routing-key

This allows RabbitMQ to preserve the routing key used when the message was published to the retry exchange.

Routing keys

Messages that may be retried are expected to have a RabbitMQ routing key.

The original routing key is preserved during the complete retry cycle.

For example:

messages exchange

routing key:
payment.created

On failure:

payment.created
      ↓
messages.retry_5s.exchange
      ↓
messages.retry_5s

After TTL expiration:

messages.retry_5s
      ↓
messages
      ↓
payment.created

The message can therefore be routed back to the correct original queue.

Multiple routing keys

A single retry queue may contain messages with different routing keys.

For example:

messages.retry_5s

├── payment.created
├── payment.failed
├── notification.email
└── notification.sms

The retry exchange uses a topic binding with:

#

Each message retains its own routing key.

After expiration, RabbitMQ dead-letters the messages back to the original exchange using their respective routing keys.

Multiple exchanges

Retry queues are separated by original exchange.

For example:

messages.retry_5s
payments.retry_5s
notifications.retry_5s

This is required because x-dead-letter-exchange is a queue-level RabbitMQ setting.

A single physical retry queue cannot safely target multiple dead-letter exchanges.

Multiple vhosts

Retry topology is created inside the same RabbitMQ vhost as the original Messenger transport.

For example:

framework:
    messenger:
        transports:
            some_queue:
                dsn: '%env(MESSENGER_TRANSPORT_DSN)%'

                options:
                    vhost: test

will create retry queues inside:

vhost: test

The bundle inherits AMQP connection options from the original Messenger transport, including the configured vhost.

Therefore the same retry stage may independently exist in multiple vhosts:

vhost test:
    messages.retry_5s

vhost payments:
    messages.retry_5s

vhost notifications:
    messages.retry_5s

They are completely independent RabbitMQ queues.

Retry state

The bundle tracks retry state using RetryStateStamp.

It contains:

originalTransport
stageIndex
attemptInStage
retryCount

Example:

originalTransport = some_queue
stageIndex         = 1
attemptInStage     = 2
retryCount         = 7

retryCount is global across all retry stages.

For example:

retry_5s attempt 1  → retryCount 1
retry_5s attempt 2  → retryCount 2
retry_5s attempt 3  → retryCount 3

retry_30s attempt 1 → retryCount 4
retry_30s attempt 2 → retryCount 5

retry_5m attempt 1  → retryCount 6

Symfony RedeliveryStamp

The bundle also adds Symfony's RedeliveryStamp.

This means existing code can continue using:

use Symfony\Component\Messenger\Stamp\RedeliveryStamp;

$retryCount = RedeliveryStamp::getRetryCountFromEnvelope($envelope);

without knowing about the bundle's internal retry implementation.

RabbitMQ headers

Retry information is also duplicated into AMQP headers for observability and debugging.

Example:

x-retry-count: 7
x-retry-stage: retry_30s
x-retry-stage-attempt: 2
x-original-transport: some_queue

This makes retry state visible directly from RabbitMQ Management UI and available to non-Symfony consumers.

The internal retry state remains the source of truth; these headers are intended for inspection, logging and monitoring.

Non-retryable exceptions

Some failures should not enter the retry cycle.

Symfony's unrecoverable exceptions are respected.

You can also mark your own exceptions as non-retryable:

use Tifed\MessengerRetryBundle\Exception\NonRetryableExceptionInterface;

final class InvalidPaymentException extends \RuntimeException implements
    NonRetryableExceptionInterface
{
}

Throwing this exception from a Messenger handler skips the bundle retry mechanism.

Normal Symfony failure handling can then take over.

Example handler

namespace App\MessageHandler;

use App\Message\TestMessage;
use Symfony\Component\Messenger\Attribute\AsMessageHandler;

#[AsMessageHandler]
final class TestMessageHandler
{
    public function __invoke(TestMessage $message): void
    {
        throw new \RuntimeException('Temporary failure');
    }
}

With:

messenger_retry:
    stages:
        retry_5s:
            delay: 5s
            attempts: 3

        retry_30s:
            delay: 30s
            attempts: 5

the failed message automatically moves through the configured retry stages.

No retry-specific code is required in the handler.

Running workers

Workers are started normally:

php bin/console messenger:consume some_queue

No separate worker is required for retry queues.

RabbitMQ handles retry delays using TTL and DLX.

The worker only consumes the original Messenger queues.

Why RabbitMQ TTL/DLX?

Symfony can implement delayed retries by scheduling messages through transports.

This bundle instead delegates the waiting period to RabbitMQ.

A retry queue has:

TTL
 ↓
DLX
 ↓
original exchange

Advantages include:

  • no sleeping workers;
  • retry messages remain inside RabbitMQ;
  • no dedicated retry workers;
  • retry queues are visible in RabbitMQ Management;
  • retry pressure is isolated from the original queue;
  • stage behaviour is explicit and configurable.

Delivery semantics

The bundle provides at-least-once delivery semantics.

As with RabbitMQ and Symfony Messenger in general, duplicate delivery is possible, especially around worker or network failures.

Message handlers should therefore be idempotent whenever possible.

For example, avoid:

$payment->charge();

without protection against processing the same message twice.

Prefer application-level idempotency using message IDs, database constraints, processed-message records or another appropriate mechanism.

Configuration reference

Minimal configuration:

messenger_retry:
    stages:
        retry_5s:
            delay: 5s
            attempts: 5

Full example:

messenger_retry:
    after_exhausted: repeat_last

    stages:
        retry_5s:
            delay: 5s
            attempts: 5

        retry_30s:
            delay: 30s
            attempts: 5

        retry_5m:
            delay: 5m
            attempts: 5

        retry_15m:
            delay: 15m
            attempts: 5

        retry_30m:
            delay: 30m
            attempts: 5

        retry_1h:
            delay: 1h
            attempts: 5

Production checklist

Before using the bundle in production:

✓ Native Messenger retry disabled for managed transports
✓ RabbitMQ transport uses the expected vhost
✓ Original exchange/routing configuration is correct
✓ Messages have routing keys
✓ RabbitMQ user can declare exchanges and queues
✓ RabbitMQ user can publish to retry exchanges
✓ Failure transport is configured if failure_transport mode is used
✓ Message handlers are idempotent

License

MIT

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages