Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Messenger] RabbitMQ - using DelayStamp produces queue which will not be removed #32588

Closed
radol91 opened this issue Jul 17, 2019 · 2 comments
Closed

Comments

@radol91
Copy link

radol91 commented Jul 17, 2019

Symfony version(s) affected: 4.3.2

Description
When using the DelayStamp in AMPQ transport the method createDelayQueue (in AmqpExt/Connection) creates a temporary queue. The attribute x-message-ttl is set to some value. After the delay time, the message goes to the target queue (it is transported by some internal mechanism (?), not by a consumer, so there is no consumer attached to that queue). As a result the temporary queue stays empty and it is not going to be removed. According to AMPQ documentation queue is removed by setting attribute auto-delete:true only if it had at least one consumer:

An auto-delete queue will be deleted when its last consumer is cancelled (e.g. using the basic.cancel in AMQP 0-9-1) or gone (closed channel or connection, or lost TCP connection with the server).

If a queue never had any consumers, for instance, when all consumption happens using the basic.get method (the "pull" API), it wont' be automatically deleted. For such cases, use exclusive queues or queue TTL.

How to reproduce
Dispatch a message using DelayStamp and AMPQ transport, check existing queues i.e in RabbitMQ.

Solution
The solution is to set additionally x-expires parameter for delay queue in createDelayQueue method. It ensures that the temporary queue is going to be removed and prevents from creating hundreds of zombie queues.

        $queue->setArguments([
            'x-message-ttl' => $delay,
            'x-expires' => $delay * self::DELAY_MULTIPLIER,
            'x-dead-letter-exchange' => $this->exchangeOptions['name'],
            // after being released from to DLX, make sure the original routing key will be used
            // we must use an empty string instead of null for the argument to be picked up
            'x-dead-letter-routing-key' => $routingKey ?? '',
        ]);

Additional context
n/a

@Tobion
Copy link
Member

Tobion commented Jul 17, 2019

LGTM. While at it, we should also make sure the delay exchange declaration is part of the setup (#29476) and the queue declaration is independing of auto_setup. The delay queue always need to be declared because the name is dynamic and cannot be declared in advance.

@Tobion
Copy link
Member

Tobion commented Jul 17, 2019

Also we don't need the routing_key_pattern option and can use the queue_name_pattern for this directly.

Tobion added a commit that referenced this issue Jul 28, 2019
…obion)

This PR was merged into the 4.3 branch.

Discussion
----------

[Messenger] expire delay queue and fix auto_setup logic

| Q             | A
| ------------- | ---
| Branch?       | 4.3
| Bug fix?      | yes
| New feature?  | yes
| BC breaks?    | no     <!-- see https://symfony.com/bc -->
| Deprecations? | no <!-- please update UPGRADE-*.md and src/**/CHANGELOG.md files -->
| Tests pass?   | yes    <!-- please add some, will be required by reviewers -->
| Fixed tickets | #32588
| License       | MIT
| Doc PR        |

Tested successfully

Commits
-------

7aee83a [Messenger] expire delay queue and fix auto_setup logic
@Tobion Tobion closed this as completed Aug 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants