Federation causes unbound connection increase in some cases

[v0lat1le]

Describe the bug

Hi Team,

We have an RPC setup (similar to https://github.com/rabbitmq/rabbitmq-tutorials/blob/main/java/RPCClient.java) where we create a new reply queue (classic, not server-named, non-durable, exclusive, auto-delete) for every request to our server.
Recently we've tried to setup federation on those queues. Functionally it appears to work very well, but we've noticed that the connection count (and memory usage) keeps growing. The queues do get removed at the end of the request as expected.

You can clearly see from the graphs below where we enabled the federation

Is this an issue with our setup or could it be an issue with federation?

We're using RabbitMQ 4.1.1 Erlang 27.3.4.1 on two 3-node clusters. Happy to provide additional info.

Reproduction steps

To be honest, I haven't had a chance yet to try it in a more controlled environment. Will try it ASAP.

1. Setup queue federation between two clusters
2. Have an application continuous create and queues matching the federation policy
3. Observe connection count and memory usage

Expected behavior

Expecting additional resource usage because of federation to be proportional to the number of active federated queues/links

Additional context

No response

[michaelklishin]

@v0lat1le I completely fail to understand why RabbitMQ is always asked to solve problems caused by applications or truly questionable policies.

Federation links do have a protection mechanism against resource leaks: every N seconds (30 or so by default), a link will check if the entity it federates is still present on the upstream and if it's not, the link will voluntarily stop.

I also completely fail to see why are inherently temporary queues in your example are federated. They should not be. They should not even be replicated in 9 cases out of 10.

If you don't trust your applications, monitor connection churn and use per-vhost limits but better yet, only federate long-lived queues that are not churned.