#1230 Disposing ConnectionPool that are not actively used under LRU policy.

[jun-lee]

Based on the issue(#1230), I'm thinking that it can be on the reactor-netty application layer as well, instead of adding consumer logic into the reactor-pool.

[pivotal-issuemaster]

@jun-lee Please sign the Contributor License Agreement!

Click here to manually synchronize the status of this Pull Request.

See the FAQ for frequently asked questions.

[pivotal-issuemaster]

@jun-lee Thank you for signing the Contributor License Agreement!

[violetagg]

@jun-lee With issue #1230 do you want to remove the connections that reached their TTL within a connection pool or to remove a connection pool from the map with all connections pool?

The feature request in Reactor Pool is for removing the connections that reached their TTL within a connection pool. While with this PR you want to remove a connection pool from the map with all connection pools.

How do you guarantee that the connection pool, that's going to be destroyed, does not have active connections at the moment?

[jun-lee]

@violetagg
Because I don't see how the resources referenced by PoolKey can be garbage collected unless the entry is removed from the map after it has established. Let's say a ConnectionPool is holding 10 subsequent connections that reached to TTL so that a process of reactor-pool evicted all the connections belong to the Connection Pool, so now we have empty ConnectionPool on the channelPools Map. However, the PoolKey and cached resource in the Connection Pool that I shared the Screenshot on the issue would not be removed.

In my application, if I'm using enough numbers for the limit, removing entry by LRU policy is a kind of the desired feature, rather than keeping them a long time or causing another heavy evicting operation that checks that all the connections in a pool have reached to TTL. I have tested that the simultaneous new request that establishing a new ConnectionPool to the destroying connection pool was not causing any problem either.

[violetagg]

@violetagg
Because I don't see how the resources referenced by PoolKey can be garbage collected unless the entry is removed from the map after it has established. Let's say a ConnectionPool is holding 10 subsequent connections that reached to TTL so that a process of reactor-pool evicted all the connections belong to the Connection Pool,

We still do not have this background eviction from Reactor Pool, so currently the connections are checked for TTL only when they are released/acquired.

so now we have empty ConnectionPool on the channelPools Map. However, the PoolKey and cached resource in the Connection Pool that I shared the Screenshot on the issue would not be removed.

Let say that the connection pool is empty then the cleanup from the map should not be done based on the eldest but based on InstrumentedPool.metrics().allocatedSize() == 0

In my application, if I'm using enough numbers for the limit, removing entry by LRU policy is a kind of the desired feature, rather than keeping them a long time or causing another heavy evicting operation that checks that all the connections in a pool have reached to TTL. I have tested that the simultaneous new request that establishing a new ConnectionPool to the destroying connection pool was not causing any problem either.

Let say that you destroy a connection pool with connections (active or idle). Then you are right this will not cause problems as the next request for a connection will create it again, BUT then you have to pay the price for establishing again a new connection to the remote host.

What if your eldest connection pool is the most heavy used at the moment?

[jun-lee]

Yeah this PR is not supposed to be merged. We need to hit the map by calling get() on the acquiring Operation as well to put the most heavy one at the recently accessed position. I just want to point that the root cause of my application was the growing entities of the map holding the connection pools. If there is any way to clean up the connection pool and related PoolKey that holding InetAddress of the remote host, then it would work for my application.

[violetagg]

@jun-lee So from what I understood you connect to many different remote hosts, is this correct?
We provide this method reactor.netty.resources.ConnectionProvider#disposeWhen will it help?

[jun-lee]

Yes, that's what I have just found and I now can handle the LRU from my applicaion side. And it works well. Thanks for the help.