New abstractions for connection and connection pools.

[ikhoon]

• ConnectionPool.maxNumConnections() will limit the maximum connection more intuitively than maxNumEventLoopsPerEndpoint.
  • For HTTP/2, maxNumConnections() will default to 1 or 2.
  • For HTTP/1, Integer.MAX_VALUE can be the sensible default value.
  • The user will be able to set the desired number of connections through the connection pool more easily.
  • Devs may not have to explain maxNumEventLoopsPerEndpoint on Discord channels and how to control the maximum number of connections.

public interface ConnectionPool extends SafeCloseable {

  @Nullable
  Connection acquire(ClientRequestContext ctx);

  void release(Connection connection);

  void add(Connection connection);

  void remove(Connection connection);

  int maxNumConnections();
}

public class Connection {
  private SessionProtocol protocol;
  private Endpoint endpoint;
  private Channel channel;
  private EventLoop eventLoop;
}

• LoadBalancer<Connection, ClientRequestContext> may be used to pick a connection in a connection pool.
  • A new LoadBalancer to pick a connection that has the least requests could be implemented later.

  ConnectionPool.builder()
                .loadBalancerFactory(connections -> LoadBalancer.ofSticky(connections, ...))
                .maxNumConnections(1)
                ...
  ClientFactory.builder()
               .http1ConnectionPoolFactory(... -> newConnectionPool())
               .http2ConnectionPoolFactory(... -> newConnectionPool())

• After an Endpoint is acquired, the Endpoint is used as a key to a ConnectionPool for the endpoint.

  Map<Endpoint, ConnectionPool> pools = new ConcurrentHashMap<>();
  ConnectionPool pool = pools.get(ctx.endpoint());
  Connection connection = pool.acquire(ctx);
  if (connection == null) {
    // create a new connection using Bootstrap and add it to the connection pool
  }

I will do more PoC on a separate branch and leave feedback on whether this proposal is possible without making significant changes to the current implementation.

Ref: #5779 (comment)