Cannot multiplex requests on a single H2 connection

[nicolas-grekas]

Spotted at symfony/symfony#35115 (comment)

The following script adapted from the examples doesn't multiplex the requests.
Note that we limit to one single connection. If we remove this limit, one connection per request is made and all works concurrently - but then this is no different than using HTTP/1.1.

<?php

use Amp\Http\Client\Connection\LimitedConnectionPool;
use Amp\Http\Client\Connection\UnlimitedConnectionPool;
use Amp\Http\Client\HttpClientBuilder;
use Amp\Http\Client\HttpException;
use Amp\Http\Client\Request;
use Amp\Http\Client\Response;
use Amp\Loop;
use Amp\Sync\LocalKeyedSemaphore;

require __DIR__ . '/vendor/autoload.php';

Loop::run(static function () use ($argv) {
    $uris = [];

    for ($i = 0; $i < 379; ++$i) {
        $uris[] = "https://http2.akamai.com/demo/tile-$i.png";
    }

    // Instantiate the HTTP client
    $pool = LimitedConnectionPool::byHost(new UnlimitedConnectionPool, new LocalKeyedSemaphore(1));
    $client = (new HttpClientBuilder)
            ->usingPool($pool)
            ->build();

    $requestHandler = static function (string $uri) use ($client) {
        /** @var Response $response */
        $response = yield $client->request(new Request($uri));

        return yield $response->getBody()->buffer();
    };

    try {
        $promises = [];

        foreach ($uris as $uri) {
            $promises[$uri] = Amp\call($requestHandler, $uri);
        }

        $bodies = yield $promises;

        foreach ($bodies as $uri => $body) {
            print $uri . " - " . \strlen($body) . " bytes" . PHP_EOL;
        }
    } catch (HttpException $error) {
        // If something goes wrong Amp will throw the exception where the promise was yielded.
        // The HttpClient::request() method itself will never throw directly, but returns a promise.
        echo $error;
    }
});

[kelunik]

Indeed, currently it limits concurrent requests instead of concurrent connections. I guess having the same connection limit for both isn't what users generally want.

[nicolas-grekas]

having the same connection limit for both isn't what users generally want.

sure! Note that this is not my issue here :) Is there a way to multiplex several requests on a single h2 connection? That's what I don't know how to do here. Is that a known limitation?

[kelunik]

Sure, that will happen by default, but if you limit to a single concurrent request, it will obviously not happen.

[nicolas-grekas]

But it should be possible to have one single connection to a remote host and still do several h2 requests, since that's a main feature of the protocol. curl allows it.
Right now, multiplexing works by creating one socket per request, isn't it? Maybe I miss something :)

[trowski]

H2 connections can have many streams per connection. Only a single connection is actually made.

The problem is that LimitedConnectionPool is too naïve and is allowing only a single request, even if the connection being used is H2.

@kelunik I think we should change LimitedConnectionPool to store the connections returned and attempt to reuse them for subsequent requests, only acquiring a lock and waiting if Connection::getStream() returns null.

[kelunik]

The way it's currently written, it could have been a simple interceptor, I guess.

@trowski I don't see a way to implement that in the current implementation, because we don't ever have access to the connection object in LimitedConnectionPool. I'd have to be a separate implementation.

[trowski]

Oh yes, right… it's only returning Stream objects. Yep, then it will have to be a whole new, separate implementation.

[nicolas-grekas]

This is the callgraph of the above script without the LimitedConnectionPool.

https://blackfire.io/profiles/20a80a98-340c-4327-b966-4c9db0c4f679/graph

As you can see, it opens several connections to the host.
stream_socket_enable_crypto is called 845 times, which makes it the most exclusive-time consuming function, and ResourceSocket::__construct is called 280 times.

I'll have a closer look, but if you have some hints meanwhile, I'd be happy to read them.

[nicolas-grekas]

OK, here is what happens: the script requests more than the maximum number of allowed concurrent streams.

This means $stream = yield $connection->getStream($request); returns null in UnlimitedConnectionPool, so that a second connection is opened once the limit is reached.
Then, all next requests open a separate connection, because $isHttps && \count($connections) === 1 is false as of now.

Here is the Blackfire profile that shows only one connection is opened:
https://blackfire.io/profiles/4b7fe9f4-c427-45fb-9b4d-ce0d67881d4f/graph

I suppose the pool logic could be improved :)

[trowski]

Thanks for figuring that out… I was scratching my head for the last half hour trying to figure out why the H2 connection wasn't being reused. Can you try removing && \count($connections) === 1. That should improve things quite a bit.

[nicolas-grekas]

Can you try removing && \count($connections) === 1.

No difference, it still opens new connections after the threshold.
Please run the script if you want to verify :)

[trowski]

Tried the script – it only used a single connection, but it also seemed to only be making one request at a time. Is there something I need to do so all requests are sent at once?

EDIT: I was trying the script you have on the symfony PR, not the one above. Let me investigate further.

[trowski]

@nicolas-grekas Just pushed 1181944, which should fix your problem. Please pull the latest master and give it a try. 😃

[kelunik]

@trowski We can't leave 1181944 as is, because that will make the performance for HTTP/1.1 much worse, because every request needs to await any currently in progress connections. I will have to think about the optimal algorithm there.

[nicolas-grekas]

Just pushed 1181944, which should fix your problem

Confirmed! That's the right direction apparently :)

[trowski]

@kelunik Yes, I realized that, but I wasn't sure about what a good solution there would be and I wanted to confirm that at least solved @nicolas-grekas's problem.

First idea I had is examine the first connection returned, and if HTTP/1.x, assume that the next one will also be HTTP/1.x and immediately make a new connection.

[kelunik]

Closing, as this has been fixed on master.

[nicolas-grekas]

Thank you!
Should I open a separate issue about LimitedConnectionPool?

[kelunik]

Yup, please.