[HttpClient] add AsyncDecoratorTrait to ease processing responses without breaking async

[nicolas-grekas]

Q	A
Branch?	master
Bug fix?	no
New feature?	yes
Deprecations?	no
Tickets	Fix #31885, fix #32367
License	MIT
Doc PR	symfony/symfony-docs#13736

This PR allows processing the stream of chunks.

<?php
$client = new class() implements HttpClientInterface {
    use AsyncDecoratorTrait;

    public function request(string $method, string $url, array $options): ResponseInterface
    {
        return new AsyncResponse($this->client, $method, $url, $options, static function (ChunkInterface $chunk, AsyncContext $context) {

            // do what you want with chunks, e.g. split them
            // in smaller chunks, group them, skip some, etc.

            yield $chunk;
        });
    }
};

Some ideas:

• custom retry/redirect logic
• align chunk boundaries with server-sent events and yield augmented chunks that know about messages (see [HttpClient] add EventSourceHttpClient to consume Server-Sent Events #36692)
• play some OAuth dance before issuing the real request
• do some live transclusion
• be creative :)

Any custom logic should fit into the $passthru filter iterator (the last constructor argument of AsyncResponse). There, one has access to an AsyncContext DTO, which allows controlling the stream, eg. to replace the current request/response, to change the passthru filter itself, etc.

The surrounding logic will catch badly behaving filters to ease spotting some mistakes (eg. never forwarding an "isLast()" chunk, or yielding extra chunks after an "isLast()" one, etc.)

For the record:

• When the chunk passthru issues many internal requests in order to complete the external one, the info of each internal request is accessible via the previous_info entry. I considered merging all internal response_headers info under the main one since that's possible, but I'm not sure it's worth the added complexity. Please tell me if you think we should do it.
• A future iteration/PR might add support for time-based events. Right now, implementing a pause in the stream involves calling usleep(), but this doesn't play really well with async. Implementing small pauses and summing them up to the target pause might be good enough - we'll need to give it a try to know better.

[nicolas-grekas]

The entry-point is now AsyncDecoratorTrait, which is a simple trait to wire AsyncResponse, where all the logic is.

I implemented test cases that illustrate some use cases:

• retry after 404
• retry after a transport error
• preflight request
• live transclusion of content, doing concurrent requests inside the main request.

This is what made me change the API proposed here.

PR ready.

[derrabus]

Could this trait be used to solve #36967? 🤔

[nicolas-grekas]

@derrabus absolutely: it would make it way easier to persist to the cache without breaking async. (bold claim, to be verified of course :) )

[nicolas-grekas]

I'm quite happy with this!

BTW, doc PR at symfony/symfony-docs#13736

[nicolas-grekas]

@symfony/mergers I'd appreciate if we could merge this PR soon as that would unlock some next steps (I'm preparing a PR to allow pausing a response, needed for #36692 and for implementing support for HTTP 429 or throttling)

[nicolas-grekas]

this anticipates #37136 to support pausing requests/responses

[fabpot]

Thank you @nicolas-grekas.