Streaming HTTP requests & awaiting promises

[bartvanhoutte]

Rather unsure on where to post this because it involves streams, async/await, the event loop and http libraries, so don't hesitate to tell me to move it somewhere else.

Currently, I'm trying to pass on a readable stream from a server request to a HTTP client. However, whenever you call await before passing on the stream*, the stream receives data in the event loop and is closed before the HTTP client is able to make a request, eventually raising an exception as the HTTP client's timeout has been reached.

*I need to do some - eventually cached - async I/O in a middleware.

$server = new HttpServer(
  new StreamingRequestMiddleware,
  async(function (ServerRequestInterface $request) {
      try {
          $body = $request->getBody();
          assert($body instanceof ReadableStreamInterface);

          await(\React\Promise\Timer\sleep(1));

          $browser = new Browser;

          // $body->closed === true
          await($browser->put(
            'http://httpstat.us/200',
            ['Content-Length' => $body->getSize()],
            $body
          ));

          fwrite(STDOUT, 'done');
          $response = new Response(200);
      } catch (Exception $e) {
          fwrite(STDERR, $e->getMessage());
          $response = new Response(500);
      }

      return $response;
  }));

Versions used: react/async:4.x-dev and react/http:1.6.0.

Is this a bug or just not possible in the current design of the libraries? Is there a workaround for this?

[SimonFrings]

Hey @bartvanhoutte

There could be two reasons for that:

1. Your incoming stream could be empty and closes immediately, because there is nothing to do.
2. Your incoming stream receives data, you sleep for one second and at the same time your stream already received all data and closes, which leads to loosing all data.

You could try to buffer everything and see if you receive data as a first step to see where the problem is and then work your way up to fixing it.

I hope this helps!

[bartvanhoutte]

Hi @SimonFrings

what happens is what you describe in case 2. When calling await control is turned over to the Loop, calling stream_select, reading data from the stream and eventually closing it.

Buffering the data beats the purpose of using a stream in my case. Is there a way to tell the loop to pause the stream after parsing headers or something like that?

[clue]

@bartvanhoutte A stream is by definition something that happens over a period of time. If you want to do something "in between", you have to manage the stream flow using explicit instructions. For this, you can pause() and resume() the input stream at any time. However, note that pausing is advisory only and there may be some bytes still left in incoming receive buffer that will still be emitted. That's why you may also have to set up a small buffer to make sure you're not missing this "initial" data.

I believe this has been answered, so I'm closing this for now. Please come back with more details if this problem persists and we can always reopen this 👍