Connection closed (1009 - ) FRAME::CLOSE_TOO_BIG = 1009

[ryancwalsh]

I'm excited to be using Pawl, and I have it working for small files (such as 350 KB).

However, when I send a larger file (say, 30 MB) via $fullFileName as shown below, I get this error: Connection closed (1009 - ).

\Ratchet\Client\connect($url)->then(function(\Ratchet\Client\WebSocket $conn) use($contentType, $fullFileName) {
    $conn->on('message', function($msg) use ($conn) {
        Log::debug("Received: {$msg}");
    });
    $conn->on('close', function($code = null, $reason = null) {
        Log::debug("Connection closed ({$code} - {$reason})");
    });
    $conn->send(json_encode([
        'content-type' => $contentType,
        'timestamps' => true,
        'speaker_labels' => true,
        'smart_formatting' => true,
        'inactivity_timeout' => 60 * 5,
        'x-watson-learning-opt-out' => true,
        'action' => 'start'
    ]));

    $frame = new \Ratchet\RFC6455\Messaging\Frame(file_get_contents($fullFileName), true, \Ratchet\RFC6455\Messaging\Frame::OP_BINARY);
    $binaryMsg = new \Ratchet\RFC6455\Messaging\Message();
    $binaryMsg->addFrame($frame);
    $conn->send($binaryMsg);
    $conn->send(json_encode([
        'action' => 'stop'
    ]));
}, function ($e) {
    echo "Could not connect: {$e->getMessage()}\n";
});

When I search for usages of Frame::CLOSE_TOO_BIG, I see that it's only ever used by \Ratchet\RFC6455\Messaging\CloseFrameChecker.

But I've been unable to figure out how \Ratchet\RFC6455\Messaging\CloseFrameChecker works and what the file size limits are and how to send large files.

I've tried first splitting my file into chunks using str_split and then adding individual frames, but then I hit session timeouts every time, even for small files.

What is the appropriate way to send larger files, avoiding the Frame::CLOSE_TOO_BIG 1009 error and session timeouts?

[mbonneau]

@ryancwalsh I am guessing that you are receiving that notification from the remote server saying the message size is too large.

[mbonneau]

I am not 100% sure what Looking at the Watson API - I have found:

• "The WebSocket interface imposes a maximum frame size of 4 MB. "
• "The client can stream a maximum of 100 MB of audio data with a single utterance (per send request)."

Can you post your code where you are breaking the message into smaller frames?

[ryancwalsh]

@mbonneau Thanks so much! Yep, I just found that page too and so have switched to doing this now:

    /**
     * @param \Ratchet\Client\WebSocket $conn
     * @param string $fileContents
     */
    public function sendBinaryMessage($conn, $fileContents) {
        $binaryMsg = new \Ratchet\RFC6455\Messaging\Message();
        $chunks = str_split($fileContents, 4000);
        foreach ($chunks as $key => $chunk) {//This is how to avoid Connection closed (1009 - ) https://cloud.ibm.com/apidocs/speech-to-text and https://cloud.ibm.com/docs/services/speech-to-text?topic=speech-to-text-websockets#WSreturn say 1009 = The connection closed because the frame size exceeded the 4 MB limit.
            $final = ($key === array_key_last($chunks)) ? true : false;
            $frame = new \Ratchet\RFC6455\Messaging\Frame($chunk, $final, \Ratchet\RFC6455\Messaging\Frame::OP_BINARY);
            $binaryMsg->addFrame($frame);
        }
        $conn->send($binaryMsg);
    }

However, this approach was just me winging it because I've never seen any example like this. I don't know what I'm doing.

And now, every time I try to run this, I get this response: "error": "Session timed out." after ~30 seconds.

And that's even if I set the Watson inactivity_timeout to -1 (infinity) and set 90 like this:

$loop = \React\EventLoop\Factory::create();
$reactConnector = new \React\Socket\Connector($loop, ['timeout' => 90]);
$connector = new \Ratchet\Client\Connector($loop, $reactConnector);
$connector($url, [], $headers)->then(...

If you have any hints, I'd really love to hear. Thanks!

[mbonneau]

Try sending each frame in the chunk instead of assembling the message. Then just send an empty final binary frame after to signify the end of the message. I am pretty sure using addFrame is causing the message to just reassemble the frames into a single blob again.

The timeout on the connector is only a connection timeout.

[ryancwalsh]

Thanks, @mbonneau .

I don't know if I interpreted your suggestion correctly; this is what I tried just now, and unfortunately it still resulted in "error": "Session timed out." after ~30 seconds:

/**
 * @param \Ratchet\Client\WebSocket $conn
 * @param string $fileContents
 */
public function sendBinaryMessage($conn, $fileContents) {
    $chunks = str_split($fileContents, 4000);
    foreach ($chunks as $key => $chunk) {
        $final = false;
        $frame = new \Ratchet\RFC6455\Messaging\Frame($chunk, $final, \Ratchet\RFC6455\Messaging\Frame::OP_BINARY);
        $binaryMsg = new \Ratchet\RFC6455\Messaging\Message();
        $binaryMsg->addFrame($frame);
        $conn->send($binaryMsg);
    }
    $final = true;
    $frame = new \Ratchet\RFC6455\Messaging\Frame($chunk, $final, \Ratchet\RFC6455\Messaging\Frame::OP_BINARY);
    $binaryMsg = new \Ratchet\RFC6455\Messaging\Message(); //empty
    $conn->send($binaryMsg);
}

[mbonneau]

I mean something like this:

public function sendBinaryMessage($conn, $fileContents) {
    $chunks = str_split($fileContents, 4000);
    foreach ($chunks as $key => $chunk) {
        $frame = new \Ratchet\RFC6455\Messaging\Frame($chunk, false, \Ratchet\RFC6455\Messaging\Frame::OP_BINARY);
        $conn->send($frame);
    }
    $frame = new \Ratchet\RFC6455\Messaging\Frame('', true, \Ratchet\RFC6455\Messaging\Frame::OP_BINARY);
    $conn->send($frame);
}

I have not tested this code - it is just for the general idea.

[ryancwalsh]

@mbonneau Good idea. I just tried that. This time I got Connection closed (1006 - Underlying connection closed), which Watson says "indicates that the connection closed abnormally."

So, a different error code. I don't know which is easier to solve.

If you have any other ideas, I'm all ears. Thanks.

[ryancwalsh]

@mbonneau I don't really understand the difference, but this seems to work!

public function sendBinaryMessage($conn, $fileContents) {
    $chunks = str_split($fileContents, 100);
    Log::debug('Split audio into this many frames: ' . count($chunks));
    $final = true;
    foreach ($chunks as $key => $chunk) {
        Log::debug('send chunk key ' . $key);
        $frame = new \Ratchet\RFC6455\Messaging\Frame($chunk, $final, \Ratchet\RFC6455\Messaging\Frame::OP_BINARY);
        $conn->send($frame);
    }
}