Sending small buffers has unexpected result - the client receives the correct buffer but messes it up

[lll000111]

I use node.js v8.2.0 and ws v3.0.0.

Below is a tiny server and client example.

• Start the server in one console
• Start the client in another console

When the client connects the server creates a small Buffer, writes two numbers at the beginning, and sends the buffer to the client.

Code

Server:

'use strict';

const WebSocketServer = require('ws');

const wss = new WebSocketServer.Server({maxPayload: Infinity, port: 3000});

console.log('WAITING FOR CLIENT...\n');

wss.on('connection', ws => {
    // ANYTHING LESS 1024 DOESN'T WORK:
    const b = Buffer.alloc(9);

    // The first 9 + 1 Byte are used for a 64bit and an 8bit number, respectively

    b.writeDoubleLE(1, 0);
    b.writeUInt8(255, 8);

    console.log('SENDING BUFFER (length ' + b.length + '):', b, '\n');

    ws.send(b);

    ws.onclose = ({code, reason}) => {
        if (wss.clients.size === 0) {
            wss.close();
        }
    };
});

Client

'use strict';

const WebSocketServer = require('ws');
const ws = new WebSocketServer('ws://localhost:3000/ONE');

ws.onmessage = message => {
    if (message.data instanceof Buffer) {
        console.log('RECEIVED BUFFER\n');
        console.log(
            'Buffer (length ' + message.data.length + '): ', message.data
        );
        console.log(
            'ArrayBuffer (length ' + message.data.buffer.byteLength + '):',
            new Uint8Array(message.data.buffer)
        );

        // The first 8 bytes are for a Javascript number
        const count = new DataView(message.data.buffer, 0, 8);
        console.log('\nNumber (Bytes 0..7):', count.getFloat64(0, true));

        const lastFragmentFlag = new DataView(message.data.buffer, 8, 1);
        console.log('Flag (Byte 8):', lastFragmentFlag.getUint8(0));

        ws.close(1000);
    }
};

Test Result

The server console output:

WAITING FOR CLIENTS...

SENDING BUFFER (length 9): <Buffer 00 00 00 00 00 00 f0 3f ff>

The client console output:

RECEIVED BUFFER

Buffer (length 9):  <Buffer 00 00 00 00 00 00 f0 3f ff>
ArrayBuffer (length 11): Uint8Array [ 130, 9, 0, 0, 0, 0, 0, 0, 240, 63, 255 ]

Number (Bytes 0..7): 1.2026e-320
Flag (Byte 8): 240

Problem

If the buffer sent by the server is less than 1024 Bytes the client receives garbage instead of the expected numbers - even though the dump of the buffer on the server side shows the correct buffer contents.

It all works as soon as I use at least 1024 for the allocated buffer size (sending a lot of unneeded zeros).

NOTE: The "1024" is not constant, that's the value I found works for this example. In my actual larger test code that does the full transfer the behavior is even more varied. I found when I use 64000 as the size it always works. The discrepancy in size between the Buffer and its ArrayBuffer (message.data.buffer) also is much larger there. Here it is just two bytes, in my stream test it's 9 vs. 57409, for example.

Also note that the exact same code for the transfers and for the log statement works find for my larger code test example where I do an actual transfer - only the last chunk, the one I modeled here, causes the problem. So my general approach with how I write those numbers into the buffer and how I get them back on the client works, apparently.

Is it possible the underlying ArrayBuffer is not zeroed? So when I have a smaller buffer than before I get in trouble?

Other Clients Work!

µws as Client (works)

Client code slightly changed for [µws](https://github.com/uNetworking/bindings/tree/master/nodejs), such as wss.clients.length instead of wss.clients.size, and I get an ArrayBuffer not a Buffer, it shows the expected result:

RECEIVED BUFFER

Buffer (length 9):  ArrayBuffer { byteLength: 9 }

Number (Bytes 0..7): 1
Flag (Byte 8): 255

Browser as Client (works)

The problem is the client side - the example works fine when I use the browser (Google Chrome).

• Start the server script in a console (same as before)
• In Google Chrome go to http://localhost:3000/ (I also tried IE Edge and the same code below worked there too)
• Open a console (F12) and paste and execute the code:

'use strict';

ws = new WebSocket('ws://localhost:3000/ONE');

ws.onmessage = message => {
    console.log(message.data);

    if (message.data instanceof Blob) {
        const reader = new FileReader();

        reader.addEventListener("loadend", function() {
            console.log('Buffer (length ' + reader.result.length + '): ', reader.result);

            const count = new DataView(reader.result, 0, 8);
            console.log('\nNumber (Bytes 0..7):', count.getFloat64(0, true));

            const lastFragmentFlag = new DataView(reader.result, 8, 1);
            console.log('Flag (Byte 8):', lastFragmentFlag.getUint8(0));

            ws.close(1000);
        });

        reader.readAsArrayBuffer(message.data);
    }
};

ws.onerror = console.error;

Background Story (optional)

My use case - not really relevant but just for the background story: I send a binary stream over websocket and since the client may issue many different requests simultaneously each one gets an ID. Now I have to add that ID to each chunk of the binary stream . I do that by putting 9 Bytes in front of the actual chunk. It all works fine during the stream, when the buffers are large - but for some inexplicable reason the last message, supposed to signify the end (Byte 9 is the "end" flag, 0 during the stream, 255 for the last chunk) has both the numeric ID (Bytes 0...7) and the flag garbled. If I blow up the last chunk to be much larger than actually necessary it works.

[lpinca]

Your conversion from Buffer to ArrayBuffer is wrong. You have to use something like this:

const ab = buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength)

where buf is the buffer.

Here is a full example with your code:

const WebSocket = require('ws');

const wss = new WebSocket.Server({ port: 3000 }, () => {
  const ws = new WebSocket('ws://localhost:3000');

  ws.onmessage = (evt) => {
    const buf = evt.data;
    const ab = buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength);

    const count = new DataView(ab, 0, 8);
    console.log('Number (Bytes 0..7):', count.getFloat64(0, true));

    const lastFragmentFlag = new DataView(ab, 8, 1);
    console.log('Flag (Byte 8):', lastFragmentFlag.getUint8(0));
  };
});

wss.on('connection', (ws) => {
  const b = Buffer.alloc(9);

  b.writeDoubleLE(1, 0);
  b.writeUInt8(255, 8);

  ws.send(b);
});

If you need an ArrayBuffer you can set the binaryType attribute to 'arraybuffer' and the conversion is done automatically:

ws.binaryType = 'arraybuffer';