Automatically split & re-combine messages over a certain size

[rsshilli]

Thank you for making this package! It's great.

This is a feature request.

AWS API Gateway allows only a maximum size of 128kb of data to be sent across 4 x 32kb frames when using Websockets:

https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html

I would love it if this package allowed me to specify the maximum frame size somewhere and then it would automatically split up messages and re-combine them on the other side to overcome this.

[lukeed]

Hi! Thanks :)

This wouldn't make it into Sockette unfortunately, mostly because of how easy it would be to achieve from the "outside" :

let ws = new Sockette(...);

// can add a new method for large payloads
// or can save & overwrite `ws.send` (chosen)
let send = ws.send.bind(ws);
ws.send = function (msg) {
  let len = msg.length;
  if (len < 32e3) return send(msg);
  
  for (let sent = 0; sent < len;) {
    send(msg.substring(sent, sent += 32e3));
  }
}

Warning: untested :)

I'm also not sure that this is a common need. I, for one, use WS a lot and have never come close to that limit 😬Curious what you need it for, haha

Hope that helps & that you understand where I'm coming from!

Thanks

[rsshilli]

This problem isn't super difficult but it's tougher than it looks. For example, your code above would break if msg is in Chinese (every char would take up 4 bytes). Also, there needs to be a basic protocol header so that the other side knows if the message has been broken up, when it's done, etc.

Still, I understand if this isn't Sockette's thing. I'll keep searching for a solution. We have a couple of large forms where people can enter a lot of data. Occasionally, but rarely, they cross this barrier. I'm surprised that I'm the first person to have to solve it.

[lukeed]

Sure – I know what I posted is far from complete, but the idea is that once you find/decide how you want to split up your messages, it's easily attached to a Sockette instance. I was just showing an example of that.

Interesting. For my purposes, those requests would live under the HTTP protocol, but it's probably not as interesting :P

[spankyed]

@rsshilli Could you share the solution you came up with for this issue?

Edit: I found this tutorial on handling WebSocket frame boundaries in stream-based data using the STOMP protocol. But the solution is anything but easy to implement (or comprehend at the least).

[rsshilli]

@spankyed It's not a simple answer. It's ~100 lines of code for us that wouldn't make sense outside of our product (qbdvision.com). I'm sorry that I don't have something simple to share.