Client disconnect when processing large data volumes

[zinc1oxide]

Using version 0.7.0, I'm strictly using the client version of the API that connects to a server providing realtime data. When I request for a large data set, I'm receiving the following:

[2017-05-09 15:56:13] [error] handle_read_frame error: websocketpp.transport:7 (End of File)

Looking at other similar issues, the answer seems to indicate the endpoint is closing the connection. However, when I test against the same server, with the same data requests in other languages such as Java and when using node.js, I do not get any disconnect.

The WebSocket server I'm testing against indicates it has received an "overflow condition", which I believe implies that my client is not processing/reading fast enough. Within my message handler, I'm not doing any processing of the buffer coming in. Literally printing a short message to stdout then returning.

I've stepped through the code, but seems to get really deep within boost. Any suggestions?

[JustSpiffy]

We are also experiencing this problem. @zaphoyd Any updates on this?

[zaphoyd]

Can you provide a little more information? When you say large volumes, do you mean large numbers of messages, large messages themselves (and if so approximately how large are we talking?), large numbers of connections? Are you using the compression extension? Have you tried the latest development branch and confirmed that it is still an issue there?

[alexpmorris]

I recently noticed a strange intermittent problem with larger packets that may be related to this issue while accessing a graphene blockchain websocket (which uses websocketpp). At first I thought it was a problem with my websocket client implementation, however now I am not so sure.

All frames returned are in json format and no compression is being used (I believe there were issues when they tried enabling compression). Every so often, I get what appears to be a "complete" websocket frame, however, the json packet is incomplete (ie. the frame size appears to be smaller than the actual packet size). The rest of the packet still comes through (but not as another frame). After appending the remainder to the original truncated packet, all seems well again. However, why would the server be sending more data than is indicated by the packet itself?

here's an example:

23432 bytes:  `{"id":5569,"result":{"asks":[{ ... `
7 bytes left: ... `3434"]}`   <---- append these bytes to frame, but DON'T process it as a new frame!

Sometimes. I even have to repeat this a few times with multiple fragments, basically until I get a closing bracket " } " to confirm the complete packet was received. So far, this "hack" seems to be working.

I took a look around the websocketpp code, and while I'm not exactly sure of the problem (it's possible it's still completely unrelated), if it does turn out to be an issue with websocketpp, perhaps there's a case in hybi13.hpp or frame.hpp where the frame size is registered as smaller than the total message size.

[jcelerier]

same problem here: past 30 megabytes (in a single ws message) I get

[2018-01-22 19:18:43] [error] handle_read_frame error: websocketpp.transport:7 (End of File)
[2018-01-22 19:18:43] [info] Error getting remote endpoint: asio.system:107 (Transport endpoint is not connected)
[2018-01-22 19:18:43] [info] asio async_write error: asio.system:32 (Broken pipe)
[2018-01-22 19:18:43] [fatal] handle_write_frame error: websocketpp.transport:2 (Underlying Transport Error)