validate_utf8 performance issues

[andresriancho]

I'm using websocket-client to communicate with Chrome devtools protocol. Some of the messages that come back from Chrome are large (they contain the whole DOM). While I was testing my software I found a lot of issues, which I initially associated with the socket and how it was read, but they were actually generated by validate_utf8.

I'm not using wsaccel.

All my issues were fixed when I started using skip_utf8_validation in the WebSocket class.

Not really sure how you guys can reproduce this, but the basics are:

• Capture websocket packets using wireshark
• Send websocket messages to Chrome (or any other server)
• Receive large message
• Verify that in wireshark the server sent the message at timestamp X
• Verify that the socket returned the message to the upper layers at X+0.001
• Verify that the websocket returned the message to the upper layers at X+3.0001 , where the extra three seconds (could be more in some cases) are associated with validate_utf8

Then run the same test with skip_utf8_validation and you'll find that there is no delay.

[engn33r]

@andresriancho thanks for the comment. If I understand correctly, you are saying that by setting the skip_utf8_validation parameter to True, as found on this line (for WebSocket) or on this line (for WebSocketApp), you improved the performance with large WebSocket messages. I think this is captured in the README.md "Performance" section, which mentions that the "validate_utf8" method is slow. I might move this README performance section to the new project documentation, but if you think there is a better way to highlight the impact of this parameter on overall performance, let me know. I will close this issue since the performance impact is mentioned, although it is an open question for how it can be made clearer.