Replacing msgpack-lite by msgpack due to data lost

[jeroenrinzema]

See issue #172

[darrachequesne]

Were you able to reproduce the issue?

[jeroenrinzema]

I was unfortunately not able to reproduce the bug in socket.io-redis but i was able to reproduce a part of the bug in msgpack-lite and msgpack-js. It looks like something goes wrong with the algorithm inside of the dependencies when encoding the message. I think that it is then anyhow a good idea to switch to a different dependency. msgpack passed the tests and didn't show the same strange behavior.

[darrachequesne]

Without a proper way to reproduce, I'm afraid that we can't make sure it's really related to msgpack-lite...

Besides, it seems that msgpack does not compile under Windows: pgriess/node-msgpack#70

Regarding your example, may I ask what it is supposed to show?

[jeroenrinzema]

I see that something went wrong this is the right example. I will meanwhile keep on trying to reproduce the bug.

[darrachequesne]

Thanks for the example.

It might be linked to #157. Are you still using the return_buffers option? Or a custom subEvent?

[valeriansaliou]

Why don't we consider just dropping MessagePack? We've been running SocketIO-Redis @crisp-im over 1+ year, and as we are gaining more and more pressure on our SocketIO workers over time we've started reviewing what we could optimize.

It turned out that SocketIO-Redis use of MessagePack was causing ~20% of our SocketIO workers CPU load (notice: we do stream some large JSON payloads sometimes). Forking SocketIO-Redis and moving it to the Node native JSON.parse() / JSON.stringify() implementation fixed the issue.

Plus, looking at the Redis MONITOR command; it appears that MessagePack does not reduce the size of messages that much, as MessagePack-optimized binary blobs get string-encoded when wired in/out of Redis (Redis only accepts string contents). Thus, a lot of MessagePack compact-binary blobs get encoded/escaped as multiple-chars strings; which increase the overall size of the MessagePack payload that is streamed across SocketIO workers.

Overall, after switching from MessagePack to bare-bone JSON we've seen an overall CPU usage decrease of ~20%, and a Redis Pub/Sub I/O increase of ~10-20%. Which is why we'll stick to our JSON fork over the long run as usually high-pressure is set on the CPU at scale.

--

Anyway, if you want to stick to a MessagePack implementation, I'd recommend Notepack. We've been previously using it on other high-load services (to pass data between our internal Node workers using RabbitMQ) w/o that much impact on CPU (however we've also replaced it w/ JSON). If you perform some benchmark between Node MessagePack implementations, you'll notice Notepack is the fastest. We did not encounter any issue using it (encoding special extended UTF-8 chars; encoding/decoding large JSON, etc).

[darrachequesne]

Closed by #218.