Support binary framing and transport #680

adammw opened this Issue Dec 9, 2011 · 15 comments

8 participants


Hi, I've had two ideas that I've been thinking of to enable sending binary with that I would like to get feedback on:

Firstly, the idea of supporting binary content by using a new flag and new message type 'binary' . This allows to handle the binary content in an abstract manner between endpoints, for example using base64 encoding over the current transport mechanisms, or if supported by the transport, using binary framing (see below). Ideally, I'd like it to optionally support/specify an event, because otherwise separating binary messages would require different endpoint identifiers.

Secondly, extending on the first idea, is to introduce a binary framing mechanism for sending data over supported binary transports (e.g. binary websockets, flashsockets, XHR2?) to reduce overall overhead and remove the need to base64 encode and decode binary data at either end.

Thirdly, the standard .send and .emit should automatically set the binary flag if you pass it an ArrayBuffer or (node-only) Buffer.

I was going to draw up also a proposal of the framing format, but it looks like most of the identifiers are variable-length, so I'm not sure of how much space to give them. But looking at it would need to contain:

  • Message Type
  • Message ID
  • Message Endpoint
  • Type-dependant data

I've made initial pushes to support this for the websocket transports. In order to do it properly there, a header message will have to be sent as non-binary, immediately followed by the binary payload. Including message / event info in the binary payload is not a good idea, as would have to do additional restructuring, of possibly vast amounts of data, before delivering everything to the end client. That would be immensely inefficient.


.emit - produce pure json-event. It can be tricky to put binary into it.

Another caveat is a support of binaries in node.js < 5.x. All data came from socket can be messed-up by V8 strings. So. all receive code should be ported into Buffer type.


The last issue will be with browsers. I wonder how they can deal with binary data. Javascript doesn't support binary strings or something similar. V8 will definitely harm all non-utf8 data. I observed such weird behavior in node.js 0.4.12 and it was caused by V8.


For binary data in buffers or other containers, on the server, it makes perfect sense to be able to send these via WebSockets to the browser, and have them decoded as ArrayBuffers, one of the Float/Int array types or blobs.

I'm not convinced that it's worth providing a cross-browser way of sending binary data, however, since browsers that are unable to handle WebSockets are most likely also unable to handle binary data. I'm open to changing my mind, though.


The flag idea is very close to what I had in mind. My question was though, how many use cases do we have for sockets where you would want to send some utf8 data and some binary data?

Because it might also make more sense to initialize it as binary: io.connect(, { binary: true })


I think a few real-world usecases are required here. I had a very similar chat with someone else a month or so ago, and what was suggested then was e.g. binary streaming audio for the html5 audio bits; image data for canvas, etc. The thing with such use, however, is that they require a browser with proper html5 and typed array support, and consequently will have websockets available also. Therefore I see it as more natural to let be the platform for binary bits, and not spend too much time working it into the current


The thing is, you still potentially do want reconnection support, namespaces. In addition, web sockets might be blocked, but we can do binary xhr. We might want to add binary support to, then expose it through


Have you looked into binary xhr which fills ArrayBuffer and similar?


To answer my own question,

So, yeah, seeing as websockets may be blocked, / should support it.


@guille I think it's worth supporting binary transport at the emission level

Usecase: I'd like to setup an identical scene on the client(s) and server. On an interval I'll be updating the clients' scene by emitting binary (for size) diffs. Over the lifetime of the scene, objects will be added and removed; these events will most likely need to be transmitted as json.

@halfblood369 halfblood369 referenced this issue in NetEase/pomelo Dec 9, 2012

如何连接非HTML5客户端? #35


How are things going,now?


@vongosling will be a part of 1.0.


thx,btw, 1.0 version release time is ?


@vongosling can't give you an exact date, but it's close.


@rase- still close?

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment