embrace the Stream

[dominictarr]

hi,

This is a very interesting project, but I'd like to make a suggestion:

Instead of wrapping a around a particular browser-server communication library.
Just expose a Stream, then connect it to the communication library like this:

var client = require('sharejs').client
var stream = client.createStream()

stream.pipe(net.connect(SERVER_ADDR)).pipe(stream)

The advantage of this pattern is that your module becomes compatible with any text stream.
since every form of io in node.js uses streams, it means that you can use share.js over tcp, http, stdio,
through temp files... anything.

The same approach can be used in the browser, here is the recommended full duplex browser-server lib: shoe it's just a tight wrapper over sockjs. It would be simple to implement a similar lib over browser-channel.

Following this pattern means that the library code is completely decoupled from the IO code, and thus can be agnostic about whether it's running in the browser, or on node. Often I've found these sorts of libs can actually be agnostic whether they are the client or the server!

This pattern is also very handy if you wish to use something like ShareJs with something like dnode you can still stream them through the same connection with something like mux-demux

[josephg]

Thats hot.

[ggoodman]

Do please consider this!

[adambreen]

Yep definite +1!

[Grynn]

Another +1

[dominictarr]

oh, incidentally,
I implemented a basic collaborative editing scuttlebutt https://github.com/dominictarr/r-edit
it doesn't have all the features that ShareJs does, but it's easy to drop in, and connect with streams.
live demo here: http://dominictarr.github.com/r-edit/

[seidtgeist]

I very much support this! With stuff like WebRTC DataChannels coming up it'll be trivial to switch from e.g. shoe to a DataChannel-implementation once there's a node module to establish peer connections.

[wmertens]

So the big problem is actually writing this 😉

[seidtgeist]

No problem Where would one start? Hack on server/session and client/connection?

[seidtgeist]

The right decision has been made and the new ShareJS' transport will use node's new streams (if I understood it correctly from this ml post). Might be a while until this exists and probably in another repo/branch but I think that's where we should base our efforts on. I'd like to clarify a few things but I think I'll move that discussion into the ML post.

[josephg]

I met up with @isaacs and got the lowdown on the new streams API a few weeks ago. This is going into the rewrite, at least on the server side. Moving forward, you're going to be responsible for managing your own connection to the browser using streams.

Here's an example wrapping the new API around browserchannel (though once browserchannel uses streams too, it will actually be as simple as calling pipe().

[mcolyer]

@josephg is there a branch or an entirely new project for the re-write?

[josephg]

I've made a prototype of it in a new project ( https://github.com/josephg/share-proto ), but it'll eventually get merged back in.