readable-stream

[Raynos]

Node 0.9 will use the new readable-stream interface.

"data" events, pause and resume will be deprecated.

We need some kind of upgrade strategy for all these streaming modules. Including scuttlebutt, mux-demux, shoe, etc.

[dominictarr]

I thought that the old stream interface was still gonna work, too?

[Raynos]

@dominictarr yes it will be fully backwards compatible.

But I want to make a clean break and write all my new streams in the readable-stream interface.

From my point of view all modules based on the old stream will be considered legacy.

[eshao]

Has this been resolved? I upgraded from node v0.8 to node v0.10 and my modules depending on event-stream are broken in a few edge cases:

For example, put the following into two seperate files. When doing cat file | node stream_processer1.js it works, but doing cat file | node stream_processer1.js | node stream_processer2.js results in an error.

module.exports = function(data, callback) {
// do some work here
  return callback(null, data)
}

if (require.main === module) {
  es.pipeline(
    process.stdin
  , es.split()
  , es.parse()
  , es.map(module.exports)
  , es.stringify()
  , process.stdout
  )
}

_stream_readable.js:735
    while (!paused && (null !== (c = stream.read())))
                                            ^
TypeError: Property 'read' of object #<Socket> is not a function
    at Socket.<anonymous> (_stream_readable.js:735:45)
    at Socket.EventEmitter.emit (events.js:92:17)
    at emitDataEvents (_stream_readable.js:761:10)
    at Socket.Readable.on (_stream_readable.js:682:5)
    at proxyStream (/Volumes/Shared_HD/Users/eshao/wsp/kites/src/kites-analytics/node_modules/event-stream/node_modules/stream-combiner/node_modules/duplexer/index.js:65:16)
    at Array.forEach (native)
    at duplex (/Volumes/Shared_HD/Users/eshao/wsp/kites/src/kites-analytics/node_modules/event-stream/node_modules/stream-combiner/node_modules/duplexer/index.js:26:16)
    at Object.module.exports (/Volumes/Shared_HD/Users/eshao/wsp/kites/src/kites-analytics/node_modules/event-stream/node_modules/stream-combiner/index.js:8:17)
    at Object.<anonymous> (/Volumes/Shared_HD/Users/eshao/wsp/kites/src/kites-analytics/bin/freshness.js:51:39)
    at Module._compile (module.js:456:26)

[dominictarr]

does this only happen when piping in bash? what about if you pipe directly?

I do not plan to rewrite all this in streams2, but i'll merge a pull request.

I'd recommend using the stream-serializer module instead,

var serialize = require('stream-serializer')()
process.stdin.pipe(serialize(es.map(fun))).pipe(process.stdout)

[eshao]

It also happens if you spawn a child and pipe between the children. If I pipe all in one file, then it works.

[dominictarr]

can you post a script to reproduce?

On Sat, May 25, 2013 at 6:12 AM, Edwin Shao notifications@github.comwrote:

It also happens if you spawn a child and pipe between the children. If I
pipe all in one file, then it works.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/21#issuecomment-18439438
.

[q2dg]

If you don't plan to rewrite "event-stream" in streams2 (oooh, what a pity!!), do you know another equivalent module that does similar things than yours?
Thanks a lot for your work.

[dominictarr]

no I don't. there are probably already stream2 modules for most of the functions and you'd just need to bundle them together into event-stream2

if you see a job that needs doing, then it's your job