Skip to content
This repository

how to write node programs with streams

tree: 29bb1884d1

Fetching latest commit…

Cannot retrieve the latest commit at this time

readme.markdown

introduction

This document covers the basics of how to write node.js programs with streams.

"We should have some ways of connecting programs like garden hose--screw in
another segment when it becomes when it becomes necessary to massage data in
another way. This is the way of IO also."

Doug McIlroy. October 11, 1964


Streams come to us from the earliest days of unix and have proven themselves over the decades as a dependable way to compose large systems out of small components that do one thing well. In unix, streams are implemented by the shell with | pipes. In node, the built-in stream module is used by the core libraries and easily be used by user-land code. Similar to unix, the node stream module's primary composition operator is called .pipe().

Streams can be useful because they restrict the implementation surface area into a single consistent interface. You can then easily plug the output of one stream to the input of another and use libraries that operate abstractly on streams to institute higher-level flow control.

Streams are an important component of small-program design and unix philosophy but there are many other important abstractions worth considering. Just remember that technical debt is the enemy and seek the best abstractions for the problem at hand.


basics

http://maxogden.com/node-streams

readable

writable

pipe

backpressure


builtin streams

These streams are built into node itself.

process.stdin

This readable stream contains the standard system input stream for your program.

It is paused by default but the first time you refer to it .resume() will be called implicitly on the next tick.

If process.stdin is a tty (check with tty.isatty()) then input events will be line-buffered. You can turn off line-buffering by calling process.stdin.setRawMode(true) BUT the default handlers for key combinations such as ^C and ^D will be removed.

process.stdout

process.stderr

child_process.spawn()

fs.createReadStream()

fs.createWriteStream()

net.connect()

This function returns a [through stream] that connects over tcp to a remote host.

You can start writing to the stream right away and the writes will be buffered until the 'connect' event fires.

net.createServer()

http.request()

http.createServer()


control streams

through

pause-stream

concat-stream

duplex

mux-demux

emit-stream

invert-stream

map-stream

remote-events


state streams

cdrt

delta-stream


io streams

request

reconnect

kv


parser streams

tar

trumpet

JSONStream

Use this module to parse and stringify json data from streams.

If you need to pass a large json collection through a slow connection or you have a json object that will populate slowly this module will let you parse data incrementally as it arrives.

json-scrape


browser streams

shoe

domnode

sorta

graph-stream

arrow-keys


rpc streams

dnode

rpc-stream


test streams

tap

Something went wrong with that request. Please try again.