Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

module suggestions #14

Open
juliangruber opened this Issue · 9 comments

3 participants

@juliangruber

Ok, fwd ain't so cool, but I have a new one:

https://github.com/juliangruber/mux

@Raynos

@juliangruber fwd is similar to re-emitter

The only feature of mux is ignoring back pressure which im not sure if its a good thing.

@juliangruber

you're right, currenlty mux doesn't handle backpressure correctly. There actually is a pause-stream between each source and destination stream, but I forgot that if dest1 applies backpressure and thus the inter-connected pause-stream pauses, the source also pauses. I'll fix that and then I think it's ready to be used

@Raynos

@juliangruber I think the fact that backpressure works is a feature.

The reason it's a feature is because imagine you have Y written to A, B, C. Imagine Y is 1000GB. if C is a 100 times slower then A then by the time C finisheds A has consumed 10GB of which the other 990GB is in memory. That's crazy.

@juliangruber

Yeah that makes sense.

@Raynos

@juliangruber if you want to optimize for the fast reader then open two readable streams. Any attempt to cache would start buffering a significant percentage of that 1000GB in memory.

Either optimize for low latency or for memory usage.

@fent

Here's an alternative to fwd

https://github.com/fent/node-eventyoshi

The difference is that you can add several child event emitters. We use it to combine several Twitter streams into one. Since each data event is a Tweet, the order that they come in is not important like in a binary stream.

And you can also start listening for events before child event emitters are added and as they are removed.

@juliangruber

@Raynos so then mux is just a shorthand for

a.pipe(b);
a.pipe(c);
a.pipe(d);

a.pipe(mux(b, c, d));

@fent your api is nicer, fwd always needs src and dest to be present.

var combined = new EventEmitter();
fwd(src1, combined);
fwd(src2, combined);

// vs

var combined = new EventYoshi();
combined.add(src1);
combined.add(src2);

The goal of fwd is to be a simple connector between api-incompatible parts of your app.

@juliangruber

mux is like tee(1), btw

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.