Add support for async through transforms

[junosuarez]

I tried a few ways of implementing through-like transforms which first buffer the input stream, for use with standard node-style async functions. In the end, I wound up re-implementing almost all of concat-stream in these modules.

It seems like either through functionality should be a part of concat-stream, as is implemented in this speculative PR, or else there should be a better primitive for writing concat-stream like modules.

This syntax seems clean to me, and there should be negligible overhead for creating Duplex rather than Writable streams. If you're into this approach, feel free to merge - otherwise I'll probably just publish this as another module and revisit it later to factor it better.

🐈

[max-mapper]

@jden I am more okay with the idea of a concat-stream being a duplex stream, but I'm less sure about the optional through function. Seems like you could just as easily just use require('through') instead

[junosuarez]

@maxogden so how would that work? something like this?

var fs = require('fs')
var concat = require('concat-stream')
var through = require('through2')

fs.createReadStream('foo.csv')
  .pipe(concat())
  .pipe(through(someTransformFn))
  ...

This would work perfectly well for my use case. There's some prior art for this pattern in gulp-buffer, although its readme doesn't make it immediately obvious.

[stevemao]

I think its a good idea to make this a duplex stream too.

[stevemao]

ping @jden , did you end up creating a new module?

[junosuarez]

I don't think I ever did. I'm going to close this feature request issue.

[stevemao]

@jden @maxogden I think making this as a duplex stream makes sense. I have valid use cases where I would concat all streams and pipe it down.