Adding a buffering input stream in your pipeline will collect a given number of objects, and send them through as an array to the next stage. This is useful if you have a stream source that emits objects, and you want to batch them together before sending them to a database.
npm install buffering-object-stream --save
For example:
var through = require("through2");
vor bufferingObjectStream = require("buffering-object-stream");
createSomeObjectStream()
.pipe(bufferingObjectStream(4))
.pipe(through.obj(function (objects, enc, cb) {
console.log("Got objects: ", objects);
cb();
});
If createObjectStream sends out 10 objects (e.g. strings), this will print
out something like:
Got objects: ["object1", "object2", "object3", "object4"]
Got objects: ["object5", "object6", "object7", "object8"]
Got objects: ["object9", "object10"]
bufferSize- integer
The number of objects to buffer before emitting.