Skip to content

Commit

Permalink
Docs for size chunker
Browse files Browse the repository at this point in the history
  • Loading branch information
Olegas committed Oct 13, 2013
1 parent bc2779f commit 4e55505
Show file tree
Hide file tree
Showing 2 changed files with 64 additions and 8 deletions.
66 changes: 59 additions & 7 deletions README.md
Expand Up @@ -24,10 +24,10 @@ Simple transform stream which counts lines (`\n` is a separator) and emit data c
### Configuration

```javascript
new LineCounter({
new LineCounter({
numLines: 1, // number of lines in a single output chunk. 1 is default
flushTail: false // on stream, end flush or not remaining buffer. false is default
});
flushTail: false // on stream end, flush or not remaining buffer. false is default
});
```

SeparatorChunker
Expand All @@ -40,8 +40,60 @@ By default separator sequence is set to `\n`, so it is equals to LineCounter wit
### Configuration

```javascript
new SeparatorChunker({
separator: '\n', // separator sequence
flushTail: false // on stream, end flush or not remaining buffer. false is default
});
new SeparatorChunker({
separator: '\n', // separator sequence
flushTail: false // on stream end, flush or not remaining buffer. false is default
});
```

SizeChunker
-----------

Split streams into chunks having at least specified size in bytes (but maybe more). It is **object mode** stream!
Each data chunk is an object with the following fields:

- id: number of chunk (starts from 1)
- data: `Buffer` with data

SizeChunker has 2 additional events:

- chunkStart: emitted on each chunk start.
- chunkEnd: emitted on each chunk finish.

Both event handlers must accept two arguments:

- id: number of chunk
- done: callback function, **must** be called then processing is completed

### Configuration

```javascript
new SizeChunker({
chunkSize: 1024 // must be a number greater than zero
});
```

### Example
```javascript
var input = fs.createReadStream('./input'),
chunker = new SizeChunker({
chunkSize: 1024
}),
output;

chunker.on('chunkStart', function(id, done) {
output = fs.createWriteStream('./output-' + id);
done();
});

chunker.on('chunkEnd', function(id, done) {
output.end();
done();
});

chunker.on('data', function(data) {
output.write(data.chunk);
});

input.pipe(chunker);
```
6 changes: 5 additions & 1 deletion lib/size-chunker.js
Expand Up @@ -14,7 +14,11 @@
Transform.call(this, options);
this._bytesPassed = 0;
this._currentChunk = 0;
this._chunkSize = options && options.chunkSize || 10;
this._chunkSize = +(options && options.chunkSize);

if(isNaN(this._chunkSize) || this._chunkSize <= 0) {
throw new Error("Invalid chunk size. Must be a number greater than zero.");
}

this._readableState.objectMode = true;

Expand Down

0 comments on commit 4e55505

Please sign in to comment.