Skip to content

Commit

Permalink
Implement an option to align final incomplete data chunk
Browse files Browse the repository at this point in the history
Writing unaligned data causes `EINVAL` (from `write(2)`):

> EINVAL
>  fd is attached to an object which is unsuitable for writing; or the
>  file was opened with the O_DIRECT flag, and either the address
>  specified in buf, the value specified in count, or the current file
>  offset is not suitably aligned.

This pull request implements an opt-in option (to preserve backwards
compatibility) called `align`, that when used in conjunction to `flush`,
will cause the incomplete data chunk to be padded with null bytes to
match the passed chunk size.

For example, in the added test, the chunk size is 4 bytes, and we push
three chunks:

- `12`: 2 bytes
- `34`: 2 bytes
- `5`: 1 byte

The final incomplete data chunk will become `5\0\0\0` (to match the
specified 4 bytes) if the `align` option was enabled.

Signed-off-by: Juan Cruz Viotti <jviottidc@gmail.com>
  • Loading branch information
jviotti committed Apr 21, 2016
1 parent 6995eab commit 296c757
Show file tree
Hide file tree
Showing 3 changed files with 31 additions and 1 deletion.
1 change: 1 addition & 0 deletions README.md
Expand Up @@ -20,6 +20,7 @@ written to this stream must be a `string` or a `buffer`.
- `chunkSize`: `integer` - Size in bytes of the desired chunks. - `chunkSize`: `integer` - Size in bytes of the desired chunks.
- `opts` - `opts`
- `flush`: `boolean` - Optional. Flush incomplete chunk data on stream end. Default is `false`. - `flush`: `boolean` - Optional. Flush incomplete chunk data on stream end. Default is `false`.
- `align`: `boolean` - Optional. Pad incomplete chunk data on stream end. Should be used in combination with `flush`. Default is `false`.
- `encoding`: `string` - Optional. Encoding of String chunks. Must be a valid Buffer encoding, such as `utf8` or `ascii`. - `encoding`: `string` - Optional. Encoding of String chunks. Must be a valid Buffer encoding, such as `utf8` or `ascii`.


## Simple Example ## Simple Example
Expand Down
7 changes: 7 additions & 0 deletions index.js
Expand Up @@ -45,6 +45,13 @@ module.exports = function (chunkSize, opts) {
var flushFunction; var flushFunction;
if (flush) { if (flush) {
flushFunction = function (next) { flushFunction = function (next) {

if (opts.align) {
var remaining = new Buffer(chunkSize - buffer.length);
remaining.fill(0);
buffer = Buffer.concat([ buffer, remaining ], chunkSize);
}

this.push(buffer); this.push(buffer);
next(); next();
}; };
Expand Down
24 changes: 23 additions & 1 deletion test/flush.js
Expand Up @@ -38,4 +38,26 @@ test('Test flush option', function (t) {
chunkerFlush.write('56'); chunkerFlush.write('56');
chunkerFlush.end(); chunkerFlush.end();


}); });

test('Test align option', function (t) {
t.plan(1);

var optsFlushAlign = {
flush: true,
align: true,
encoding: 'utf8'
}

function checkFlushAlign(data) {
t.equals(data, '12345\0\0\0', 'Received flush data');
}
var chunkerFlushAlign = Chunker(4, optsFlushAlign);
var concatStreamFlushAlign = concat(checkFlushAlign);
chunkerFlushAlign.pipe(concatStreamFlushAlign);
chunkerFlushAlign.write('12');
chunkerFlushAlign.write('34');
chunkerFlushAlign.write('5');
chunkerFlushAlign.end();

});

0 comments on commit 296c757

Please sign in to comment.