Skip to content
Permalink
Browse files

Implement an option to align final incomplete data chunk

Writing unaligned data causes `EINVAL` (from `write(2)`):

> EINVAL
>  fd is attached to an object which is unsuitable for writing; or the
>  file was opened with the O_DIRECT flag, and either the address
>  specified in buf, the value specified in count, or the current file
>  offset is not suitably aligned.

This pull request implements an opt-in option (to preserve backwards
compatibility) called `align`, that when used in conjunction to `flush`,
will cause the incomplete data chunk to be padded with null bytes to
match the passed chunk size.

For example, in the added test, the chunk size is 4 bytes, and we push
three chunks:

- `12`: 2 bytes
- `34`: 2 bytes
- `5`: 1 byte

The final incomplete data chunk will become `5\0\0\0` (to match the
specified 4 bytes) if the `align` option was enabled.

Signed-off-by: Juan Cruz Viotti <jviottidc@gmail.com>
  • Loading branch information...
Juan Cruz Viotti
Juan Cruz Viotti committed Apr 20, 2016
1 parent 6995eab commit 296c757dad13a66caee9ff71f19ab91f216e7bb4
Showing with 31 additions and 1 deletion.
  1. +1 −0 README.md
  2. +7 −0 index.js
  3. +23 −1 test/flush.js
@@ -20,6 +20,7 @@ written to this stream must be a `string` or a `buffer`.
- `chunkSize`: `integer` - Size in bytes of the desired chunks.
- `opts`
- `flush`: `boolean` - Optional. Flush incomplete chunk data on stream end. Default is `false`.
- `align`: `boolean` - Optional. Pad incomplete chunk data on stream end. Should be used in combination with `flush`. Default is `false`.
- `encoding`: `string` - Optional. Encoding of String chunks. Must be a valid Buffer encoding, such as `utf8` or `ascii`.

## Simple Example
@@ -45,6 +45,13 @@ module.exports = function (chunkSize, opts) {
var flushFunction;
if (flush) {
flushFunction = function (next) {

if (opts.align) {
var remaining = new Buffer(chunkSize - buffer.length);
remaining.fill(0);
buffer = Buffer.concat([ buffer, remaining ], chunkSize);
}

this.push(buffer);
next();
};
@@ -38,4 +38,26 @@ test('Test flush option', function (t) {
chunkerFlush.write('56');
chunkerFlush.end();

});
});

test('Test align option', function (t) {
t.plan(1);

var optsFlushAlign = {
flush: true,
align: true,
encoding: 'utf8'
}

function checkFlushAlign(data) {
t.equals(data, '12345\0\0\0', 'Received flush data');
}
var chunkerFlushAlign = Chunker(4, optsFlushAlign);
var concatStreamFlushAlign = concat(checkFlushAlign);
chunkerFlushAlign.pipe(concatStreamFlushAlign);
chunkerFlushAlign.write('12');
chunkerFlushAlign.write('34');
chunkerFlushAlign.write('5');
chunkerFlushAlign.end();

});

0 comments on commit 296c757

Please sign in to comment.
You can’t perform that action at this time.