Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Initial import; this is just a range coder w/ some tests.
Range Coder ported from http://www.compressconsult.com/rangecoder
- Loading branch information
0 parents
commit c55af7b
Showing
9 changed files
with
659 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
node_modules | ||
*~ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,114 @@ | ||
# dmcjs | ||
|
||
`dmcjs` is a fast pure JavaScript implementation of Dynamic Markov Chain | ||
compression/decompression. It is written by C. Scott Ananian. | ||
The Range Coder used is a port of Michael Schindler's C range coder, | ||
found at http://www.compressconsult.com/rangecoder. | ||
|
||
## How to install | ||
|
||
``` | ||
npm install dmcjs | ||
``` | ||
or | ||
``` | ||
volo add cscott/dmcjs | ||
``` | ||
|
||
This package uses | ||
[Typed Arrays](https://developer.mozilla.org/en-US/docs/JavaScript/Typed_arrays) | ||
and so requires node.js >= 0.5.5. Full browser compatibility table | ||
is available at [caniuse.com](http://caniuse.com/typedarrays); briefly: | ||
IE 10, Firefox 4, Chrome 7, or Safari 5.1. | ||
|
||
## Testing | ||
|
||
``` | ||
npm install | ||
npm test | ||
``` | ||
|
||
## Usage | ||
|
||
There is a binary available in bin: | ||
``` | ||
$ bin/dmcjs --help | ||
$ echo "Test me" | bin/dmcjs -z > test.dmc | ||
$ bin/dmcjs -d test.dmc | ||
Test me | ||
``` | ||
|
||
From JavaScript: | ||
``` | ||
var dmcjs = require('dmcjs'); | ||
var data = new Buffer('Example data', 'utf8'); | ||
var compressed = dmcjs.compressFile(data); | ||
var uncompressed = dmcjs.uncompressFile(compressed); | ||
// convert from array back to string | ||
var data2 = new Buffer(uncompressed).toString('utf8'); | ||
console.log(data2); | ||
``` | ||
There is a streaming interface as well. | ||
|
||
See the tests in the `tests/` directory for further usage examples. | ||
|
||
## Documentation | ||
|
||
`require('dmcjs')` returns a `dmcjs` object. It contains two main | ||
methods. The first is a function accepting one, two or three parameters: | ||
|
||
`dmcjs.compressFile = function(input, [output], [Number compressionLevel])` | ||
|
||
The `input` argument can be a "stream" object (which must implement the | ||
`readByte` method), or a `Uint8Array`, `Buffer`, or array. | ||
|
||
If you omit the second argument, `compressFile` will return a JavaScript | ||
array containing the byte values of the compressed data. If you pass | ||
a second argument, it must be a "stream" object (which must implement the | ||
`writeByte` method). | ||
|
||
The third argument may be omitted, or a number between 1 and 9 indicating | ||
a compression level (1 being largest/fastest compression and 9 being | ||
smallest/slowest compression). The default is `1`. `6` is about twice | ||
as slow but creates 10% smaller files. | ||
|
||
The second exported method is a function accepting one or two parameters: | ||
|
||
`dmcjs.decompressFile = function(input, [output])` | ||
|
||
The `input` parameter is as above. | ||
|
||
If you omit the second argument, `decompressFile` will return a | ||
`Uint8Array`, `Buffer` or JavaScript array with the decompressed | ||
data, depending on what your platform supports. For most modern | ||
platforms (modern browsers, recent node.js releases) the returned | ||
value will be a `Uint8Array`. | ||
|
||
If you provide the second argument, it must be a "stream", implementing | ||
the `writeByte` method. | ||
|
||
## Related projects | ||
|
||
* wikipedia article XXX | ||
* range coder source XXX | ||
|
||
## Other JavaScript compressors | ||
|
||
* https://github.com/cscott/lzjb LZJB | ||
* https://github.com/cscott/lzma-purejs LZMA | ||
* https://github.com/cscott/seek-bzip Bzip2 (random-access decompression) | ||
|
||
## License (GPLv2) | ||
|
||
This program is free software: you can redistribute it and/or modify | ||
it under the terms of the GNU General Public License as published by | ||
the Free Software Foundation, either version 2 of the License, or | ||
(at your option) any later version. | ||
|
||
This program is distributed in the hope that it will be useful, | ||
but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
GNU General Public License for more details. | ||
|
||
You should have received a copy of the GNU General Public License | ||
along with this program. If not, see http://www.gnu.org/licenses/. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,127 @@ | ||
#!/usr/bin/env node | ||
|
||
var program = require('commander'); | ||
var dmcjs = require('../'); | ||
var fs = require('fs'); | ||
|
||
program | ||
.version(dmcjs.version) | ||
.usage('-d|-z [infile] [outfile]') | ||
.option('-d, --decompress', | ||
'Decompress stdin to stdout') | ||
.option('-z, --compress', | ||
'Compress stdin to stdout') | ||
.option('-1', 'Fastest/largest compression') | ||
.option('-2') | ||
.option('-3') | ||
.option('-4') | ||
.option('-5') | ||
.option('-6') | ||
.option('-7') | ||
.option('-8') | ||
.option('-9', 'Slowest/smallest compression'); | ||
program.on('--help', function() { | ||
console.log(' If <infile> is omitted, reads from stdin.'); | ||
console.log(' If <outfile> is omitted, writes to stdout.'); | ||
}); | ||
program.parse(process.argv); | ||
|
||
if (!program.decompress) { program.compress = true; } | ||
|
||
if (program.decompress && program.compress) { | ||
console.error('Must specify either -d or -z.'); | ||
return; | ||
} | ||
|
||
var level = undefined; | ||
for (var l=1; l<=9; l++) { | ||
if (program[''+l]) { | ||
if (level) { | ||
console.error("Can't specify both -"+level+" and -"+l); | ||
return; | ||
} | ||
level = l; | ||
} | ||
} | ||
if (level && program.decompress) { | ||
console.error('Compression level has no effect when decompressing.'); | ||
return; | ||
} | ||
|
||
var makeInStream = function(in_fd) { | ||
var stat = fs.fstatSync(in_fd); | ||
var stream = { | ||
buffer: new Buffer(4096), | ||
pos: 0, | ||
end: 0, | ||
_fillBuffer: function() { | ||
this.end = fs.readSync(in_fd, this.buffer, 0, this.buffer.length); | ||
this.pos = 0; | ||
}, | ||
readByte: function() { | ||
if (this.pos >= this.end) { this._fillBuffer(); } | ||
if (this.pos < this.end) { | ||
return this.buffer[this.pos++]; | ||
} | ||
return -1; | ||
}, | ||
read: function(buffer, bufOffset, length) { | ||
if (this.pos >= this.end) { this._fillBuffer(); } | ||
var bytesRead = 0; | ||
while (bytesRead < length && this.pos < this.end) { | ||
buffer[bufOffset++] = this.buffer[this.pos++]; | ||
bytesRead++; | ||
} | ||
return bytesRead; | ||
} | ||
}; | ||
if (stat.size) { | ||
stream.size = stat.size; | ||
} | ||
return stream; | ||
}; | ||
|
||
var makeOutStream = function(out_fd) { | ||
return { | ||
buffer: new Buffer(4096), | ||
pos: 0, | ||
flush: function() { | ||
fs.writeSync(out_fd, this.buffer, 0, this.pos); | ||
this.pos = 0; | ||
}, | ||
writeByte: function(byte) { | ||
if (this.pos >= this.buffer.length) { this.flush(); } | ||
this.buffer[this.pos++] = byte; | ||
} | ||
}; | ||
}; | ||
|
||
var in_fd = 0, close_in = function(){}; | ||
var out_fd = 1, close_out = function(){}; | ||
if (program.args.length > 0) { | ||
in_fd = fs.openSync(program.args.shift(), 'r'); | ||
close_in = function() { fs.closeSync(in_fd); }; | ||
} | ||
if (program.args.length > 0) { | ||
out_fd = fs.openSync(program.args.shift(), 'w'); | ||
close_out = function() { fs.closeSync(out_fd); }; | ||
} | ||
|
||
var inStream = makeInStream(in_fd); | ||
var outStream= makeOutStream(out_fd); | ||
|
||
if (program.decompress) { | ||
dmcjs.decompressFile(inStream, outStream); | ||
outStream.flush(); | ||
close_in(); | ||
close_out(); | ||
return 0; | ||
} | ||
if (program.compress) { | ||
dmcjs.compressFile(inStream, outStream, level); | ||
outStream.flush(); | ||
close_in(); | ||
close_out(); | ||
return 0; | ||
} | ||
return 1; |
Oops, something went wrong.