Skip to content
Browse files

It works! Sweet test coverage, too.

  • Loading branch information...
0 parents commit eb3ed5d9ef8a1e61f6d430ac85f399dd77c00a28 @boutell boutell committed Sep 23, 2012
73 README.md
@@ -0,0 +1,73 @@
+= node-netpbm
+
+node-netpbm scales and converts GIF, JPEG and PNG images asynchronously, without running out of memory, even if the original image is very large. It does this via the netpbm utilities, a classic package of simple Unix tools that convert image formats and perform operations row-by-row so that memory is not exhausted. If you have ever tried to import 16-megapixel JPEGs with gd or imagemagick, you know exactly why you need this package.
+
+node-netpbm also provides a simple way to check the dimensions of an existing image file without paying the full price of converting it.
+
+== System Requirements
+
+You must have the netpbm utilities installed, and you must have a modern version due to node-netpbm's support for preserving the alpha channel of PNG images. On Ubuntu systems this is all you need to do:
+
+ apt-get install netpbm
+
+`node-netpbm` is designed for use on Linux, MacOS X and other Unix systems. No guarantees are made that it will work on Windows systems or anywhere else where shell pipelines don't behave reasonably and/or simple utilities like `head` and `tail` do not exist. But we'll accept pull requests. Don't break the tests!
+
+== Converting and scaling images
+
+node-netpbm offers a very simple API for converting and scaling images:
+
+ var convert = require('netpbm').convert;
+
+ convert('input/file.png',
+ 'output/file.jpg',
+ { width: 300, height: 400 },
+ function(err) {
+ if (!err) {
+ console.log("Hooray, your image is ready!");
+ }
+ }
+ );
+
+This code creates an image as close to 300x400 pixels as possible without distorting the aspect ratio of the original image. See below for more information about the options available.
+
+node-netpbm will automatically detect file types from file extensions. Uppercase is automatically converted to lowercase in file extensions, and jpeg is accepted as a synonym for jpg.
+
+== Options for converting and scaling images
+
+The third parameter to `convert` is an object containing options such as `alpha`, `width`, `height` and `limit`.
+
+* If you specify `alpha: true` and you have a very up to date version of the netpbm utilities that includes the `pngtopam` and `pamrgbatopng` utilities (check at the command line), alpha channel will be preserved when scaling a PNG input file to a PNG output file. As of this writing Ubuntu does not include these in its netpbm package. For more information and source code download links, see the [netpbm site](http://netpbm.sourceforge.net/getting_netpbm.php). The "stable" and "advanced" tarballs have both utilities ("super stable" does not).
+
+* If you specify just `width`, the output image will be that wide, and the height will scale to maintain the aspect ratio of the original.
+
+* If you specify just `height`, the output image will be that tall, and the width will scale to maintain the aspect ratio of the original.
+
+* If you specify both `width` and `height` properties for the options parameter, the output image will be as close to that size as possible without changing the aspect ratio of the original. For instance, if the original is 2000x2000 and you specify 300x400, the output will be 300x300. If the original is 500x5000 and you specify 300x400, the output will be 40x400.
+
+A common use for the third approach is to specify the width you typically want but also specify a maximum width to avoid unwanted results if the original is extremely tall, like an infographic.
+
+* If you are processing many image uploads for many users simultaneously, spawning lots of image processing Unix pipelines asynchronously could use a lot of resources. To prevent this, node-netpbm automatically throttles the number of simultaneously pending pipelines to 10. Additional requests will automatically wait until a slot is available. You can override this by setting the `limit` option to a different value. There isn't much benefit in setting this option higher than the number of cores available to you. In fact, if you are using the cluster module to run a node process for each core, you might want to set `limit` to 1 so that each process does not spawn up to 10 image pipelines.
+
+* Additional options exist for advanced uses such as overriding the netpbm utilities used for each conversion. See the source code for details.
+
+== Obtaining image dimensions
+
+Here's how:
+
+var info = require('netpbm').info;
+
+info('file.jpg', function(err, result) {
+ if (!err) {
+ console.log("Type: " + result.type +
+ " width: " + result.width +
+ " height: " + result.height);
+ }
+});
+
+Like `convert`, `info` is asynchronous. If there is no error, the type, width and height are passed to the callback via the `result` object.
+
+The `type` property will contain `gif`, `jpg` or `png`. `width` and `height` are hopefully self-explanatory.
+
+You can also call `info` with three parameters: the filename, an options object, and the callback. Usually you won't need this, but `info` does support the same advanced parameters for overriding types as `convert` does. `info` currently does not support the `limit` option, however obtaining image dimensions has a much smaller impact on the system than actually converting or scaling a complete image.
+
+Although the `info` function is reasonably fast, you should not rely on calling it every time you display an image. For good performance you should cache everything you know about each image in your database.
254 node-netpbm.js
@@ -0,0 +1,254 @@
+var child_process = require('child_process');
+
+var active = 0;
+
+// Callback's first argument is error if any. Second
+// argument (if no error) is an object with width, height
+// and type properties (gif, jpg, png or as redefined by
+// options, see convert). The "options" argument is not
+// mandatory and is usually unnecessary for info().
+
+module.exports.info = function(fileIn, options, callback)
+{
+ if (typeof(options) === 'function') {
+ callback = options;
+ options = {};
+ }
+ if (!options) {
+ options = {};
+ }
+ options.infoOnly = true;
+ module.exports.convert(fileIn, null, options, callback);
+}
+
+module.exports.convert = function(fileIn, fileOut, options, callback)
+{
+ if (!options) {
+ options = {};
+ }
+ // By default no more than 10 image processing pipelines are forked at any one time.
+ // Additional requests wait patiently for their turn. If you are using the cluster
+ // module you might want a lower limit as each node process gets its own pool
+ // of image conversion processes
+ var limit = options.limit ? options.limit : 10;
+
+ var typeMap = options.typeMap ? options.typeMap : {
+ 'jpeg': 'jpg'
+ };
+
+ var jpegQuality = options.jpegQuality ? options.jpegQuality : 85;
+
+ var types = options.types ? options.types : [
+ {
+ name: 'jpg',
+ importer: 'jpegtopnm ',
+ exporter: 'pnmtojpeg -quality ' + jpegQuality + ' ',
+ fileCommandReports: 'JPEG'
+ },
+ {
+ name: 'png',
+ // Preseve alpha channel in PNGs. Generally they are not used
+ // in this context unless it's worth it. However most (all?)
+ // Linux distributions don't include a new enough netpbm, so
+ // default to not doing it
+ importer: options.alpha ? 'pngtopam -byrow -alphapam ' : 'pngtopnm -mix ',
+ exporter: 'pnmtopng ',
+ sameTypeExporter: options.alpha ? 'pamrgbatopng ' : 'pnmtopng ',
+ fileCommandReports: 'PNG'
+ },
+ {
+ name: 'gif',
+ importer: 'giftopnm ',
+ exporter: 'ppmquant 256 | ppmtogif ',
+ fileCommandReports: 'GIF'
+ }
+ ];
+
+ if (options.extraTypes) {
+ types = types.concat(options.extraTypes);
+ }
+
+ var typesByName = {};
+ var i;
+ for (i = 0; (i < types.length); i++) {
+ typesByName[types[i].name] = types[i];
+ }
+
+ var typeOut;
+ if (!options.infoOnly) {
+ var typeOut = typeByExtension(fileOut);
+ if (!typeOut) {
+ callback('unsupported output file extension: ' + fileOut);
+ return;
+ }
+ }
+
+ var typeIn = typeByExtension(fileIn);
+ if (typeIn) {
+ preparePipeline();
+ }
+ else
+ {
+ // typeByHeader is async
+ typeByHeader(fileIn, function(err, result) {
+ if (err) {
+ callback(err);
+ return;
+ }
+ typeIn = result;
+ preparePipeline();
+ });
+ }
+
+ function preparePipeline() {
+
+ // Coding convention: each time cmd is appended to, you are responsible for including a trailing space. Not the next guy.
+ var cmd = typesByName[typeIn].importer + "< " + escapeshellarg(fileIn) + " ";
+
+ if (options.infoOnly) {
+ var result = {};
+ // Due to the row-by-row processing of the netpbm utilities,
+ // reading the width and height from the intermediate
+ // .pam/.ppm file and then shutting down the pipeline is
+ // fast, much faster than completely converting the whole thing
+ // just to learn the dimensions would be. So this is not as
+ // inefficient as it seems
+ cmd += "| head -3 ";
+ result.type = typeIn;
+ child_process.exec(cmd, function(err, stdout, stderr) {
+ if (err) {
+ callback(err + ': ' + stderr);
+ return;
+ }
+ var lines = stdout.split(/[\r\n]+/);
+ // PAM files are different, sigh
+ if (lines[1].match(/^WIDTH (\d+)/) && lines[2].match(/^HEIGHT (\d+)/))
+ {
+ var matches = lines[1].match(/^WIDTH (\d+)/);
+ result.width = parseInt(matches[1]);
+ var matches = lines[2].match(/^HEIGHT (\d+)/);
+ result.height = parseInt(matches[1]);
+ } else {
+ var matches = lines[1].match(/^(\d+) (\d+)/);
+ if (matches) {
+ result.width = parseInt(matches[1]);
+ result.height = parseInt(matches[2]);
+ }
+ }
+ if (result.width && result.height) {
+ callback(null, result);
+ return;
+ }
+ callback("Unexpected netpbm output");
+ return;
+ });
+ return;
+ }
+
+ if (options.alpha) {
+ scaler = 'pamscale ';
+ fitter = '-xyfit ';
+ } else {
+ scaler = 'pnmscale ';
+ fitter = '-xysize ';
+ }
+
+ if (options.width && options.height) {
+ cmd += "| " + scaler + fitter + options.width + ' ' + options.height + ' ';
+ } else if (options.width) {
+ cmd += "| " + scaler + "-width " + options.width + ' ';
+ } else if (options.height) {
+ cmd += "| " + scaler + "-height " + options.height + ' ';
+ } else {
+ // Size unchanged
+ }
+
+ // Watermark image is centered on the main image. Must be a .pam file
+ // (with an alpha channel, for best results)
+ if (options.watermark) {
+ cmd += "| pamcomp -align=center -valign=middle " + escapeshellarg(options.watermark) + ' ';
+ }
+
+ var exporter = (typesByName[typeOut].sameTypeExporter && (typeIn === typeOut)) ? typesByName[typeOut].sameTypeExporter : typesByName[typeOut].exporter;
+ cmd += "| " + exporter + "> " + escapeshellarg(fileOut);
+
+ execPipeline();
+
+ function execPipeline() {
+ if (active >= limit) {
+ setTimeout(function() {
+ pipeline();
+ return;
+ },
+ 100);
+ }
+ active++;
+ child_process.exec(cmd, function(err, stdout, stderr) {
+ active--;
+ if (err) {
+ callback(err + ': ' + stderr);
+ }
+ else
+ {
+ // All is well - the desired result is in fileOut
+ callback(null);
+ }
+ });
+ }
+ }
+
+ function typeByExtension(filename) {
+ var result = filename.match(/\.(\w+)$/);
+ if (result) {
+ var extension = result[1];
+ extension = extension.toLowerCase();
+ if (typeMap[extension]) {
+ extension = typeMap[extension];
+ }
+ if (typesByName[extension]) {
+ return extension;
+ }
+ }
+ return false;
+ }
+
+ function typeByHeader(filename, callback) {
+ var cmd = 'file ' + escapeshellarg(filename);
+ child_process.exec(cmd, function(err, stdout, stderr) {
+ if (err) {
+ callback(err + ': ' + stderr);
+ return;
+ }
+ var i;
+ for (i = 0; (i < types.length); i++) {
+ var type = types[i];
+ if (stdout.indexOf(type.fileCommandReports) !== -1) {
+
+ callback(null, type.name);
+ return;
+ }
+ }
+ callback('Unknown');
+ });
+ }
+
+ // http://phpjs.org/functions/escapeshellarg:866
+ function escapeshellarg(arg) {
+ // Quote and escape an argument for use in a shell command
+ //
+ // version: 1109.2015
+ // discuss at: http://phpjs.org/functions/escapeshellarg
+ // + original by: Felix Geisendoerfer (http://www.debuggable.com/felix)
+ // + improved by: Brett Zamir (http://brett-zamir.me)
+ // * example 1: escapeshellarg("kevin's birthday");
+ // * returns 1: "'kevin\'s birthday'"
+ var ret = '';
+
+ ret = arg.replace(/[^\\]'/g, function (m, i, s) {
+ return m.slice(0, 1) + '\\\'';
+ });
+
+ return "'" + ret + "'";
+ }
+};
+
BIN tests/sample.gif
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
BIN tests/sample.jpg
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
BIN tests/sample.mystery
Binary file not shown.
BIN tests/sample.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
112 tests/test.js
@@ -0,0 +1,112 @@
+var failures = 0;
+
+var convert = require('../node-netpbm.js').convert;
+var info = require('../node-netpbm.js').info;
+var exec = require('child_process').exec;
+
+// Change me to true to test alpha preservation
+// support for PNG-to-PNG conversions. This won't
+// work for you if you don't have a newer netpbm
+// with several new commands, see the documentation.
+var alpha = false;
+
+// Make sure netpbm is available before plunging ahead
+
+requirements();
+
+function requirements() {
+ process.stdout.write('System requirements: ');
+ exec('pnmtopng --version && pngtopnm --version && pnmscale --version', function(err, stdout, stderr) {
+ if (err) {
+ console.log("NOT MET");
+ console.log("You do not have the netpbm utilities installed, or they are");
+ console.log("out of date. Make sure you have pnmtopng, pngtopnm and pamscale");
+ console.log("commands. Ubuntu hint: apt-get install netpbm");
+ process.exit(2);
+ }
+ console.log("MET");
+ test1();
+ });
+}
+
+// Pass-through conversion: GIF to GIF, no size change
+
+function test1() {
+ process.stdout.write('test 1: ');
+ convert('sample.gif', 'test1.gif', { alpha: alpha }, next('test1.gif', 'gif', 350, 361, test2));
+}
+
+// Convert GIF to JPEG
+
+function test2() {
+ process.stdout.write('test 2: ');
+ convert('sample.gif', 'test2.jpg', { alpha: alpha }, next('test2.jpg', 'jpg', 350, 361, test3));
+}
+
+// JPEG to JPEG with size change: as big as possible while
+// fitting in 300px x 300px without changing the aspect ratio
+
+function test3() {
+ process.stdout.write('test 3: ');
+ convert('sample.jpg', 'test3.jpg', { alpha: alpha, 'width': 300, 'height': 300}, next('test3.jpg', 'jpg', 224, 300, test4));
+}
+
+// JPEG to PNG with size change: width of exactly 300px
+
+function test4() {
+ process.stdout.write('test 4: ');
+ convert('sample.jpg', 'test4.png', { alpha: alpha, 'width': 300 }, next('test4.png', 'png', 300, 402, test5));
+}
+
+// PNG to PNG with size change: height of exactly 300px.
+// Note the 'file' command shows the alpha channel was preserved
+
+function test5() {
+ process.stdout.write('test 5: ');
+ convert('sample.png', 'test5.png', { alpha: alpha, 'height': 300 }, next('test5.png', 'png', 279, 300, test6));
+}
+
+// Just like test6, but the file type has to be determined
+// by inspection of the file
+
+function test6() {
+ process.stdout.write('test 6: ');
+ convert('sample.mystery', 'test6.png', { alpha: alpha, 'height': 300 }, next('test6.png', 'png', 279, 300, null));
+}
+
+// Confirm that we like the results, then call the next test if any
+
+function next(filename, type, width, height, nextTest) {
+ return function(err) {
+ if (err) {
+ console.log("FAILED: " + err);
+ }
+ else
+ {
+ // Confirm file type, width and height are actually correct
+ // before going on to the next test
+ info(filename, {}, function(err, info) {
+ if (err) {
+ console.log("FAILED: " + err);
+ } else if (info.type !== type) {
+ console.log("FAILED: type is " + info.type + ", not " + type);
+ } else if (info.width !== width) {
+ console.log("FAILED: width is " + info.width + ", not " + width);
+ } else if (info.height !== height) {
+ console.log("FAILED: height is " + info.height + ", not " + height);
+ } else {
+ console.log("SUCCESS");
+ }
+ if (nextTest) {
+ nextTest();
+ return;
+ }
+ // If any tests failed return a nonzero exit status
+ // so that tools like Jenkins can spot it
+ if (failures) {
+ process.exit(1);
+ }
+ });
+ }
+ };
+}
BIN tests/test1.gif
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
BIN tests/test2.jpg
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
BIN tests/test3.jpg
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
BIN tests/test4.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
BIN tests/test5.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
BIN tests/test6.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit eb3ed5d

Please sign in to comment.
Something went wrong with that request. Please try again.