Browse files

initial version

  • Loading branch information...
0 parents commit f5abbf894d18ca8632a8b18f58d927a1dc55c742 Jeremy Bornstein committed Dec 3, 2011
Showing with 881 additions and 0 deletions.
  1. +1 −0 AUTHORS
  2. +133 −0 README.md
  3. +615 −0 bastard.js
  4. +58 −0 package.json
  5. +74 −0 start.js
1 AUTHORS
@@ -0,0 +1 @@
+Jeremy Bornstein <jeremy@jeremy.org>
133 README.md
@@ -0,0 +1,133 @@
+BASTARD
+=======
+
+The purpose of bastard is to serve static content over the web quickly, according to best practices, in a way that is easy to set up, run, and administer. It is implemented as a module intended to be run from within node.js. It may be invoked as part of another server process, serving only URLs with a given set of prefixes, or it may be the entire server all on its own. It will automatically minify and compress data, cache the data on disk, and verify the validity of its cached data on startup. While running, it keeps cached data in memory and does not expire data from the cache. Restarts should be relatively quick and easy because minified/compressed data will be read from the disk cache on the first request for that item.
+
+Additionally, bastard will automatically generate cryptographic fingerprints for all files it serves. You can programmatically ask it for the current fingerprinted URL for a file so that you can use that URL in HTML you generate external to the server. When bastard serves fingerprinted files, they are served with very long cache times because those URLs should always serve the same content.
+
+Both CSS and Javascript are minified, and HTML will probably be added soon. Files of other types are not touched, though they will be compressed if they're not image files. (Image files are never gzipped by this software.)
+
+
+Installing
+==========
+
+ npm install bastard
+
+
+Running Standalone
+==================
+
+Configure the settings via npm. There are reasonable defaults but you definitely need to specify the base directory where your files are:
+
+ npm config bastard:base /path/with/good/intentions
+ npm start bastard
+
+If you are running the standalone server and want to programmatically find out the current fingerprint for a file, make a request for the file with an incorrect fingerprint such as "BASTARD". The server's response will contain the valid fingerprint, which you may then parse out and use in your own externally-generated HTML.
+
+
+Running from your own code
+==========================
+
+1. Create the bastard object:
+
+ var bastard = require ('bastard');
+ var Bastard = bastard.Bastard;
+ var bastardObj = new Bastard (config);
+
+ // if you want to load every file into the cache before you get started:
+ bastardObj.loadEveryFile (callback);
+
+2. Create your own HttpServer object and pass requests to it from within the associated handler:
+
+ var handled = bastardObj.possiblyHandleRequest (request, response);
+
+If the above function returns true, the request has been handled and you don't need to do anything else. Depending on how you want to structure your server, you can check bastard before or after your own URLs.
+
+
+3. To find out the current fingerprint of a file:
+
+ bastardObj.getFingerprint (filePath, basePath, callback);
+
+* filePath: full path to the file
+* basePath: path to the file within the base directory (may be the same as the URL path for the file)
+* callback: if present, will be called with the first argument being any error (or null) and the second argument being the fingerprint
+
+If callback is not present and the fingerprint is already known, it will be returned immediately as the result of the function call. If callback is not present and the fingerprint is not already known, the fingerprint will be internally calculated and null will be returned from the function call.
+
+You only need to specify one of filePath and basePath.
+
+For an example of how to run bastard from your own code, examine the file start.js in the bastard package.
+
+
+Configuration
+=============
+
+host Hostname or IP address at which to listen. If empty, will bind to all available IP addresses. (Default: empty)
+
+port Port number at which to listen. (Default: 80)
+
+base Directory where files to be served reside. (Default: empty)
+
+rawURLPrefix The prefix for URLs from which raw files should be served. These will be just as they are on disk: not minified, not compressed. (Default: /raw/)
+
+fingerprintURLPrefix The prefix for URLs from which fingerprinted files should be served. The fingerprint will appear in the URLs after this prefix followed by the relative pathname to the file in the base directory. (Default: /f/)
+
+urlPrefix The prefix for URLs from which non-fingerprinted files should be served.
+
+workingDir The location for temporary files to be kept. This includes on-disk copies of minified and compressed files that originate in the base directory. (Default: /tmp/bastard.dat)
+
+debug If true, turns on some debugging functionality. (Default: false)
+
+directories If true, will generate directory listings. (Not yet implemented.) (Default: false)
+
+
+
+Note that first the raw prefix is checked, then the fingerprint prefix, and then only the regular prefix--and the first match is considered to be definitive. This means that with the default values, if you have a directory called "raw" in your base directory, those files will never be served.
+
+
+Limitations
+===========
+
+If the mime type for a file begins with "image/", it will not be gzipped. All other files will be gzipped if the client indicates that it can understand gzipped data. This may not be the best choice for all file types.
+
+Does not do virtual hosting.
+
+
+Project Status
+==============
+
+This is a project built by the author for his own use. Contributions are welcomed.
+
+The public repository for the project is found at: TK
+
+
+License
+=======
+
+Copyright 2011, Jeremy Bornstein <jeremy@jeremy.org>
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in the
+ documentation and/or other materials provided with the distribution.
+
+ * Neither the name of the <organization> nor the
+ names of its contributors may be used to endorse or promote products
+ derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL JEREMY BORNSTEIN BE LIABLE FOR ANY
+DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
615 bastard.js
@@ -0,0 +1,615 @@
+'use strict';
+
+var childProcess = require ('child_process');
+var fs = require ('fs');
+var uglify = require ("uglify-js");
+var csso = require ("csso");
+var gzbz2 = require ("gzbz2");
+var mime = require ('mime');
+
+/*
+
+TODO:
+ commandline
+ npm packaging
+ renaming
+ use a mime type guesser to figure out mime types and whether or not a file is binary
+
+LATER:
+ optionally upload fingerprinted files to S3
+ generate S3 URLs for files
+ allow preloading of all files into memory
+
+*/
+
+// These are reusable
+var JSP = uglify.parser;
+var PRO = uglify.uglify;
+
+function minifyJavascript (data) {
+ try {
+ var ast = JSP.parse (data); // parse code and get the initial AST
+ ast = PRO.ast_mangle (ast); // get a new AST with mangled names
+ ast = PRO.ast_squeeze (ast); // get an AST with compression optimizations
+ return PRO.gen_code (ast); // compressed code here
+ }
+ catch (err) {
+ console.error ("Problem parsing/minifying Javascript: " + JSON.stringify(err));
+ return "// Problem parsing Javascript -- see server logs\n" + data;
+ }
+}
+
+function gzip (data) {
+ var compressor = new gzbz2.Gzip;
+ compressor.init ({level: 9});
+ var result0 = compressor.deflate (data/*, 'ascii'*/);
+ var result1 = compressor.end ();
+
+ var result = new Buffer (result0.length + result1.length);
+ result0.copy (result, 0, 0);
+ result1.copy (result, result0.length, 0);
+
+ var percent = Math.floor ((result.length * 100 / data.length) * 100 + 0.5)/100;
+ console.info ("Compression: " + data.length + " -> " + result.length + ' (' + percent + '%)');
+
+ return result;
+}
+
+
+
+function Bastard (config) {
+ var baseDir = config.base;
+ var errorHandler = config.errorHandler;
+ var storageDir = config.workingDir || '/tmp/bastard.dat';
+ var debug = config.debug;
+ var urlPrefix = config.urlPrefix;
+ var rawURLPrefix = config.rawURLPrefix;
+ var fingerprintURLPrefix = config.fingerprintURLPrefix;
+
+ if (baseDir.charAt (baseDir.length-1) != '/') baseDir += '/';
+
+ setupStorageDir ();
+
+ // console.info ("*** " + config.workingDir);
+ // console.info ("*** " + storageDir);
+
+ var me = this;
+ var CACHE_INFO_FILENAME = 'cache_info.json';
+ var cacheData = {};
+ if (errorHandler && !(errorHandler instanceof Function)) errorHandler = null;
+
+ var preprocessors = {
+ '.js': minifyJavascript,
+ '.css': csso.justDoIt,
+ '.html': null
+ };
+
+ var ONE_WEEK = 60 * 60 * 24 * 7;
+
+ function formatCacheRecord (cacheRecord) {
+ var keys = [];
+ for (var key in cacheRecord) keys.push (key);
+ keys.sort ();
+ var result = [];
+ for (var i = 0; i < keys.length; i++) {
+ var key = keys[i];
+ var value = cacheRecord[key];
+ var valueType = typeof value;
+ if (valueType == 'number') result.push (key + ': ' + value);
+ if (value == null) result.push (key + ': null');
+ else result.push (key + ': ' + value.toString ().substring(0,64));
+ }
+ return result.join ('; ');
+ }
+
+
+ function setupStorageDir () {
+ fs.stat (storageDir, function (err, statobj) {
+ if (err) {
+ if (err.code == 'ENOENT') {
+ console.info ("Storage dir does not exist yet.");
+ // does not exist. can we make it?
+ fs.mkdir (storageDir, 448 /* octal: 0700 */, function (exception) {
+ if (exception) throw exception;
+ console.info ("Created storage directory for processed file cache");
+ finishStorageDirSetup ();
+ });
+ } else {
+ throw 'Problem with working directory: ' + err.message;
+ }
+ } else {
+ if (!statobj.isDirectory ()) {
+ throw "Storage directory is something I can't work with.";
+ } else {
+ // it is a directory already.
+ finishStorageDirSetup ();
+ }
+ }
+ });
+
+ function loadOldCache (oldCache) {
+ function checkCacheRecord (path, record) {
+ // compare size and modtime with the live ones from the file.
+ // if those are the same, we assume the fingerprint and cached processed/compressed files are still good.
+ // NOTE that this is vulnerable to sabotage or disk errors, etc.
+
+ fs.stat (path, function (err, statObj) {
+ if (err) return;
+ // console.info ("* Rechecking file: " + path);
+ // console.info ("Stored size: " + record.rawSize);
+ // console.info (" Live size: " + statObj.size);
+ if (record.rawSize != statObj.size) return;
+ var cacheWhen = Date.parse (record.modified);
+ //console.info ("Stored mtime: " + cacheWhen);
+ //console.info (" Live mtime: " + statObj.mtime);
+ if (cacheWhen != statObj.mtime) return;
+ //console.info ("**** ELIGIBLE FOR REUSE");
+ record.reloaded = true;
+ cacheData[path] = record; // keep the info but load the data on demand only.
+ });
+ }
+
+
+ //we have filepath -> rawSize, fingerprint, modified
+ for (var filePath in oldCache) {
+ if (filePath.indexOf (baseDir) != 0) continue; // not in our current purview
+ var cacheRecord = oldCache[filePath];
+ checkCacheRecord (filePath, cacheRecord);
+ }
+
+
+ }
+
+
+ function finishStorageDirSetup () {
+ storageDir = fs.realpathSync (storageDir);
+ if (storageDir.charAt (storageDir.length-1) != '/') storageDir += '/';
+ me.storageDir = storageDir;
+ // console.info ("Using working directory: " + storageDir);
+
+ me.cacheInfoFilePath = me.storageDir + CACHE_INFO_FILENAME;
+
+ fs.readFile (me.cacheInfoFilePath, 'utf8', function (err, data) {
+ if (err) {
+ //console.warn ("Could not reload cache info.");
+ return;
+ }
+ try {
+ var oldCache = JSON.parse (data);
+ loadOldCache (oldCache);
+ }
+ catch (err) {
+ console.warn ("Could not parse reloaded cache info");
+ }
+
+ });
+
+ setupProcessedFileCacheDir ();
+ setupGzippedFileCacheDir ();
+
+ // TODO: read the cache info file
+ }
+ }
+
+ function setupProcessedFileCacheDir () {
+ var processedFileCacheDir = me.storageDir + 'processed';
+ fs.stat (processedFileCacheDir, function (err, statobj) {
+ if (err) {
+ if (err.code == 'ENOENT') {
+ console.info ("Processed file cache dir does not exist yet.");
+ // does not exist. can we make it?
+ fs.mkdir (processedFileCacheDir, 448 /* octal: 0700 */, function (exception) {
+ if (exception) throw exception;
+ console.info ("Created directory for processed files");
+ finishSetup ();
+ });
+ } else {
+ throw 'Problem with processed file cache directory: ' + err.message;
+ }
+ } else {
+ if (!statobj.isDirectory ()) {
+ throw "Processed file cache directory is something I can't work with.";
+ } else {
+ // it is a directory already.
+ finishSetup ();
+ }
+ }
+ });
+
+ function finishSetup () {
+ processedFileCacheDir = fs.realpathSync (processedFileCacheDir);
+ if (processedFileCacheDir.charAt (processedFileCacheDir.length-1) != '/') processedFileCacheDir += '/';
+ me.processedFileCacheDir = processedFileCacheDir;
+ // console.info ("Using directory for cached processed files: " + processedFileCacheDir);
+ }
+ }
+
+ function setupGzippedFileCacheDir () {
+ var gzippedFileCacheDir = me.storageDir + 'gzipped';
+ fs.stat (gzippedFileCacheDir, function (err, statobj) {
+ if (err) {
+ if (err.code == 'ENOENT') {
+ console.info ("Gzipped file cache dir does not exist yet.");
+ // does not exist. can we make it?
+ fs.mkdir (gzippedFileCacheDir, 448 /* octal: 0700 */, function (exception) {
+ if (exception) throw exception;
+ console.info ("Created directory for gzipped files");
+ finishSetup ();
+ });
+ } else {
+ throw 'Problem with gzipped file cache directory: ' + err.message;
+ }
+ } else {
+ if (!statobj.isDirectory ()) {
+ throw "Gzipped file cache directory is something I can't work with.";
+ } else {
+ // it is a directory already.
+ finishSetup ();
+ }
+ }
+ });
+
+ function finishSetup () {
+ gzippedFileCacheDir = fs.realpathSync (gzippedFileCacheDir);
+ if (gzippedFileCacheDir.charAt (gzippedFileCacheDir.length-1) != '/') gzippedFileCacheDir += '/';
+ me.gzippedFileCacheDir = gzippedFileCacheDir;
+ // console.info ("Using directory for cached gzipped files: " + gzippedFileCacheDir);
+ }
+ }
+
+ function prepareCacheForFile (filePath, basePath, callback) {
+ var cacheRecord = {};
+
+ function writeCacheData (filePath, data) {
+ // if there is any problem here, just bail with an informational message. errors are not critical.
+
+ var parts = filePath.split ('/');
+ var curDir = '';
+ var dirsToCheck = [];
+ for (var i = 0; i < parts.length - 1; i++) { // NOTE that we are skipping the last element, which is the filename itself.
+ curDir += '/' + parts[i];
+ dirsToCheck.push (curDir);
+ }
+ dirsToCheck.reverse (); // put the top dir at the end, so we can pop.
+
+ function checkNextDir () {
+ if (dirsToCheck.length == 0) {
+ doneMakingDirectories ();
+ return;
+ }
+
+ var dir = dirsToCheck.pop ();
+ fs.stat (dir, function (err, statObj) {
+ if (err) {
+ if (err.code != 'ENOENT') {
+ console.warn ("Unexpected error investigating directory: " + dir);
+ } else {
+ // did not exist. this is fine; create it.
+ fs.mkdir (dir, 448 /* octal: 0700 */, function (err) {
+ if (err) {
+ console.info ("Problem creating " + dir + ": " + err);
+ } else {
+ checkNextDir ();
+ }
+ });
+ }
+ } else {
+ if (!statObj.isDirectory ()) {
+ console.warn ("Should be a directory: " + dir);
+ } else {
+ checkNextDir ();
+ }
+ }
+ });
+ }
+ checkNextDir ();
+
+ function doneMakingDirectories () {
+ fs.writeFile (filePath, data, 'utf8', function (err) {
+ if (err) {
+ console.warn ("Could not write data into: " + basePath + ": " + err.message);
+ }
+ });
+ }
+ }
+
+
+ function prerequisitesComplete () {
+ //console.info ('Setting cache for file ' + fileName);
+ // TODO: do we want to NOT store the data if it was an error?
+ cacheData[filePath] = cacheRecord; // set it all at once
+ if (callback instanceof Function) callback (cacheRecord);
+ }
+
+ var dataComplete = false; // we need to know this explicitly, in case there was an error
+ var statComplete = false;
+ var fingerprintComplete = false;
+ var suffix = filePath.substring (filePath.lastIndexOf ('.'));
+ var preprocessor = preprocessors[suffix];
+ var mimeType = mime.lookup (suffix);
+ var charset = mime.charsets.lookup (mimeType);
+ if (charset) {
+ cacheRecord.contentType = mimeType + '; charset=' + charset;
+ cacheRecord.charset = charset;
+ } else {
+ cacheRecord.contentType = mimeType;
+ }
+
+ childProcess.execFile ('/usr/bin/env', ['openssl', 'dgst', '-sha256', filePath], function (err, stdout, stderr) {
+ if (err) {
+ console.error ("Error from fingerprinting: " + JSON.stringify (err));
+ cacheRecord.fingerprintError = err;
+ } else {
+ cacheRecord.fingerprint = stdout.substr (-65, 64);
+ //console.info ("Fingerprint: " + cacheRecord.fingerprint);
+ }
+ fingerprintComplete = true;
+ if (dataComplete && statComplete) prerequisitesComplete ();
+ });
+
+ fs.readFile (filePath, charset, function (err, data) {
+ if (err) {
+ //console.log("Error from file " + filePath + ": " + err);
+ cacheRecord.fileError = err;
+ } else {
+ if (rawURLPrefix) cacheRecord.raw = data; // only keep it if we might be asked for it later
+
+ if (!basePath) basePath = filePath.substring (baseDir.length);
+
+ // console.info ("Preprocessor: " + preprocessor);
+ cacheRecord.processed = (preprocessor) ? preprocessor (data) : data;
+ writeCacheData (me.processedFileCacheDir + basePath, cacheRecord.processed);
+
+ if (cacheRecord.contentType && cacheRecord.contentType.indexOf ('image/') != 0) {
+ cacheRecord.gzip = gzip (cacheRecord.processed);
+ writeCacheData (me.gzippedFileCacheDir + basePath + '.gz', cacheRecord.gzip);
+ } else {
+ console.info ("Not gzipping an image");
+ }
+ }
+ dataComplete = true;
+ if (statComplete && fingerprintComplete) prerequisitesComplete ();
+ });
+
+ fs.stat (filePath, function (err, stat) {
+ if (err) {
+ //console.log ("Err from stat on file: " + filePath);
+ } else {
+ cacheRecord.rawSize = stat.size;
+ cacheRecord.modified = stat.mtime;
+ }
+ statComplete = true;
+ if (dataComplete && fingerprintComplete) prerequisitesComplete ();
+ });
+ }
+
+ // NOTE: does this work for binary data? it should....
+ function serveDataWithEncoding (response, data, contentType, encoding, modificationTime, fingerprint, maxAgeInSeconds) {
+ var responseHeaders = {
+ 'Content-Length': data.length,
+ 'Content-Type': contentType,
+ 'Vary': 'Accept-Encoding',
+ 'Cache-Control': "max-age=" + maxAgeInSeconds
+ };
+ if (encoding) responseHeaders['Content-Encoding'] = encoding;
+ if (modificationTime) responseHeaders['Last-Modified'] = modificationTime;
+ if (fingerprint) responseHeaders['Etag'] = fingerprint;
+ response.writeHead (200, responseHeaders);
+ response.end (data, null /*'utf8'*/);
+ }
+
+ function serve (response, filePath, basePath, fingerprint, gzipOK, ifModifiedSince) {
+ // console.info ("Serving " + basePath + ' out of ' + filePath);
+ var cacheRecord = cacheData[filePath];
+
+ function serveFromCacheRecord (cacheRecordParam, isRefill) {
+ // console.info ("Serve " + basePath + " from cache record: " + formatCacheRecord (cacheRecordParam));
+ if (gzipOK && cacheRecordParam.contentType && cacheRecordParam.contentType.indexOf ('image/') == 0) {
+ gzipOK = false; // do not gzip image files.
+ }
+
+ function remakeCacheRecord (gzip) {
+ prepareCacheForFile (filePath, basePath, function (newCacheRecord) {
+ newCacheRecord.remade = true;
+ serveFromCacheRecord (newCacheRecord);
+ });
+ }
+
+
+ function refillCacheRecord (gzip) {
+ delete cacheRecordParam.reloaded;
+ if (gzip) {
+ fs.readFile (me.gzippedFileCacheDir + basePath + '.gz', null, function (err, fileData) {
+ if (!err) {
+ cacheRecord.gzip = fileData;
+ }
+ serveFromCacheRecord (cacheRecordParam, true);
+ });
+ return;
+ }
+
+ // not gzip; get the regular processed data
+ fs.readFile (me.processedFileCacheDir + basePath, cacheRecord.charset, function (err, fileData) {
+ if (!err) {
+ cacheRecord.processed = fileData;
+ }
+ serveFromCacheRecord (cacheRecordParam, true);
+ });
+ }
+
+ var data = (gzipOK && cacheRecordParam.gzip) ? cacheRecordParam.gzip : cacheRecordParam.processed;
+
+ if (!data) {
+ if (cacheRecordParam.reloaded && !isRefill) { // if it is a reloaded record and we haven't tried yet
+ refillCacheRecord (gzipOK);
+ return;
+ }
+
+ if (!cacheRecordParam.remade && !cacheRecordParam.fileError && !cacheRecordParam.fingerprintError) {
+ console.info ("Remaking...");
+ remakeCacheRecord (gzipOK);
+ return;
+ }
+
+ // check the specific error. TODO: cover more cases here?
+ var errorMessage;
+ var errorCode;
+ if (cacheRecordParam.fileError && cacheRecordParam.fileError.code == 'ENOENT') {
+ errorCode = 404;
+ errorMessage = "File not found.";
+ } else {
+ errorCode = 500;
+ errorMessage = "Internal error.";
+ console.error ("Problem serving " + filePath);
+ if (cacheRecordParam.fileError) console.error ("File error: " + JSON.stringify (fileError));
+ if (cacheRecordParam.fingerprintError) console.error ("Fingerprint error: " + JSON.stringify (fingerprintError));
+ }
+
+ if (errorHandler) {
+ errorHandler (response, errorCode, errorMessage);
+ } else {
+ response.writeHead (errorCode, {'Content-Type': 'text/plain; charset=utf-8'});
+ response.end (errorMessage, 'utf8');
+ }
+ return;
+ }
+
+ // if we have a fingerprint and it does not match, it is probably best to redirect to the current version, right?
+ // until we put in a mechanism for calling back out to the appserver for that, we'll just send an error.
+ if (fingerprint && fingerprint != cacheRecordParam.fingerprint) {
+ var errorMessage = "That file is out of date. Current fingerprint: " + cacheRecordParam.fingerprint;
+ if (errorHandler) {
+ errorHandler (response, 404, errorMessage);
+ } else {
+ response.writeHead (404, {'Content-Type': 'text/plain; charset=utf-8'});
+ response.end (errorMessage, 'utf8');
+ }
+ return;
+ }
+
+ var modificationTime = cacheRecordParam.modified;
+ if (ifModifiedSince && modificationTime && modificationTime <= ifModifiedSince) {
+ response.writeHead (304, {});
+ response.end ();
+ } else {
+ serveDataWithEncoding (response, data, cacheRecordParam.contentType, gzipOK ? 'gzip' : null, modificationTime, cacheRecordParam.fingerprint, ONE_WEEK);
+ }
+ }
+
+ if (cacheRecord) {
+ serveFromCacheRecord (cacheRecord);
+ } else {
+ prepareCacheForFile (filePath, basePath, serveFromCacheRecord);
+ }
+ }
+
+ me.getFingerprint = function (filePath, basePath, callback) {
+ var cacheRecord = cacheData[filePath];
+ var callbackOK = callback instanceof Function;
+
+ // if filePath is null but basePath is not, figure out filePath
+ if (!filePath && basePath) filePath = baseDir + basePath;
+
+ if (!callbackOK) {
+ if (cacheRecord) {
+ return cacheRecord.fingerprint;
+ } else {
+ prepareCacheForFile (filePath, basePath);
+ return null;
+ }
+ }
+
+ function serveFromCacheRecord (cacheRecordParam) {
+ response.writeHead (200, {'Content-Type': 'text/plain'});
+ response.end (errorMessage, 'utf8');
+ }
+
+ if (cacheRecord) {
+ callback (cacheRecord.fingerprintErr, cacheRecord.fingerprint);
+ } else {
+ prepareCacheForFile (filePath, basePath, function (cacheRecord) {
+ callback (cacheRecord.fingerprintErr, cacheRecord.fingerprint);
+ });
+ }
+ }
+
+ var fingerprintPrefixLen = fingerprintURLPrefix.length;
+ var urlPrefixLen = urlPrefix.length;
+ me.possiblyHandleRequest = function (request, response) {
+ // console.info ("PFC maybe handling: " + request.url);
+ // console.info ('fup: ' + fingerprintURLPrefix);
+ // console.info ('up: ' + urlPrefix);
+ if (request.url.indexOf (fingerprintURLPrefix) == 0) {
+ var base = request.url.substring (fingerprintPrefixLen);
+ var slashPos = base.indexOf ('/');
+ var basePath = base.substring (slashPos + 1);
+ var fingerprint = base.substring (0, slashPos)
+ var filePath = baseDir + basePath;
+ // console.info (" fingerprint filePath: " + filePath);
+ // console.info (" fingerprint: " + fingerprint);
+ var acceptEncoding = request.headers['accept-encoding'];
+ var gzipOK = acceptEncoding && (acceptEncoding.split(',').indexOf ('gzip') >= 0);
+ var ifModifiedSince = request.headers['if-modified-since']; // fingerprinted files are never modified, so what do we do here?
+ serve (response, filePath, basePath, fingerprint, gzipOK, ifModifiedSince);
+ return true;
+ }
+ if (request.url.indexOf (urlPrefix) == 0) {
+ var basePath = request.url.substring (urlPrefixLen);
+
+ var filePath = baseDir + basePath;
+ // console.info (" filePath: " + filePath);
+ var acceptEncoding = request.headers['accept-encoding'];
+ var gzipOK = acceptEncoding && (acceptEncoding.split(',').indexOf ('gzip') >= 0);
+ var ifModifiedSince = request.headers['if-modified-since']; // fingerprinted files are never modified, so what do we do here?
+ serve (response, filePath, basePath, null, gzipOK, ifModifiedSince);
+ return true;
+ }
+ // console.info ("NO MATCH: " + request.url);
+ return false; // do not want
+ }
+
+ var prefixLengthToRemove = baseDir.length;
+ me.urlForFile = function (filePath) {
+ var basePath = filePath.substring (prefixLengthToRemove);
+
+ var fingerprint = me.getFingerprint (filePath, basePath);
+
+ if (fingerprint) {
+ return fingerprintURLPrefix + fingerprint + '/' + basePath;
+ } else {
+ return urlPrefix + basePath;
+ }
+ }
+
+ me.loadEveryFile = function (callback) {
+ var callbackOK = callback instanceof Function;
+ if (callbackOK) callback ("Not yet implemented");
+ }
+
+ me.cleanupForExit = function (tellMeWhenDone, eventName) {
+ console.info ("\nCleaning up Bastard.");
+ var trimmed = {};
+ for (var fileName in cacheData) {
+ var cacheRecord = cacheData[fileName];
+ var record = {
+ fingerprint: cacheRecord.fingerprint,
+ rawSize: cacheRecord.rawSize,
+ modified: cacheRecord.modified,
+ contentType: cacheRecord.contentType,
+ charset: cacheRecord.charset
+ };
+ trimmed[fileName] = record;
+ }
+ //console.info ("Will write data to: " + me.cacheInfoFilePath);
+ fs.writeFile (me.cacheInfoFilePath, JSON.stringify (trimmed), 'utf8', function (err) {
+ if (err) {
+ console.info ("Problem writing :" + me.cacheInfoFilePath + ': ' + err.message);
+ }
+ if (tellMeWhenDone && eventName) tellMeWhenDone.emit (eventName);
+ });
+
+ }
+}
+
+
+exports.Bastard = Bastard;
58 package.json
@@ -0,0 +1,58 @@
+{
+ "private": true,
+ "name": "bastard",
+ "version": "0.5.0",
+ "description": "A webserver for static files that does things right.",
+ "keywords": "webserver, fingerprint, static",
+ "homepage": "http://jeremy.org/bastard/",
+
+ "bugs": {
+ "email" : "jeremy-bastard@jeremy.org"
+ },
+
+ "author": {
+ "name": "Jeremy Bornstein",
+ "email": "jeremy@jeremy.org",
+ "url": "http://jeremy.org/"
+ },
+
+ "files": ["bastard.js", "start.js", "README.md"],
+ "main": "bastard.js",
+ "bin": {
+ "bastard" : "./start.js"
+ },
+
+ "NOT_REALLY_man" : [ "./man/bastard.1" ],
+ "repository": {
+ "type": "hg",
+ "url" : "ssh://hg.kuhu.org/bastard"
+ },
+
+ "config": {
+ "port": 80,
+ "host": null,
+ "base": null,
+ "debug": false,
+ "directories": false,
+ "rawURLPrefix": "/raw/",
+ "fingerprintURLPrefix": "/f/",
+ "urlPrefix": "/",
+ "workingDir": "/tmp/bastard"
+ },
+
+ "scripts": {"start": "node start.js"},
+
+ "dependencies": {
+ "uglify-js": ">=1.1.1",
+ "csso": ">=1.2.8",
+ "gzbz2": ">=0.1.4",
+ "mime": ">=1.2.4"
+ },
+
+ "engines": {
+ "node": ">=0.4.12",
+ "npm": ">=1.0.106"
+ }
+
+
+}
74 start.js
@@ -0,0 +1,74 @@
+#!/usr/bin/env node
+
+var http = require ('http');
+var fs = require ('fs');
+
+var bastard = require ('./bastard.js');
+
+function startBastard () {
+ var base = process.env.npm_package_config_base;
+ try {
+ var statObj = fs.statSync (base);
+ if (!statObj.isDirectory ()) {
+ throw "Configured base directory (" + base + ") is not a directory. To change config: npm config set bastard:base /path/with/good/intentions";
+ }
+ }
+ catch (err) {
+ if (err.code == 'ENOENT') {
+ throw "Configured base directory (" + base + ") does not exist. To change config: npm config set bastard:base /path/with/good/intentions";
+ }
+ throw "Problem accessing configured base directory (" + base + "): " + err.message + ". To change config: npm config set bastard:base /path/with/good/intentions";
+ }
+
+ var config = {
+ base: base,
+ debug: process.env.npm_package_config_debug == 'true',
+ directories: process.env.npm_package_config_directories == 'true',
+ rawURLPrefix: process.env.npm_package_config_rawURLPrefix,
+ fingerprintURLPrefix: process.env.npm_package_config_fingerprintURLPrefix,
+ urlPrefix: process.env.npm_package_config_urlPrefix
+ };
+ var bastardObj = new bastard.Bastard (config);
+
+ var host = process.env.npm_package_config_host;
+ var port = process.env.npm_package_config_port;
+
+ if (host == 'null' || host == '') host = null;
+
+ /*
+ function test () {
+ bastardObj.getFingerprint (null, '/html/example.html', function (err, fingerprint) {
+ console.info ("FINGERPRINT: " + fingerprint);
+ });
+ bastardObj.getFingerprint ('/Users/oao/src/fishing/base/html/example.html', null, function (err, fingerprint) {
+ console.info ("FINGERPRINT: " + fingerprint);
+ });
+ }
+ */
+
+ function webServerRequest (request, response) {
+ var handled = bastardObj.possiblyHandleRequest (request, response);
+ if (!handled) {
+ console.warn ("Request not handled by bastard: " + request.method + " " + request.url);
+ response.writeHead (404, {'Content-Type': 'text/plain; charset=utf-8'});
+ response.end ("Not found.");
+ }
+ }
+
+ var httpServer = http.createServer ();
+ httpServer.addListener ('request', webServerRequest);
+ httpServer.listen (port, host, function () {
+ if (host) console.log ('Server running at http://' + host + ':' + port + '/');
+ else console.log ('Server running at port ' + port + '.');
+
+ //test ();
+
+ process.once ('SIGINT', function () {
+ httpServer.close ();
+ bastardObj.cleanupForExit ();
+ });
+ });
+
+}
+
+startBastard ();

0 comments on commit f5abbf8

Please sign in to comment.