Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Add helper for adding index based mappings from files #25

Closed
wants to merge 8 commits into from

2 participants

@twolfson

I just created node-jsmin-sourcemap and during its creation I needed index support from source-map.

As a result, I created the attached code to assist in my dilemma. I have included tests, updated the README, and bumped the patch version.

It should be a huge help for others when adding source-map's to their compressors/minifiers.

@fitzgen
Owner

I don't think this abstraction belongs in the main library.

@fitzgen fitzgen closed this
@twolfson

That's fair. Do you think we could add something to the README?

I feel like it would be a very useful resource for any developer generating sourcemaps off of character index versus line and column.

@fitzgen
Owner

I can make a "Additional Tools and Libraries" wiki page, and you can feel free to list it there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Oct 8, 2012
  1. @twolfson
  2. @twolfson

    Moving along with test

    twolfson authored
Commits on Oct 9, 2012
  1. @twolfson
  2. @twolfson

    Debugging coordmap

    twolfson authored
  3. @twolfson
  4. @twolfson
  5. @twolfson

    Bumped package version and added

    twolfson authored
    > source-map@0.1.3 test /home/todd/github/source-map
    > node test/run-tests.js
    
    ..................................
    34 / 34 tests passed. functionality
  6. @twolfson

    Added to README

    twolfson authored
This page is out of date. Refresh to see the latest.
View
34 README.md
@@ -152,6 +152,40 @@ should have the following properties:
Renders the source map being generated to a string.
+### SourceMapFileCollector
+
+An instance of the SourceMapFileCollector represents a source map which is being built in files.
+
+#### new SourceMapFileCollector(startOfSourceMap)
+
+To create a new one, you must pass an object with the following properties:
+
+* `file`: The filename of the generated source that this source map is
+ associated with.
+
+* `sourceRoot`: An optional root for all relative URLs in this source map.
+
+#### SourceMapFileCollector.prototype.addIndexMapping(mapping)
+
+Add code with an index based mapping to the file collection.
+
+The mapping object
+should have the following properties:
+
+* `src`: Filepath to original src.
+
+* `input`: Unminified JavaScript.
+
+* `output`: Minified JavaScript.
+
+* `map`: Map of character index to character index (number -> number)
+
+* `lineOffset`: An optional line offset to add to mappings.
+
+#### SourceMapFileCollector.prototype.toString()
+
+Renders the source map being generated to a string.
+
### SourceNode
SourceNodes provide a way to abstract over interpolating and/or concatenating
View
4 lib/source-map.js
@@ -16,9 +16,11 @@ requirejs.config({
requirejs([
'source-map/source-map-generator',
'source-map/source-map-consumer',
+ 'source-map/source-map-file-collector',
'source-map/source-node'
-], function (generatorModule, consumerModule, sourceNodeModule) {
+], function (generatorModule, consumerModule, fileCollectorModule, sourceNodeModule) {
exports.SourceMapGenerator = generatorModule.SourceMapGenerator;
exports.SourceMapConsumer = consumerModule.SourceMapConsumer;
+ exports.SourceMapFileCollector = fileCollectorModule.SourceMapFileCollector;
exports.SourceNode = sourceNodeModule.SourceNode;
});
View
118 lib/source-map/char-props.js
@@ -0,0 +1,118 @@
+/* -*- Mode: js; js-indent-level: 2; -*- */
+/*
+ * char-props from https://github.com/twolfson/char-props
+ * with slimmed down for source-map
+ */
+define(function (require, exports, module) {
+ /**
+ * Indexer constructor (takes index and performs pre-emptive caching)
+ * @constructor
+ * @param {String} input Content to index
+ */
+ function Indexer(input) {
+ this.input = input;
+
+ // Break up lines by line breaks
+ var lines = input.split('\n');
+
+ // Iterate over the lines until we reach the end or we hit our index
+ var i = 0,
+ len = lines.length,
+ line,
+ lineStart = 0,
+ lineEnd,
+ lineMap = {'length': len};
+ for (; i < len; i++) {
+ // Grab the line
+ line = lines[i];
+
+ // Calculate the line end (includes \n we removed)
+ lineEnd = lineStart + line.length + 1;
+
+ // Save the line to its map
+ lineMap[i] = {'start': lineStart, 'end': lineEnd};
+
+ // Overwrite lineStart with lineEnd
+ lineStart = lineEnd;
+ }
+
+ // Save the lineMap to this
+ this.lineMap = lineMap;
+ }
+ Indexer.prototype = {
+ /**
+ * Get the line of the character at a certain index
+ * @param {Number} index Index of character to retrieve line of
+ * @param {Object} [options] Options to use for search
+ * @param {Number} [options.minLine=0] Minimum line for us to search on
+ * TODO: The following still have to be built/implemented
+ * @param {Number} [options.maxLine=lines.length] Maximum line for us to search on
+ * @param {String} [options.guess="average"] Affects searching pattern -- can be "high", "low", or "average" (linear top-down, linear bottom-up, or binary)
+ * @returns {Number} Line number of character
+ */
+ 'lineAt': function (index, options) {
+ // Fallback options
+ options = options || {};
+
+ // TODO: We can binary search here
+ // Grab the line map and iterate over it
+ var lineMap = this.lineMap,
+ i = options.minLine || 0,
+ len = lineMap.length,
+ lineItem;
+
+ for (; i < len; i++) {
+ // TODO: If binary searching, this requires both above and below
+ // If the index is under end of the lineItem, stop
+ lineItem = lineMap[i];
+ if (index < lineItem.end) {
+ break;
+ }
+ }
+
+ // Return the line we stopped on
+ return i;
+ },
+ /**
+ * Get the column of the character at a certain index
+ * @param {Number} index Index of character to retrieve column of
+ * @returns {Number} Column number of character
+ */
+ 'columnAt': function (index) {
+ // Start at the index - 1
+ var input = this.input,
+ char,
+ i = index - 1;
+
+ // If the index is negative, return now
+ if (index < 0) {
+ return 0;
+ }
+
+ // Continue left until index < 0 or we hit a line break
+ for (; i >= 0; i--) {
+ char = input.charAt(i);
+ if (char === '\n') {
+ break;
+ }
+ }
+
+ // Return the col of our index - 1 (line break is not in the column count)
+ var col = index - i - 1;
+ return col;
+ }
+ };
+
+ function charProps(input) {
+ // Create and return a new Indexer with the content
+ var indexer = new Indexer(input);
+ return indexer;
+ }
+
+ // Expose Indexer to charProps
+ charProps.Indexer = Indexer;
+
+ // Export charProps
+ exports['char-props'] = charProps;
+
+});
View
96 lib/source-map/source-map-file-collector.js
@@ -0,0 +1,96 @@
+/* -*- Mode: js; js-indent-level: 2; -*- */
+/*
+ * Copyright 2011 Mozilla Foundation and contributors
+ * Licensed under the New BSD license. See LICENSE or:
+ * http://opensource.org/licenses/BSD-3-Clause
+ */
+define(function (require, exports, module) {
+
+ var SourceMapGenerator = require('source-map/source-map-generator').SourceMapGenerator;
+ var charPropsObj = require('source-map/char-props');
+ var charProps = charPropsObj['char-props'];
+
+ /**
+ * Collect multiple files into a source map
+ *
+ * - file: The filename of the generated source.
+ * - sourceRoot: An optional root for all URLs in this source map.
+ */
+ function SourceMapFileCollector(params) {
+ // Create and save a new SourceMapGenerator
+ var srcMapGenerator = new SourceMapGenerator(params);
+ this.srcMapGenerator = srcMapGenerator;
+ }
+
+ /**
+ * Add code with an index based mapping to our srcMapGenerator
+ * - params Object holding multiple parameters
+ * - params.src Filepath to original src
+ * - params.input Unminified JavaScript
+ * - params.output Minified JavaScript
+ * - params.map Map of character index to character index (number -> number)
+ * - params.lineOffset (OPTIONAL) Line offset to add to mappings
+ */
+ SourceMapFileCollector.prototype.addIndexMapping = function (params) {
+ // Localize items from params
+ var srcFile = params.src,
+ srcProps = charProps(params.input),
+ destProps = charProps(params.output),
+ codeMap = params.map,
+ lineOffset = params.lineOffset || 0;
+
+ // Grab the keys of the codeMap
+ // DEV: If columnAt starts to be the slow part, create a map which memoizes each of the indicies it `while` loops over -- an LRU is probably best here.
+ var srcMapGenerator = this.srcMapGenerator,
+ lastSrcLine = 0,
+ lastDestLine = 0,
+ srcPoints = Object.getOwnPropertyNames(codeMap);
+ srcPoints.forEach(function (srcPoint) {
+ // Get the line and col of the src
+ var srcLine = srcProps.lineAt(srcPoint, {'minLine': lastSrcLine}),
+ srcCol = srcProps.columnAt(srcPoint);
+
+ // Save the srcLine as our next guess
+ lastSrcLine = srcLine;
+
+ // Get the line and col of the dest
+ var destPoint = codeMap[srcPoint],
+ destLine = destProps.lineAt(destPoint, {'minLine': lastDestLine}),
+ destCol = destProps.columnAt(destPoint);
+
+ // Save the destLine for our next guess
+ lastDestLine = destLine;
+
+ // Create our mapping
+ var mapping = {
+ 'original': {
+ 'line': srcLine + 1,
+ 'column': srcCol
+ },
+ 'generated': {
+ 'line': destLine + 1 + lineOffset,
+ 'column': destCol
+ },
+ 'source': srcFile
+ };
+
+ // Add the mapping to our generator
+ srcMapGenerator.addMapping(mapping);
+ });
+
+ // Return this for a fluent interface
+ return this;
+ };
+
+ /**
+ * Export the current collection as a sourcemap
+ */
+ SourceMapFileCollector.prototype.toString = function () {
+ var srcMapGenerator = this.srcMapGenerator;
+ return srcMapGenerator.toString();
+ };
+
+ // Export SourceMapFileCollector
+ exports.SourceMapFileCollector = SourceMapFileCollector;
+
+});
View
5 package.json
@@ -1,7 +1,7 @@
{
"name": "source-map",
"description": "Generates and consumes source maps",
- "version": "0.1.2",
+ "version": "0.1.3",
"homepage": "https://github.com/mozilla/source-map",
"author": "Nick Fitzgerald <nfitzgerald@mozilla.com>",
"contributors": [ ],
@@ -17,5 +17,8 @@
],
"dependencies": {
"requirejs": "==0.26.0"
+ },
+ "scripts": {
+ "test": "node test/run-tests.js"
}
}
View
138 test/source-map/test-source-map-file-collector.js
@@ -0,0 +1,138 @@
+/* -*- Mode: js; js-indent-level: 2; -*- */
+/*
+ * Copyright 2011 Mozilla Foundation and contributors
+ * Licensed under the New BSD license. See LICENSE or:
+ * http://opensource.org/licenses/BSD-3-Clause
+ */
+define(function (require, exports, module) {
+
+ var SourceMapFileCollector = require('source-map/source-map-file-collector').SourceMapFileCollector;
+ var SourceMapGenerator = require('source-map/source-map-generator').SourceMapGenerator;
+
+ // Generate test data for mapping
+ var input = [
+ '// First line comment',
+ 'var test = {',
+ ' a: "b"',
+ '};'
+ ].join('\n'),
+ output = 'var test={a:"b"};',
+ srcFile = 'input.js';
+ // Generated by node-jsmin2
+ var coordmap = {"22":0,"23":1,"24":2,"25":3,"26":4,"27":5,"28":6,"29":7,"31":8,"33":9,"37":10,"38":11,"40":12,"41":13,"42":14,"44":15,"45":16};
+
+ // Generated by hand ;_;
+ var propmap = [{
+ generated: {'line': 1, 'column': 0}, // v
+ original: {'line': 2, 'column': 0},
+ source: srcFile
+ }, {
+ generated: {'line': 1, 'column': 1}, // a
+ original: {'line': 2, 'column': 1},
+ source: srcFile
+ }, {
+ generated: {'line': 1, 'column': 2}, // r
+ original: {'line': 2, 'column': 2},
+ source: srcFile
+ }, {
+ generated: {'line': 1, 'column': 3}, //
+ original: {'line': 2, 'column': 3},
+ source: srcFile
+ }, {
+ generated: {'line': 1, 'column': 4}, // t
+ original: {'line': 2, 'column': 4},
+ source: srcFile
+ }, {
+ generated: {'line': 1, 'column': 5}, // e
+ original: {'line': 2, 'column': 5},
+ source: srcFile
+ }, {
+ generated: {'line': 1, 'column': 6}, // s
+ original: {'line': 2, 'column': 6},
+ source: srcFile
+ }, {
+ generated: {'line': 1, 'column': 7}, // t
+ original: {'line': 2, 'column': 7},
+ source: srcFile
+ }, {
+ generated: {'line': 1, 'column': 8}, // =
+ original: {'line': 2, 'column': 9},
+ source: srcFile
+ }, {
+ generated: {'line': 1, 'column': 9}, // {
+ original: {'line': 2, 'column': 11},
+ source: srcFile
+ }, {
+ generated: {'line': 1, 'column': 10}, // a
+ original: {'line': 3, 'column': 2},
+ source: srcFile
+ }, {
+ generated: {'line': 1, 'column': 11}, // :
+ original: {'line': 3, 'column': 3},
+ source: srcFile
+ }, {
+ generated: {'line': 1, 'column': 12}, // "
+ original: {'line': 3, 'column': 5},
+ source: srcFile
+ }, {
+ generated: {'line': 1, 'column': 13}, // b
+ original: {'line': 3, 'column': 6},
+ source: srcFile
+ }, {
+ generated: {'line': 1, 'column': 14}, // "
+ original: {'line': 3, 'column': 7},
+ source: srcFile
+ }, {
+ generated: {'line': 1, 'column': 15}, // }
+ original: {'line': 4, 'column': 0},
+ source: srcFile
+ }, {
+ generated: {'line': 1, 'column': 16}, // ;
+ original: {'line': 4, 'column': 1},
+ source: srcFile
+ }];
+
+ exports['test some simple stuff'] = function (assert, util) {
+ var map = new SourceMapFileCollector({
+ file: 'foo.js',
+ sourceRoot: '.'
+ });
+
+ assert.ok(true);
+ };
+
+ exports['test generates the same mapping as SourceMapGenerator'] = function (assert, util) {
+ var generatorProps = {
+ file: 'min.js'
+ },
+ generator = new SourceMapGenerator(generatorProps);
+
+ // Loop over the property map and add it to the SourceMapGenerator
+ var i = 0,
+ len = propmap.length;
+ for (; i < len; i++) {
+ generator.addMapping(propmap[i]);
+ }
+
+ // Collect the source map
+ var genSourceMap = generator.toString();
+
+ // Generate another source map via SourceMapFileCollector
+ var fileCollector = new SourceMapFileCollector(generatorProps);
+
+ // Add the file coordinate mapping
+ fileCollector.addIndexMapping({
+ src: srcFile,
+ input: input,
+ output: output,
+ map: coordmap
+ });
+
+ // Collect the second source map
+ var fileSourceMap = fileCollector.toString();
+
+ // Assert that they generate the same map
+ assert.strictEqual(genSourceMap, fileSourceMap, 'The sourcemap from SourceMapGenerator does not match the source map from SourceMapFileCollector');
+ };
+
+});
Something went wrong with that request. Please try again.