Sourcemaps lose track of line numbers with pragmas

[futuraprime]

I'm using require.js with Alex Sexton's handlebars plugin, which uses r.js pragmas fairly extensively. When I build my app with sourcemaps on, however, the generated sourcemaps lose track of the pragma lines, causing them to be off by several lines over the course of the build, like so:

console.log('l170');     // logs as on line 170
//>>excludeStart('excludeHbsParser', pragmas.excludeHbsParser)
console.log('l172');     // logs as on line 171

This error adds up over the course of concatenating.

[jrburke]

The pragma support is from many years ago, and it is kind of an oddball thing. The sourcemap support is more recent, and has frankly been nothing but headaches, getting different tools in the toolchain to cooperate and for figuring out what is a browser bug vs an actual issue with the sourcemap support.

I am open to pull requests with tests in this area (which means changes in r.js, so a pull request for the r.js repo), but just setting the expectation now that it is not a priority for me, so unlikely to be addressed any time soon. Apologies for any trouble that causes.

Closing for now, and any fix would be in the r.js repo.

[alex-kowalczyk]

Have the same problem. Is there any workaround for now, like replacing those //>> lines with empty lines, or something?

[bartzy]

@jrburke

What is the alternative for pragmas? I'm using the same popular Handlebars plugin and experiencing the same problem. These are the pragmas I have in my build.js file:

    pragmasOnSave: {
        //removes Handlebars.Parser code (used to compile template strings) set
        //it to `false` if you need to parse template strings even after build
        excludeHbsParser : true,
        // kills the entire plugin set once it's built.
        excludeHbs: true,
        // removes i18n precompiler, handlebars and json2
        excludeAfterBuild: true
    },

Thanks.

[jrburke]

I suppose an alternative would be to use the onBuildRead and onBuildWrite hooks to do your own sort of pragma support and modify or move away from the syntax that r.js looks for so it does not then reprocess them. Maybe by just adding comments to all the lines between the pragmas? Not sure if that is enough for the source maps to line up, just a thought.

[bartzy]

Is it possible to use has.js in these cases?

Why do pragmas break the source maps?

[jrburke]

The benefit of the pragmas or the has.js support in the r.js optimizer is to remove code that is not needed after a build. However, for source maps to work, the lines that are removed needed to be accounted for in the source maps, and that facility is not set up in either the pragma or has.js build support in r.js. If you just use your own sort of config module object, and it does if/else logic based on the config, then that is not removed by the build, and would still play well with the source maps.

[asciidisco]

I know it´s an old issue, but we recently had to deal with this to get our legacy applications (that heavily rely on pragmas) running with Sentry reporting.
Therefor I wrote some prototypical code (please don't judge, improve if you like) to deal with this problem. I ended up writing a sloppy pragma parser/exclude mechanism in an onBuildWritehook:

{
      preserveLicenseComments: false,
      generateSourceMaps: true,
      skipPragmas: true,
      // exclude everything within the pragmas of `excludeAfterBuild`
      // exclude Hbs & Hbs parser (handlebars loader plugin & handlebars parser) as we do not load templates dynamically (and only need the runtime)
      // exclude PO, as the language strings gets precompiled into javascript (and we do not load po files dynamically)
      // exclude MD, as markdown gets precompiled to plain javascript (and we do not load md files dynamically)
      pragmasOnSave: {
        excludeAfterBuild: true,
        excludeHbsParser: true,
        excludeHbs: true,
        excludePo: true,
        excludeMd: true
      },
      // THE MIRACOLOUS SOURCE MAP FIX
      // Requirejs source map implementation can't cope with pragmas (https://gist.github.com/mattsahr/4190206#file-require-main-js-L204-L234)
      // As we heavily rely on them, we wrote our own sloppy pragma parser & remove code
      // See RequireJs Bug description: https://github.com/requirejs/requirejs/issues/1054
      onBuildWrite: function (moduleName, modulePath, contents) {
        var newContents = [];
        // all pragma declarations start with `//>>`
        var pragmaIndicator = '//>>';
        // helper funtion to determine if this is a start or an end pragma
        var isPragma = function (line) {
          return line.trim().indexOf(pragmaIndicator + 'excludeStart') !== -1 || line.trim().indexOf(pragmaIndicator + 'excludeEnd') !== -1;
        };
        // helper function to determine the name of the pragma
        var getPragmaSymbol = function (line) {
          return line.trim().split('(')[1].split(',')[0].trim().replace(';', '').replace(')', '').replace(/'/g, '').replace(/"/g, '');
        };
        // herlper funtion to determine if this is a start or an end pragma
        var isStartingPragma = function (line) {
          return line.trim().split('(')[0].indexOf(pragmaIndicator + 'excludeStart') !== -1;
        };
        // if there are no pragmas in this file there is nothing to do for us, just return the raw contents
        if (!isPragma(contents)) return contents;
        // variables that indicate if we´re in a pragma replacing situation & for which pragma
        var excludeStarted = false;
        var currentPragma = null;
        var currentPragmaCount = 0;
        // as pragmas are line based (strip everything inbetween two comments)
        // we´re splitting the file by line to process it further
        contents.split(/\r?\n/).forEach(function (line) {
          // check if there is a pragma comment
          if (isPragma(line)) {
            // determine name of the pragma & if it starts an exclude or ends one
            var pragmaSymbol = getPragmaSymbol(line);
            var startingPragma = isStartingPragma(line);
            // if we have an exclude running & see another starting pragma with a different name, we´ll return (nothign to do, it already gets excluded)
            if (excludeStarted && startingPragma && (pragmaSymbol !== currentPragma)) return;
            // if we have an exclude running & see another starting pragma with the same name, we´ll count that & remember for the closing one
            if (excludeStarted && startingPragma && (pragmaSymbol === currentPragma)) currentPragmaCount++;
            // if its an end pragma, well check for the number of open pragmas with the same name
            if (excludeStarted && !startingPragma && (pragmaSymbol === currentPragma)) {
              // if we have more than one open pragma of the same name, well just return
              if (currentPragmaCount > 0) return currentPragmaCount--;
              // if that was the finishing pragma, we´ll reset the indicators
              currentPragma = null;
              currentPragmaCount = 0;
              excludeStarted = false;
              return;
            }
            // if there is no exclude running & the found pragma is within our pragma list, we set the indicators
            if (!excludeStarted && this.pragmasOnSave[pragmaSymbol]) {
              excludeStarted = true;
              currentPragma = pragmaSymbol;
            }
          // if we´re not in excluding mode, we´ll just add the line contents to the temp var
          } else if(!excludeStarted) {
            newContents.push(line);
          }
        }.bind(this));
        // combine the contents of the temp var with a new line & output them
        return newContents.join('\n');
      }
}

Please note that this code probably isn't bulletproof & only deals with pragmasOnSave, but maybe it helps you if you bump into that issue as well.