Source maps not parsing with Webpack

[fullykubed]

Receiving parse errors when bundling project with Webpack. In particular, it does not look like the source-map-loader Webpack loader is correctly parsing the source maps in the project.

Details

• Webpack Version: 3.5.1
• OS version: Ubuntu 17.04
• xterm.js version: 2.9.2

Steps to reproduce

1. Relevant webpack configuration

devtool: 'source-map',

    module : {
        rules : [
            { 
                test: /\.tsx?$/, 
                use: ['awesome-typescript-loader']},
            {
                test: /\.scss$/,
                use: [
                    {
                        loader: "style-loader"
                    }, {
                        loader: "css-loader"
                    }, {
                        loader: "sass-loader"
                    }
                ]
            },
            {
                test: /\.css$/,
                use: [
                    {
                        loader: "style-loader"
                    }, {
                        loader: "css-loader"
                    }
                ]
            },
            {
                test: /\.woff($|\?)|\.woff2($|\?)|\.ttf($|\?)|\.eot($|\?)/,
                use: ['url-loader']
            },
            {
                test: /\.(jpe?g|png|gif|svg)$/i,
                use: [
                    'url-loader',
                    'image-webpack-loader'
                ]
            },
            {
                test: /\.js\.map$/,
                use: {
                    loader: 'source-map-loader'
                },
            },
            {
                test: /\.js$/,
                use: {
                    loader: 'source-map-loader'
                },
            }
        ]
    }

2. Example of importing into React Typescript project

class Home extends React.Component {

    componentDidMount() {
        var term = new Terminal();
        // Exposes the terminal in an existing DOM object.
        term.open(document.getElementById('xterm-container'));
    }

    render() {
        return (
            <div id="xterm-container">
            </div>
        );
    }
}

3. Error messages when running webpack

WARNING in ./node_modules/xterm/lib/addons/search/search.js.map
Module parse failed: /home/jack/Projects/C290/website/node_modules/source-map-loader/index.js!/home/jack/Projects/C290/website/node_modules/xterm/lib/addons/search/search.js.map Unexpected token (1:10)
You may need an appropriate loader to handle this file type.
| {"version":3,"sources":["../../../src/addons/search/search.ts"],"names":[],"mappings":";;AAIA,+CAA8C;AAQ9C,CAAC,UAAU,KAAK;IACd,EAAE,CAAC,CAAC,UAAU,IAAI,MAAM,CAAC,CAAC,CAAC;QAIzB,KAAK,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;IACzB,CAAC;IAAC,IAAI,CAAC,EAAE,CAAC,CAAC,OAAO,OAAO,KAAK,QAAQ,IAAI,OAAO,MAAM,KAAK,QAAQ,CAAC,CAAC,CAAC;QAIrE,MAAM,CAAC,OAAO,GAAG,KAAK,CAAC,OAAO,CAAC,aAAa,CAAC,CAAC,CAAC;IACjD,CAAC;IAAC,IAAI,CAAC,EAAE,CAAC,CAAC,OAAO,MAAM,IAAI,UAAU,CAAC,CAAC,CAAC;QAIvC,MAAM,CAAC,CAAC,aAAa,CAAC,EAAE,KAAK,CAAC,CAAC;IACjC,CAAC;AACH,CAAC,CAAC,CAAC,UAAC,QAAa;IAOf,QAAQ,CAAC,SAAS,CAAC,QAAQ,GAAG,UAAS,IAAY;QACjD,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC,CAAC;YACxB,IAAI,CAAC,YAAY,GAAG,IAAI,2BAAY,CAAC,IAAI,EAAE,QAAQ,CAAC,2BAA2B,CAAC,CAAC;QACnF,CAAC;QACD,MAAM,CAAgB,IAAI,CAAC,YAAa,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC;IAC1D,CAAC,CAAC;IAQF,QAAQ,CAAC,SAAS,CAAC,YAAY,GAAG,UAAS,IAAY;QACrD,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC,CAAC;YACxB,IAAI,CAAC,YAAY,GAAG,IAAI,2BAAY,CAAC,IAAI,EAAE,QAAQ,CAAC,2BAA2B,CAAC,CAAC;QACnF,CAAC;QACD,MAAM,CAAgB,IAAI,CAAC,YAAa,CAAC,YAAY,CAAC,IAAI,CAAC,CAAC;IAC9D,CAAC,CAAC;AACJ,CAAC,CAAC,CAAC","file":"search.js","sourceRoot":"."}
 @ ./node_modules/xterm/lib/addons ^\.\/.*$
 @ ./node_modules/xterm/lib/xterm.js
 @ ./client/hof-content.tsx
 @ ./client/hof.tsx
 @ ./client/app.tsx
 @ ./client/main.tsx
 @ multi (webpack)-dev-server/client?http://localhost:3000 webpack-dev-server/client?http://localhost:3000/ ./client/main.tsx

WARNING in ./node_modules/xterm/lib/addons/search/SearchHelper.js.map
Module parse failed: /home/jack/Projects/C290/website/node_modules/source-map-loader/index.js!/home/jack/Projects/C290/website/node_modules/xterm/lib/addons/search/SearchHelper.js.map Unexpected token (1:10)
You may need an appropriate loader to handle this file type.
| {"version":3,"file":"SearchHelper.js","sources":["../../../src/addons/search/SearchHelper.ts"],"names":[],"mappings":";;AAgBA;IACE,sBAAoB,SAAc,EAAU,4BAAiC;QAAzD,cAAS,GAAT,SAAS,CAAK;QAAU,iCAA4B,GAA5B,4BAA4B,CAAK;IAK7E,CAAC;IAQM,+BAAQ,GAAf,UAAgB,IAAY;QAC1B,EAAE,CAAC,CAAC,CAAC,IAAI,IAAI,IAAI,CAAC,MAAM,KAAK,CAAC,CAAC,CAAC,CAAC;YAC/B,MAAM,CAAC,KAAK,CAAC;QACf,CAAC;QAED,IAAI,MAAqB,CAAC;QAE1B,IAAI,QAAQ,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,CAAC,KAAK,CAAC;QAC3C,EAAE,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,gBAAgB,CAAC,YAAY,CAAC,CAAC,CAAC;YAEjD,QAAQ,GAAG,IAAI,CAAC,SAAS,CAAC,gBAAgB,CAAC,YAAY,CAAC,CAAC,CAAC,CAAC;QAC7D,CAAC;QAGD,GAAG,CAAC,CAAC,IAAI,CAAC,GAAG,QAAQ,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,CAAC,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,IAAI,EAAE,CAAC,EAAE,EAAE,CAAC;YACtF,MAAM,GAAG,IAAI,CAAC,WAAW,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;YACnC,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC;gBACX,KAAK,CAAC;YACR,CAAC;QACH,CAAC;QAGD,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC;YACZ,GAAG,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,QAAQ,EAAE,CAAC,EAAE,EAAE,CAAC;gBAClC,MAAM,GAAG,IAAI,CAAC,WAAW,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;gBACnC,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC;oBACX,KAAK,CAAC;gBACR,CAAC;YACH,CAAC;QACH,CAAC;QAGD,MAAM,CAAC,IAAI,CAAC,aAAa,CAAC,MAAM,CAAC,CAAC;IACpC,CAAC;IAQM,mCAAY,GAAnB,UAAoB,IAAY;QAC9B,EAAE,CAAC,CAAC,CAAC,IAAI,IAAI,IAAI,CAAC,MAAM,KAAK,CAAC,CAAC,CAAC,CAAC;YAC/B,MAAM,CAAC,KAAK,CAAC;QACf,CAAC;QAED,IAAI,MAAqB,CAAC;QAE1B,IAAI,QAAQ,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,CAAC,KAAK,CAAC;QAC3C,EAAE,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,gBAAgB,CAAC,cAAc,CAAC,CAAC,CAAC;YAEnD,QAAQ,GAAG,IAAI,CAAC,SAAS,CAAC,gBAAgB,CAAC,cAAc,CAAC,CAAC,CAAC,CAAC;QAC/D,CAAC;QAGD,GAAG,CAAC,CAAC,IAAI,CAAC,GAAG,QAAQ,GAAG,CAAC,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC;YACvC,MAAM,GAAG,IAAI,CAAC,WAAW,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;YACnC,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC;gBACX,KAAK,CAAC;YACR,CAAC;QACH,CAAC;QAGD,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC;YACZ,GAAG,CAAC,CAAC,IAAI,CAAC,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,CAAC,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,IAAI,GAAG,CAAC,EAAE,CAAC,GAAG,QAAQ,EAAE,CAAC,EAAE,EAAE,CAAC;gBACtF,MAAM,GAAG,IAAI,CAAC,WAAW,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;gBACnC,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC;oBACX,KAAK,CAAC;gBACR,CAAC;YACH,CAAC;QACH,CAAC;QAGD,MAAM,CAAC,IAAI,CAAC,aAAa,CAAC,MAAM,CAAC,CAAC;IACpC,CAAC;IAQO,kCAAW,GAAnB,UAAoB,IAAY,EAAE,CAAS;QACzC,IAAM,UAAU,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC;QACtD,IAAM,eAAe,GAAG,IAAI,CAAC,4BAA4B,CAAC,UAAU,EAAE,IAAI,CAAC,CAAC,WAAW,EAAE,CAAC;QAC1F,IAAM,SAAS,GAAG,IAAI,CAAC,WAAW,EAAE,CAAC;QACrC,IAAM,WAAW,GAAG,eAAe,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;QACvD,EAAE,CAAC,CAAC,WAAW,IAAI,CAAC,CAAC,CAAC,CAAC;YACrB,MAAM,CAAC;gBACL,IAAI,MAAA;gBACJ,GAAG,EAAE,WAAW;gBAChB,GAAG,EAAE,CAAC;aACP,CAAC;QACJ,CAAC;IACH,CAAC;IAOO,oCAAa,GAArB,UAAsB,MAAqB;QACzC,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC;YACZ,MAAM,CAAC,KAAK,CAAC;QACf,CAAC;QACD,IAAI,CAAC,SAAS,CAAC,gBAAgB,CAAC,YAAY,CAAC,MAAM,CAAC,GAAG,EAAE,MAAM,CAAC,GAAG,EAAE,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACzF,IAAI,CAAC,SAAS,CAAC,UAAU,CAAC,MAAM,CAAC,GAAG,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;QAC3E,MAAM,CAAC,IAAI,CAAC;IACd,CAAC;IACH,mBAAC;AAAD,CA3HA,AA2HC,IAAA;AA3HY,oCAAY","sourceRoot":"."}
 @ ./node_modules/xterm/lib/addons ^\.\/.*$
 @ ./node_modules/xterm/lib/xterm.js
 @ ./client/hof-content.tsx
 @ ./client/hof.tsx
 @ ./client/app.tsx
 @ ./client/main.tsx
 @ multi (webpack)-dev-server/client?http://localhost:3000 webpack-dev-server/client?http://localhost:3000/ ./client/main.tsx

The project works fine if I redirect the main option in the package.json to dist/xterm.js and also works if I delete the search.js.map and SearchHelper.js.map files. However, I would ideally like to preserve the package's integrity to maintain portability to other devices.

[lucat1]

Try changing this:

 { 
  test: /\.tsx?$/, 
  use: ['awesome-typescript-loader']},

to

 { 
  test: /\.(tsx|ts)?$/, 
  use: ['awesome-typescript-loader']},

Or try ignoring the node_modules folder(which should be done anyway):

 { 
  test: /\.(tsx|ts)?$/, 
  exclude: /node_modules/,
  use: ['awesome-typescript-loader']},

[fullykubed]

Thanks for the tip. I changed that but it still didn't work. However, I think I know why it breaks.

The /lib/xterm.js file has variable dependencies in the Terminal.loadAddon function (line 477). The way that webpack works is to go ahead and bundle together all possible resolutions to the variable dependency; in this case it tries to include every file in the the addons folder.

It gets tripped up on the .min.js files because webpack normally only accesses these files through their corresponding .js (or other format) file.

My dumb brain said that obviously we should process the .js.map files with the source-map-loader but that's really just for .js files, hence the error.

Changing this

            {
                test: /\.js\.map$/,
                use: {
                    loader: 'source-map-loader'
                },
            },

to

            {
                test: /\.js\.map$/,
                use: {
                    loader: 'file-loader'
                },
            },

will just bundle the file as-is without trying to process it in any way.

It just tripped me up because apparently this is a pretty uncommon problem since noone really intentionally loads .js.map files with a require statement directly. Hence the 2 hours of fruitless google searches on the issue.

Now the real question is how to get it so that the addon files don't get bundled by webpack and cause unnecessary bundle bloat and funky errors. I don't know if it would be better to do a PR or if there is some webpack config option that I am missing.

[parisk]

@jclangst thanks for reporting this. A PR is definitely welcome if there is a transparent way of fixing this.

Just mentioning that we are using the v3 branch, as base now.

[jnodev]

I had the very same issue, even though I wasn't using source-map-loader. I was able to get rid of the warnings by using the webpack ignore-loader as seen in this Stackoverflow post. This is the line I included in module.rules:

{ test: /node_modules.+xterm.+\.map$/, loader: 'ignore-loader' },

Possibly this aproach could also be used to prevent webpack from bundling the other addons.

[KagamiChan]

Here's another possible solution: use webpack.IngorePlugin

new webpack.IgnorePlugin(/^\.\/.*js.map$/ ,/.*xterm\/lib\/addons/),

[Tyriar]

I'm consolidating all bundling/loadAddon related issues into #1018, please comment on that issue if you have thoughts 😃