How to "attach" source maps of JS files copied via CopyWebpackPlugin ?

[renepupil]

In our Symfony application, we are using webpack Encore to create the webpack configuration that generates our assets.

For some modern modules we have added entry points e.g. list_table: './assets/packages/Common/ListTable/Module.js' that we use with twig function {{ encore_entry_script_tags('list_table') }}, but we have a lot of "legacy JS files" in /assets/static/ that we simply copy via CopyWebpackPlugin to our dist directory and minify with the TerserWebpackPlugin and for these legacy files we use twig function {{ asset('pupil/assets/dist/modules/courses/registration/modals/email.js') }}.

The point I want to make from the start, that I can not replace usage of CopyWebpackPlugin with a dozen entry points.

In the output of running encore production --progress which runs webpack, SourceMapDevToolPlugin says it "generated" source maps for the copied legacy files e.g.

#12 122.5 <s> [webpack.Progress] 92% sealing asset processing SourceMapDevToolPlugin modules/courses/registration/modals/email.2e4d4bd2.js generate SourceMap
#12 122.5 <s> [webpack.Progress] 92% sealing asset processing SourceMapDevToolPlugin modules/courses/registration/modals/email.2e4d4bd2.js generated SourceMap

Unlike with the specified entry points, the SourceMapDevToolPlugin does not "attach" these files e.g. #12 123.4 <s> [webpack.Progress] 92% sealing asset processing SourceMapDevToolPlugin list_table.1806adc5.js attached SourceMap

I tried to "debug" webpack to understand the underlying issue, but as I have spent already a lot of time with this, so I first try to get an answer here but what I found out is that the "compiler" resolves the copied JS files as RawSource objects without source maps while the entry point JS files are resolved as SourceMapSource objects, but no idea how to get SourceMapSource objects for the copied JS files...

This is the dumped webpack config, reduced to relevant properties only:

{
  context: '/home/wwwroot/pupil',
  entry: {
    ...
    'bt5-npm': './assets/bt5-npm.js',
    'bt5-onload': './assets/bt5-onload.js',
    ...
    list_table: './assets/packages/Common/ListTable/Module.js',
    ...
  },
  mode: 'production',
  output: {
    clean: {},
    path: '/home/wwwroot/pupil/pupil/assets/dist',
    filename: '[name].[contenthash:8].js',
    assetModuleFilename: 'assets/[name].[hash:8][ext]',
    publicPath: '/pupil/assets/dist/',
    pathinfo: false
  },
  module: {
    rules: [
      {
        test: /\.(m?jsx?)$/ { [lastIndex]: 0 },
        exclude: /(node_modules|bower_components)/ { [lastIndex]: 0 },
        use: [
          {
            loader: '/home/wwwroot/pupil/node_modules/babel-loader/lib/index.js',
            options: {
              cacheDirectory: false,
              sourceType: 'unambiguous',
              presets: [
                [
                  '/home/wwwroot/pupil/node_modules/@babel/preset-env/lib/index.js',
                  {
                    modules: false,
                    targets: {},
                    useBuiltIns: 'usage',
                    corejs: 3
                  }
                ]
              ],
              plugins: [
                '@babel/plugin-proposal-class-properties'
              ]
            }
          }
        ]
      },
      ...
    ]
  },
  plugins: [
    ...
    DeleteUnusedEntriesJSPlugin { entriesToDelete: [ [length]: 0 ] },
    WebpackManifestPlugin {
      options: {
        basePath: 'pupil/assets/dist/',
        fileName: 'manifest.json',
        filter: [Function: filter] { [length]: 1, [name]: 'filter' },
        generate: undefined,
        map: [Function (anonymous)] { [length]: 1, [name]: '' },
        publicPath: null,
        removeKeyHash: /([a-f0-9]{32}\.?)/gi { [lastIndex]: 0 },
        seed: {},
        serialize: [Function: serialize] { [length]: 1, [name]: 'serialize' },
        sort: null,
        transformExtensions: /^(gz|map)$/i { [lastIndex]: 0 },
        useEntryKeys: false,
        writeToFileEmit: true
      }
    },
    DefinePlugin {
      definitions: { 'process.env.NODE_ENV': '"production"' }
    },
    WebpackNotifierPlugin {
      options: { title: 'Webpack Encore' },
      lastBuildSucceeded: false,
      isFirstBuild: true
    },
    FriendlyErrorsWebpackPlugin { ... },
    AssetOutputDisplayPlugin {
      outputPath: 'pupil/assets/dist',
      friendlyErrorsPlugin: FriendlyErrorsWebpackPlugin { ... }
    },
    CopyPlugin {
      patterns: [
        {
          from: './assets/static/',
          to: './[path][name].[hash:8].[ext]',
          globOptions: { ignore: [ '**/*.scss' ] }
        },
        ...
      ],
      options: {}
    },
    { apply: [Function: apply] { [length]: 1, [name]: 'apply' } },
    EntryPointsPlugin {
      publicPath: '/pupil/assets/dist/',
      outputPath: '/home/wwwroot/pupil/pupil/assets/dist',
      integrityAlgorithms: [ [length]: 0 ]
    }
  ],
  optimization: {
    minimizer: [
      TerserPlugin {
        options: {
          test: /\.[cm]?js(\?.*)?$/i { [lastIndex]: 0 },
          extractComments: true,
          parallel: true,
          include: undefined,
          exclude: /^.*\.min\.[a-z0-8]{8}\.js$/ { [lastIndex]: 0 },
          minimizer: {
            implementation: [AsyncFunction: terserMinify] {
              [length]: 4,
              [name]: 'terserMinify',
              getMinimizerVersion: [Function (anonymous)] { [length]: 0, [name]: '' },
              [Symbol(Symbol.toStringTag)]: 'AsyncFunction'
            },
            options: {}
          }
        }
      }
    ],
    splitChunks: { chunks: 'all', cacheGroups: {} }
  },
  watchOptions: { ignored: /node_modules/ { [lastIndex]: 0 } },
  devtool: 'source-map',
  performance: { hints: false },
  stats: {
    hash: false,
    version: false,
    timings: false,
    assets: false,
    chunks: false,
    modules: false,
    reasons: false,
    children: false,
    source: false,
    errors: false,
    errorDetails: false,
    warnings: false,
    publicPath: false,
    builtAt: false
  },
  resolve: {
    extensions: [
      '.wasm',   '.mjs',
      '.js',     '.json',
      '.jsx',    '.vue',
      '.ts',     '.tsx',
      '.svelte', [length]: 9
    ],
    alias: { Pupil: '/home/wwwroot/pupil/assets/packages' }
  },
  externals: [ [length]: 0 ]
}

As we need source maps for sentry, I want a source map file for each copied JS file in the same directory e.g. modules/courses/registration/modals/email.2e4d4bd2.js.map.

I also noticed that webpack generates numeric bundled JS files for these copied JS files e.g. 1677.a49ac51a.js with 1677.a49ac51a.js.map. 1677.a49ac51a.js contains minified code from our legacy JS files under /assets/static/. I have no idea why these files are generated.

What I have tried so far:

• set devtool to eval and eval-source-map manually, but then I don't get any source maps at all (maybe this option is disallowed for production).
• set TerserPlugin config options.terserOptions.sourceMap = true;, but this doesn't make a difference to not using that.
• checked if I can use the TerserPlugin to generate the source maps, but AFAIK there is no possibility option to generate them.

---

1. How to get source maps for the legacy JS files under /assets/static/ that we copy with the CopyWebpackPlugin ?
2. Why do we have bundled JS files for these copied JS files?

I really appreciate your help, if you need more information, pls ask and I will add them in the question.

[renepupil]

@sokra Is this a bug or expected behaviour ?

[alexander-akait]

Copy plugin just copy files and never generate source maps (because it happened before copying), if you need source maps, just copy them too. Where you take your legacy JS file? If you generate them, just enable source map generation and setup copy plugin to copy them too, we can't just generate source maps from raw file (without original mapping). If you need just minimize legacy files and use copy plugin, just use https://github.com/webpack-contrib/copy-webpack-plugin/?tab=readme-ov-file#transformAll, minime them and return array of JS and source maps files

[alexander-akait]

As specified in the question they lay under /assets/static/modules, so I don't generate them. Did you even read the question properly?

Don't be rude, I asked this to understand where the ORIGINAL file came from. It was generated by tool? By hand? Where did you take them? Why you just don't import them as a regular JS module?

Source map generation is enabled via devtool: 'source-map'.

Yes, but it works only for modules in graph, i.e. when you import them using import/require/new URL(...)/etc, you use copy plugin, and they are not in modules graph, they are just assets, like any binary files (like images/videos/any binary files), binary files don't have source maps

I don't understand that, AFAIK source maps are always generated from "original" source files, hence they are called "source maps". What "mapping" is missing if you have the actual source file?

When a plugin create an assets it uses a special structure in webpack to allow generate source maps (and regenerate them if original file was changed, i.e. RawSource and SourceMapSource), copy plugin don't do it, it create a structure for binary files, also hook for source maps generation happens early then copying assets.

So if these files were generated by tool, we need to create SourceMapSource and use hook before source map generation. That's is why I ask this firstly.

According to the transformAll documentation, transformAll "Allows you to modify the contents of multiple files and save the result to one file", which means it generates ONE file, but you said I should return an array for multiple files?

My mistake wrong link, I mean - https://github.com/webpack-contrib/copy-webpack-plugin/?tab=readme-ov-file#transformer

Also you can use https://github.com/webpack-contrib/terser-webpack-plugin?tab=readme-ov-file#terseroptions, using test to filter only copied files and use terserOptions.sourceMap: true to say terser always generate source maps

My first question was about where you take them to provide a valid solution with the best perf and without complex configurations

[renepupil]

Thank you for your response.

Don't be rude, I asked this to understand where the ORIGINAL file came from.

My response was not meant to be rude. I have no idea what you mean with ORIGINAL file. I can only say that JS files under /assets/static/modules are handwritten JS, hence the word "static".

Why you just don't import them as a regular JS module?

Because these are LEGACY JS files that add functions to the global window object and we can not just replace or refactor them easily.

When a plugin create an assets it uses a special structure in webpack to allow generate source maps

Ok, can you also elaborate why webpack says that it "generates" the source maps ?

(As we don't add entry points for those files, I would have expected them to be ignored entirely. But source maps are generated according to output)

My mistake wrong link, I mean - https://github.com/webpack-contrib/copy-webpack-plugin/?tab=readme-ov-file#transformer

I tried this approach yesterday. So, transformer only allows changing the file content being copied, hence I can not use transformer for the already existing "copy" entry, because then I would only have source maps and no longer the original copied file, right ?

So, my idea was to just duplicate the copy pattern for these files, the first pattern copies the files, the second pattern creates the source maps, but I would need the file hash that is created in the first pattern as it is needed for the filename attribute in the source map to properly specify the source file for the source map.

(Mitigating the hash is not an option, as it serves the purpose of mitigating browser cache issues for same names files)

Do you have an idea how I can create the source maps with the transformer using the same hash as in the copy of the original file ?

Also you can use https://github.com/webpack-contrib/terser-webpack-plugin?tab=readme-ov-file#terseroptions, using test to filter only copied files and use terserOptions.sourceMap: true to say terser always generate source maps

I tried terserOptions.sourceMap: true already, source maps are still not "attached", meaning the copied JS files under the public directory pupil/assets/dist/modules don't have source maps.

As described in the original question, it says it "generates" these files, but there is no "attached" log for them.

My first question was about where you take them to provide a valid solution with the best perf and without complex configurations

That is a viable request, but refactoring our JS files would take probably 3-4 weeks of fulltime (not to mention likely endless bugs in the process) work load, hence I would prefer a quick "hacky" solution that "just makes it work".

[alexander-akait]

Ok, can you also elaborate why webpack says that it "generates" the source maps ?

Webpack generates source maps, but copy plugin use non source map structure, for copy plugin everything are raw/binary assets, which can't have source maps.

I tried terserOptions.sourceMap: true already, source maps are still not "attached", meaning the copied JS files under the public directory pupil/assets/dist/modules don't have source maps.

As described in the original question, it says it "generates" these files, but there is no "attached" log for them.

Do you mean source maps were not copied? Can you clarify?

[renepupil]

Do you mean source maps were not copied? Can you clarify?

Source maps were not added to the public directory: Using terserOptions.sourceMap results in source maps not in our public pupil directory for pupil/assets/dist/modules beside the corresponding copied JS file (in general for all copied files that are copied with the CopyWebpackPlugin with configuration from: './assets/static/' no source maps exist after webpack "runs")

But I don't know what you mean with "souce maps copied"?
As they should be created and put in place in the process, right?

I can not tell you if they are created somewhere temporary, I can only see the end results which is that the source maps are missing.

But thanks for you effort in answering my question.

[alexander-akait]

I got it and see you problems, unfortunately, the official plugins were not designed for such a solution, terser only works with assets generated by webpack itself, copy plugin starts working after source maps were generated and files were minimized.

So I see two solution here:

1. Create a simple plugin (you can copy/paste logic from the copy plugin and just change and change state - https://github.com/webpack-contrib/copy-webpack-plugin/blob/master/src/index.js#L825 on more early, if you want you can send a PR and I will accept it)
2. If it is just copy and they are not build process (as you described above) you can just use create a command in your package.json using terser CLI and just run this command after webpack build