Fails to compile with Yarn PnP on Windows

[ylemkimon]

• Operating System: Windows 10 / Server 2019
• Node Version: 14.16.1 / 12.22.1
• Yarn Version: 2.2.0 / 2.4.1
• webpack Version: 5.33.2 / 5.36.1
• webpack-dev-server Version: 4.0.0-beta.0 / 4.0.0-beta.2
• Browser: N/A

• This is a bug
• This is a modification request

Code

I couldn't create a minimally reproducible repo at this time. However, necessary conditions for reproducing bugs seems to be:

• Windows
• Yarn PnP (nodeLinker: pnp, default on Yarn >=2)
• Multiple compilers (configuration array) Seems like a race condition, sometimes I cannot reproduce the issue and it's more likely if there are multiple compilers

Expected Behavior

Compile without any errors

Actual Behavior

CI log: https://github.com/ylemkimon/KaTeX/runs/2464979777?check_suite_focus=true

Compilation (webpack serve) fails with the following error:

D:\a\KaTeX\KaTeX\.yarn\$$virtual\webpack-virtual-433a40c0cc\0\cache\webpack-npm-5.36.1-0533c3a17e-d68088da31.zip\node_modules\webpack\lib\NormalModule.js:1059
		return this.generator.getTypes(this);
		                      ^

TypeError: Cannot read property 'getTypes' of undefined
    at NormalModule.getSourceTypes (D:\a\KaTeX\KaTeX\.yarn\$$virtual\webpack-virtual-433a40c0cc\0\cache\webpack-npm-5.36.1-0533c3a17e-d68088da31.zip\node_modules\webpack\lib\NormalModule.js:1059:25)
    at _ (D:\a\KaTeX\KaTeX\.yarn\$$virtual\webpack-virtual-433a40c0cc\0\cache\webpack-npm-5.36.1-0533c3a17e-d68088da31.zip\node_modules\webpack\lib\stats\DefaultStatsFactoryPlugin.js:1086:36)
    at D:\a\KaTeX\KaTeX\.yarn\$$virtual\webpack-virtual-433a40c0cc\0\cache\webpack-npm-5.36.1-0533c3a17e-d68088da31.zip\node_modules\webpack\lib\stats\DefaultStatsFactoryPlugin.js:2245:9
    at Hook.eval [as call] (eval at create (D:\a\KaTeX\KaTeX\.yarn\cache\tapable-npm-2.2.0-516ee6aa5b-f8ed725aed.zip\node_modules\tapable\lib\HookCodeFactory.js:19:10), <anonymous>:5:16)
    at D:\a\KaTeX\KaTeX\.yarn\$$virtual\webpack-virtual-433a40c0cc\0\cache\webpack-npm-5.36.1-0533c3a17e-d68088da31.zip\node_modules\webpack\lib\stats\StatsFactory.js:277:7
    at StatsFactory._forEachLevel (D:\a\KaTeX\KaTeX\.yarn\$$virtual\webpack-virtual-433a40c0cc\0\cache\webpack-npm-5.36.1-0533c3a17e-d68088da31.zip\node_modules\webpack\lib\stats\StatsFactory.js:100:19)
    at StatsFactory._create (D:\a\KaTeX\KaTeX\.yarn\$$virtual\webpack-virtual-433a40c0cc\0\cache\webpack-npm-5.36.1-0533c3a17e-d68088da31.zip\node_modules\webpack\lib\stats\StatsFactory.js:276:9)
    at StatsFactory.create (D:\a\KaTeX\KaTeX\.yarn\$$virtual\webpack-virtual-433a40c0cc\0\cache\webpack-npm-5.36.1-0533c3a17e-d68088da31.zip\node_modules\webpack\lib\stats\StatsFactory.js:137:16)
    at D:\a\KaTeX\KaTeX\.yarn\$$virtual\webpack-virtual-433a40c0cc\0\cache\webpack-npm-5.36.1-0533c3a17e-d68088da31.zip\node_modules\webpack\lib\stats\StatsFactory.js:215:24
    at Array.map (<anonymous>)

Logging the offending module by:

	getSourceTypes() {
		let type;
		try {
			type = this.generator.getTypes(this);
		} catch (e) {
			console.log(this);
			throw e;
		}
		return type;
	}

gives:

NormalModule {
  dependencies: [
    CommonJsSelfReferenceDependency {
      weak: false,
      optional: false,
      loc: [SourceLocation],
      range: [Array],
      base: 'module.exports',
      names: [],
      call: false
    }
  ],
  blocks: [],
  type: 'javascript/auto',
  context: 'C:\\cygwin64\\home\\ylem\\KaTeX-test\\KaTeX\\.yarn\\cache\\querystring-npm-0.2.0-421b870c92-1e76c51462.zip\\node_modules\\querystring',
  layer: null,
  needId: true,
  debugId: 1221,
  resolveOptions: undefined,
  factoryMeta: undefined,
  useSourceMap: true,
  useSimpleSourceMap: false,
  _warnings: undefined,
  _errors: undefined,
  buildMeta: {},
  buildInfo: {
    cacheable: true,
    parsed: true,
    fileDependencies: undefined,
    contextDependencies: undefined,
    missingDependencies: undefined,
    buildDependencies: undefined,
    valueDependencies: Map(1) { 'webpack/DefinePlugin' => [Set] },
    hash: 'ef63ea8c5f6d895ab0daf846c7375eaf',
    assets: undefined,
    assetsInfo: undefined,
    strict: true,
    topLevelDeclarations: Set(1) { 'hasOwnProperty' },
    snapshot: Snapshot {
      _flags: 513,
      startTime: 1619687723569,
      fileTimestamps: undefined,
      fileHashes: undefined,
      fileTshs: undefined,
      contextTimestamps: undefined,
      contextHashes: undefined,
      contextTshs: undefined,
      missingExistence: undefined,
      managedItemInfo: undefined,
      managedFiles: [Set],
      managedContexts: undefined,
      managedMissing: undefined,
      children: undefined
    }
  },
  presentationalDependencies: [
    ConstDependency {
      weak: false,
      optional: false,
      loc: [SourceLocation],
      expression: '',
      range: [Array],
      runtimeRequirements: null
    }
  ],
  request: 'C:\\cygwin64\\home\\ylem\\KaTeX-test\\KaTeX\\.yarn\\cache\\querystring-npm-0.2.0-421b870c92-1e76c51462.zip\\node_modules\\querystring\\decode.js',
  userRequest: 'C:\\cygwin64\\home\\ylem\\KaTeX-test\\KaTeX\\.yarn\\cache\\querystring-npm-0.2.0-421b870c92-1e76c51462.zip\\node_modules\\querystring\\decode.js',
  rawRequest: './decode',
  binary: false,
  parser: undefined,
  parserOptions: undefined,
  generator: undefined,
  generatorOptions: undefined,
  resource: 'C:\\cygwin64\\home\\ylem\\KaTeX-test\\KaTeX\\.yarn\\cache\\querystring-npm-0.2.0-421b870c92-1e76c51462.zip\\node_modules\\querystring\\decode.js',
  matchResource: undefined,
  loaders: [],
  error: null,
  _source: OriginalSource {
    _value: '// Copyright Joyent, Inc. and other Node contributors.\n' +
      '//\n' +
      '// Permission is hereby granted, free of charge, to any person obtaining a\n' +
      '// copy of this software and associated documentation files (the\n' +
      '// "Software"), to deal in the Software without restriction, including\n' +
      '// without limitation the rights to use, copy, modify, merge, publish,\n' +
      '// distribute, sublicense, and/or sell copies of the Software, and to permit\n' +
      '// persons to whom the Software is furnished to do so, subject to the\n' +
      '// following conditions:\n' +
      '//\n' +
      '// The above copyright notice and this permission notice shall be included\n' +
      '// in all copies or substantial portions of the Software.\n' +
      '//\n' +
      '// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS\n' +
      '// OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\n' +
      '// MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN\n' +
      '// NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,\n' +
      '// DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR\n' +
      '// OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE\n' +
      '// USE OR OTHER DEALINGS IN THE SOFTWARE.\n' +
      '\n' +
      "'use strict';\n" +
      '\n' +
      '// If obj.hasOwnProperty has been overridden, then calling\n' +
      '// obj.hasOwnProperty(prop) will break.\n' +
      '// See: https://github.com/joyent/node/issues/1707\n' +
      'function hasOwnProperty(obj, prop) {\n' +
      '  return Object.prototype.hasOwnProperty.call(obj, prop);\n' +
      '}\n' +
      '\n' +
      'module.exports = function(qs, sep, eq, options) {\n' +
      "  sep = sep || '&';\n" +
      "  eq = eq || '=';\n" +
      '  var obj = {};\n' +
      '\n' +
      "  if (typeof qs !== 'string' || qs.length === 0) {\n" +
      '    return obj;\n' +
      '  }\n' +
      '\n' +
      '  var regexp = /\\+/g;\n' +
      '  qs = qs.split(sep);\n' +
      '\n' +
      '  var maxKeys = 1000;\n' +
      "  if (options && typeof options.maxKeys === 'number') {\n" +
      '    maxKeys = options.maxKeys;\n' +
      '  }\n' +
      '\n' +
      '  var len = qs.length;\n' +
      '  // maxKeys <= 0 means that we should not limit keys count\n' +
      '  if (maxKeys > 0 && len > maxKeys) {\n' +
      '    len = maxKeys;\n' +
      '  }\n' +
      '\n' +
      '  for (var i = 0; i < len; ++i) {\n' +
      "    var x = qs[i].replace(regexp, '%20'),\n" +
      '        idx = x.indexOf(eq),\n' +
      '        kstr, vstr, k, v;\n' +
      '\n' +
      '    if (idx >= 0) {\n' +
      '      kstr = x.substr(0, idx);\n' +
      '      vstr = x.substr(idx + 1);\n' +
      '    } else {\n' +
      '      kstr = x;\n' +
      "      vstr = '';\n" +
      '    }\n' +
      '\n' +
      '    k = decodeURIComponent(kstr);\n' +
      '    v = decodeURIComponent(vstr);\n' +
      '\n' +
      '    if (!hasOwnProperty(obj, k)) {\n' +
      '      obj[k] = v;\n' +
      '    } else if (Array.isArray(obj[k])) {\n' +
      '      obj[k].push(v);\n' +
      '    } else {\n' +
      '      obj[k] = [obj[k], v];\n' +
      '    }\n' +
      '  }\n' +
      '\n' +
      '  return obj;\n' +
      '};\n',
    _valueAsBuffer: <Buffer 2f 2f 20 43 6f 70 79 72 69 67 68 74 20 4a 6f 79 65 6e 74 2c 20 49 6e 63 2e 20 61 6e 64 20 6f 74 68 65 72 20 4e 6f 64 65 20 63 6f 6e 74 72 69 62 75 74 ... 2349 more bytes>,
    _name: 'webpack://./.yarn/cache/querystring-npm-0.2.0-421b870c92-1e76c51462.zip/node_modules/querystring/decode.js'
  },
  _sourceSizes: undefined,
  _lastSuccessfulBuildMeta: {},
  _forceBuild: false,
  _isEvaluatingSideEffects: false,
  _addedSideEffectsBailout: undefined,
  _ast: null
}

For Bugs; How can we reproduce the behavior?

git clone https://github.com/KaTeX/KaTeX.git
cd KaTeX
yarn install
yarn start

The following workflow can also reproduce the issue: https://github.com/ylemkimon/KaTeX/blob/master/.github/workflows/ci.yml

[alexander-akait]

Hard to debug with reproducible example, maybe you can monkey patching and output this.generator? Maybe something wrong with your yarn env, maybe webpack bug, maybe something external, yep, here no magic

[ylemkimon]

@alexander-akait Logging this gives above, with this.generator being undefined. Could you guess what might cause the generator to be undefined and where to start debugging?

[edemaine]

Also, it's not minimal, but https://github.com/KaTeX/KaTeX is a (large-ish) reproduction repo. git clone, yarn, yarn start triggers the issue, whereas npm install, npm start works fine.

[alexander-akait]

I will look at this in near future

[alexander-akait]

@edemaine hm, [webpack-dev-middleware] Compiled successfully., tried to do different changes, no luck, but yes, nothing strange in your configuration...

[alexander-akait]

It was bug in webpack, should be fixed, please update webpack to the latest version, feel free to feedback

[edemaine]

I confirmed that the webpack update fixed our problem. Thank you for following up!