es-abstract@1.17.0-next.1 release breaking old applications

[derNoaa]

I am currently experiencing an issue where which was already raised by another person on stackoverflow where the latest release es-abstract@1.17.0-next.1 which is apparently being auto-installed by common libraries (redux-form, react-date) as a semver range.

Upon building the frontend with webpack and webpack dev server the following error occurs:

Error in ./~/object.entries/implementation.js
Module not found: Can't resolve 'es-abstract/2019/RequireObjectCoercible' in 'someDummyPath/host/client/node_modules/object.entries'

Upon investigating whether the path is really not existing I found out that it is indeed present -> therefore I don't quite understand why this not being resolved properly.

We are using npm modules on a locked version state which stayed the same for a few years - but some of them install dependencies using semver ranges which leads to es-abstract@1.17.0-next.1 being installed as a peer dependency.

Since we arent the owner of the npm modules we can't simply manually requirea locked version.

├─┬ INTERNAL_LIBRARY
│ └─┬ react-dates@10.1.1
│   ├─┬ airbnb-prop-types@2.15.0
│   │ ├─┬ function.prototype.name@1.1.2
│   │ │ └── es-abstract@1.17.0-next.1 
│   │ └─┬ object.entries@1.1.1
│   │   └── es-abstract@1.17.0-next.1 
│   └─┬ array-includes@3.1.0
│     └── es-abstract@1.17.0-next.1 
└─┬ redux-form@6.6.2
  └─┬ deep-equal@1.1.1
    └─┬ regexp.prototype.flags@1.3.0
      └── es-abstract@1.17.0-next.1 

[andjelboskovic]

I'm having the same issue

[ljharb]

Since the file is present there’s no reason it shouldn’t resolve. What do you mean by “dynamic dependency statement”? Nothing I’m aware of uses es-abstract as a peer dep, since it’s not stateful.

Is there any possibility your webpack config has some kind of special overrides for es-abstract?

[derNoaa]

Dynamic dependency statement is maybe a weird phrase but something like ^1.0.0 for instance as stated in the original post.

[andjelboskovic]

I've tried making a new clean project and importing only the library that uses airbnb-prop-types and object.entries.

While running: webpack --mode=production there is no error, but as soon as I switch to webpack --mode=development --watch I get the same error as in the original post.

[ljharb]

@norpisdev ok, "using semver ranges", which is a best practice (not using them is hostile; a lockfile is how an app-level consumer should pin things).

@andjelboskovic that sounds like an issue that should be filed with webpack then, as there's nothing es-abstract is doing that, say, lodash isn't doing (deep requires).

[derNoaa]

@ljharb thanks. I edited my post to use the correct phrasing.
I agree that that's a correct approach, but having breaking changes in minor or patch releases is something that should be pointed out.

Undenyibly some parts of our codebase are really outdated and need to be updated ASAP (webpack version 2.x ...) so for now I am running the development build under production flag similar how @andjelboskovic mentioned it. Which lets the code compile just fine.

[ninosaurus]

You updated minor version and you use #next version tag on some dependecies, I think that's not good way, you should use #next tag for this testing phase, for example package function.prototype.name (below is package-lock.json). I suggest to revert changes and create next tag for dependencies that use es-abstract#next

Previous version

"function.prototype.name": {
      "version": "1.1.1",
      "resolved": "https://registry.npmjs.org/function.prototype.name/-/function.prototype.name-1.1.1.tgz",
      "integrity": "sha512-e1NzkiJuw6xqVH7YSdiW/qDHebcmMhPNe6w+4ZYYEg0VA+LaLzx37RimbPLuonHhYGFGPx1ME2nSi74JiaCr/Q==",
      "requires": {
        "define-properties": "^1.1.3",
        "function-bind": "^1.1.1",
        "functions-have-names": "^1.1.1",
        "is-callable": "^1.1.4"
      }
    }

New minor version

"function.prototype.name": {
      "version": "1.1.2",
      "resolved": "https://registry.npmjs.org/function.prototype.name/-/function.prototype.name-1.1.2.tgz",
      "integrity": "sha512-C8A+LlHBJjB2AdcRPorc5JvJ5VUoWlXdEHLOJdCI7kjHEtGTpHQUiqMvCIKUwIsGwZX2jZJy761AXsn356bJQg==",
      "requires": {
        "define-properties": "^1.1.3",
        "es-abstract": "^1.17.0-next.1",
        "functions-have-names": "^1.2.0"
      },
      "dependencies": {
        "es-abstract": {
          "version": "1.17.0-next.1",
          "resolved": "https://registry.npmjs.org/es-abstract/-/es-abstract-1.17.0-next.1.tgz",
          "integrity": "sha512-7MmGr03N7Rnuid6+wyhD9sHNE2n4tFSwExnU2lQl3lIo2ShXWGePY80zYaoMOmILWv57H0amMjZGHNzzGG70Rw==",
          "requires": {
            "es-to-primitive": "^1.2.1",
            "function-bind": "^1.1.1",
            "has": "^1.0.3",
            "has-symbols": "^1.0.1",
            "is-callable": "^1.1.4",
            "is-regex": "^1.0.4",
            "object-inspect": "^1.7.0",
            "object-keys": "^1.1.1",
            "object.assign": "^4.1.0",
            "string.prototype.trimleft": "^2.1.0",
            "string.prototype.trimright": "^2.1.0"
          }
        }
      }
    },

Quick solution is that you override version buy manually check package-lock.json and find all packages that use es-abstract@next and put them as your project dependency with fixed version for example ``` npm i -D function.prototype.name@1.1.1 ```

[andjelboskovic]

Not sure what the real issue is, but @ninosaurus's suggestion worked.

I needed to override both dependencies from the error:

npm i -D function.prototype.name@1.1.1
npm i -D object.entries@1.1.0

[ljharb]

There are no breaking changes - this is a minor, and thus new things are added, and thus if you use an older version it won’t have that functionality.

Using the next tag is irrelevant since the dist-tag only matters when installing something by name without a semver specifier.

This is a problem with webpack in development mode, it seems, so I’m going to close this. Please file an issue with webpack.

[GabrielFerrarini]

While I can agree that the issue might not be here, can we be sure that it's a webpack issue? In our case, code is breaking while running tests with jest.

Test suite failed to run

    Cannot find module 'es-abstract/2019/ArraySpeciesCreate' from 'implementation.js'

      at Resolver.resolveModule (../home/devusr/node_modules/jest-resolve/build/index.js:221:17)
      at Object.<anonymous> (../home/devusr/node_modules/array.prototype.flat/implementation.js:3:26)

[GabrielFerrarini]

This is what node_modules/array.prototype.flat/implementation.js looks like in the working version installation in node_modules

var ES = require('es-abstract/es2019');

module.exports = function flat() {
	var O = ES.ToObject(this);
	var sourceLen = ES.ToLength(ES.Get(O, 'length'));

	var depthNum = 1;
	if (arguments.length > 0 && typeof arguments[0] !== 'undefined') {
		depthNum = ES.ToInteger(arguments[0]);
	}

	var A = ES.ArraySpeciesCreate(O, 0);
	ES.FlattenIntoArray(A, O, sourceLen, 0, depthNum);
	return A;
};

This is for the version failing

'use strict';

var ES = require('es-abstract/es2017');

var MAX_SAFE_INTEGER = Number.MAX_SAFE_INTEGER || (Math.pow(2, 53) - 1);

// eslint-disable-next-line max-params, max-statements
var FlattenIntoArray = function FlattenIntoArray(target, source, sourceLen, start, depth) {
	var targetIndex = start;
	var sourceIndex = 0;

	/*
	var mapperFunction;
	if (arguments.length > 5) {
		mapperFunction = arguments[5];
	}
	*/

	while (sourceIndex < sourceLen) {
		var P = ES.ToString(sourceIndex);
		var exists = ES.HasProperty(source, P);
		if (exists) {
			var element = ES.Get(source, P);
			/*
			if (typeof mapperFunction !== 'undefined') {
				if (arguments.length <= 6) {
					throw new TypeError('Assertion failed: thisArg is required when mapperFunction is provided');
				}
				element = ES.Call(mapperFunction, arguments[6], [element, sourceIndex, source]);
			}
			*/
			var shouldFlatten = false;
			if (depth > 0) {
				shouldFlatten = ES.IsArray(element);
			}
			if (shouldFlatten) {
				var elementLen = ES.ToLength(ES.Get(element, 'length'));
				targetIndex = FlattenIntoArray(target, element, elementLen, targetIndex, depth - 1);
			} else {
				if (targetIndex >= MAX_SAFE_INTEGER) {
					throw new TypeError('index too large');
				}
				ES.CreateDataPropertyOrThrow(target, ES.ToString(targetIndex), element);
				targetIndex += 1;
			}
		}
		sourceIndex += 1;
	}

	return targetIndex;
};

module.exports = function flat() {
	var O = ES.ToObject(this);
	var sourceLen = ES.ToLength(ES.Get(O, 'length'));

	var depthNum = 1;
	if (arguments.length > 0 && typeof arguments[0] !== 'undefined') {
		depthNum = ES.ToInteger(arguments[0]);
	}

	var A = ES.ArraySpeciesCreate(O, 0);
	FlattenIntoArray(A, O, sourceLen, 0, depthNum);
	return A;
};

They both call ES.ArraySpeciesCreate(O, 0); at some point but the new one is requering ES from /es2017.

@ljharb Do you think this could be the cause of this issue?

[ljharb]

No, those are all valid requires. Are you sure you aren’t mocking anything out in jest? Does npm ls exit successfully?

[GabrielFerrarini]

We are not mocking anything in this failing project.
npm ls has a couple errors

npm ERR! peer dep missing: react-dom@^15.4.2, required by react-addons-perf@15.4.2
npm ERR! peer dep missing: eslint@^5 || ^6, required by axobject-query@2.1.1
npm ERR! peer dep missing: eslint@>=5.0.0, required by eslint-plugin-eslint-plugin@2.1.0

The working project has a few errors like above as well.

[ljharb]

While all of those errors are very serious and should be fixed ASAP, I agree they're not relevant here.

https://unpkg.com/browse/array.prototype.flat@1.2.3/implementation.js is the latest version of array.prototype.flat, can you try updating all relevant deps and seeing what happens?

[GabrielFerrarini]

Ok, I think I understand what's happening.

With the current state of things, my package-lock includes:

    "array.prototype.find": {
      "version": "2.1.0",
      "resolved": "https://registry.npmjs.org/array.prototype.find/-/array.prototype.find-2.1.0.tgz",
      "integrity": "sha512-Wn41+K1yuO5p7wRZDl7890c3xvv5UBrfVXTVIe28rSQb6LS0fZMDrQB6PAcxQFRFy6vJTLDc3A2+3CjQdzVKRg==",
      "requires": {
        "define-properties": "^1.1.3",
        "es-abstract": "^1.13.0"
      }
    },

and

    "array-includes": {
      "version": "3.1.0",
      "resolved": "https://registry.npmjs.org/array-includes/-/array-includes-3.1.0.tgz",
      "integrity": "sha512-ONOEQoKrvXPKk7Su92Co0YMqYO32FfqJTzkKU9u2UpIXyYZIzLSvpdg4AwvSw4mSUW0czu6inK+zby6Oj6gDjQ==",
      "dev": true,
      "requires": {
        "define-properties": "^1.1.3",
        "es-abstract": "^1.17.0-next.0"
      },
      "dependencies": {
        "es-abstract": {
          "version": "1.17.0-next.1",
          "resolved": "https://registry.npmjs.org/es-abstract/-/es-abstract-1.17.0-next.1.tgz",

...

Pre-release verions have lower precedence (https://semver.org/#spec-item-9) and worse, are not resolved by this pattern "^1.0.0" (https://semver.npmjs.com/) will not match 1.0.0-rc.1 for example.

That said, my node_modules folder ended up with es-abstract@1.16.3 installed

es-abstract@1.17.0-next.1 will be installed as a subdependency of array.prototype.flat

However having it required like this require('es-abstract/2019/ArraySpeciesCreate'); will make node to look for the root folder. To prove my theory I just created the path 2019/ArraySpeciesCreate.js in node_modules/es-abstract and the error changed to

[GabrielFerrarini]

I'd say that releasing this without a pre-release tag should solve the issues for everybody becuase 1.17.0 would be the version resolved for all these subdependencies.

[ljharb]

That’s fair; i do plan to do that very soon.

[GabrielFerrarini]

Thinking about this, I believe that the underlying issue is to use absolute paths when requiring because it opens up a door for this kind of issues. It's up to you though how to proceed.

[ljharb]

What do you mean, absolute paths?

That it's listed as a dependency means that a deep require into it is guaranteed to work (assuming the package manager isn't broken). I'm still not sure why webpack is breaking on it, but it might be the hot module reloader?

[GabrielFerrarini]

I mean, instead of require('es/2019/etc') would be better to have require('./es/2019/etc'). It's not webpack or jest not being able to resolve. It's node going to the root node_modules because of the lack of './' in the require call. Since there are multiple versions of es-abstract being installed (because not every package is using next), the sub ones are in fact ignored (and they shouldn't)

[ljharb]

That wouldn't work - because the . refers to relative paths, and a bare specifier means node_modules.

[GabrielFerrarini]

That's exactly the issue. The bare specifier is going to node_modules of my project, but the installation of es-abstracts 1.17.0-next is in the node_modules folder of my project's dependency (node_modules/array.prototype.flat/node_modules/es-abstracts - this is 1.17). Node is resolving ALL of the requires to node_modules/es-abstracts - which is 1.16.

Anyway, up to how to proceed.

[ljharb]

if node_modules/array.prototype.flat/node_modules/es-abstract exists, a require from array.prototype.flat will check it's node_modules first; if it doesn't exist, it will (correctly) pull from the root level node_modules/es-abstract. Since array.prototype.flat requires v1.17, anything that dedupes it and treats v1.16 as satisfying the dependency is broken - so please do file an issue on npm, or yarn, or whichever package manager you're using if that's the case.

[GabrielFerrarini]

All. Check your configuration for module directories https://jestjs.io/docs/en/configuration#moduledirectories-arraystring for jest and https://webpack.js.org/configuration/resolve/ for webpack. Jest docs says An array of directory names to be searched recursively up and Setting this option will override the default, which happens to be node_modules.

@ljharb is right when he says that

if node_modules/array.prototype.flat/node_modules/es-abstract exists, a require from array.prototype.flat will check it's node_modules first

I fixed that in jest in a broken scenario and it works fine now.

[quanganhtran]

In our case we had to change webpack resolve.modules to include 'node_modules' instead of path.resolve(__dirname, 'node_modules') in order for it to find this module as a nested dependency. (1.17.0 is not deduped below)

@myteam/myapp@0.7.0
├─┬ @myteam/mycomponent@0.7.0
│ └─┬ react-vis@1.11.7
│   └─┬ deep-equal@1.1.1
│     └─┬ regexp.prototype.flags@1.3.0
│       └── es-abstract@1.17.0 
├─┬ enzyme@3.10.0
│ ├─┬ array.prototype.flat@1.2.1
│ │ └── es-abstract@1.13.0 
│ ├─┬ object.entries@1.1.0
│ │ └── es-abstract@1.13.0  deduped
│ ├─┬ object.values@1.1.0
│ │ └── es-abstract@1.13.0  deduped
│ └─┬ string.prototype.trim@1.2.0
│   └── es-abstract@1.13.0  deduped
├─┬ enzyme-adapter-react-16@1.14.0
│ └─┬ enzyme-adapter-utils@1.12.0
│   ├─┬ airbnb-prop-types@2.15.0
│   │ └─┬ array.prototype.find@2.1.0
│   │   └── es-abstract@1.13.0  deduped
│   └─┬ object.fromentries@2.0.0
│     └── es-abstract@1.13.0  deduped
└─┬ jest@24.8.0
  └─┬ jest-cli@24.8.0
    └─┬ realpath-native@1.1.0
      └─┬ util.promisify@1.0.0
        └─┬ object.getownpropertydescriptors@2.0.3
          └── es-abstract@1.13.0  deduped

[yogithesymbian]

ERROR Error: Cannot find module 'es-abstract/2020/RequireObjectCoercible'

[ljharb]

@yogithesymbian do you have a v1.18 prerelease installed? v1.17 doesn't have any ES2020 operations.

[ctappy]

ERROR Error: Cannot find module 'es-abstract/2020/RequireObjectCoercible'

i used yarn add es-abstract@1.18.0 to resolve this issue in the main repo

[ljharb]

I’d go up to v1.18.5 at this point :-)