util: fix inspection of module namespaces #20962

devsnek · 2018-05-25T13:37:34Z

unbreaks 6.7 upgrade

/cc @targos i don't have 6.7 checked out or the room to do so, can you confirm this patch fixes it?

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
commit message follows commit guidelines

BridgeAR

Why did this break in 6.7? And this will likely have a minor influence on the performance.

BridgeAR · 2018-05-25T14:02:03Z

lib/util.js

+
+    if (symbols.length !== 0) {
+      symbols = symbols.filter((key) =>
+        propertyIsEnumerable.call(value, key));


This should fit on a single line as before.

BridgeAR · 2018-05-25T14:02:25Z

lib/util.js

+    try {
+      keys = Object.keys(value);
+    } catch (err) {
+      if (err instanceof ReferenceError) {


Are there other possibilities for this to throw a ReferenceError?

not that I know of

BridgeAR · 2018-05-25T14:10:30Z

lib/util.js

+      keys = Object.keys(value);
+    } catch (err) {
+      if (err instanceof ReferenceError) {
+        keys = Object.getOwnPropertyNames(value);


Why does this not throw compared to Object.keys? Because of https://www.ecma-international.org/ecma-262/6.0/#sec-enumerableownnames 5.a.i.?

[[GetOwnProperty]] -> [[Get]] -> [[GetBindingValue]] which throws

BridgeAR · 2018-05-25T14:11:38Z

lib/util.js

-    keys = Object.keys(value);
-    if (symbols.length !== 0)
-      symbols = symbols.filter((key) => propertyIsEnumerable.call(value, key));
+    // this might throw if `value` is a Module Namespace Object from an


Please use upper cases for the first letter of a sentence.

BridgeAR · 2018-05-25T14:12:29Z

Benchmarks: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/201/

devsnek · 2018-05-25T14:22:04Z

@BridgeAR v8 fixed a bug where they weren't throwing a reference error and they should have been

mmarchini · 2018-05-25T14:32:31Z

Why did this break in 6.7?

For reference: https://bugs.chromium.org/p/v8/issues/detail?id=7780

mmarchini · 2018-05-25T14:39:35Z

I don't have 6.7 checked out or the room to do so, can you confirm this patch fixes it?

Yep, tests are passing on targos:v8-6.7 with these changes.

BridgeAR · 2018-05-25T14:48:16Z

I suggest to take a completely different approach: I saw that there is already a custom inspect function for modules. So when calling that function it should just wrap the util.inspect call in a try / catch and handle the error there because it should not be needed in the general util.inspect function.

Btw: normally only the object should be returned instead of calling util.inspect in a custom inspect function. That reduces the overhead. In this specific case it is useful in case that call is wrapped as suggested.

devsnek · 2018-05-25T14:49:38Z

@BridgeAR ~~keys are collected before we check the type or branch to custom inspect functions~~

edit: i misunderstood; module namespace objects can't have custom inspect functions

BridgeAR · 2018-05-25T15:22:19Z

Let's see how the benchmarks come out. I am not a fan of adding a lot of special handling for the namespace edge case.

BridgeAR · 2018-05-25T15:25:26Z

lib/util.js

+    try {
+      keys = Object.keys(value);
+    } catch (err) {
+      if (err instanceof ReferenceError) {


Instead of relying on the reference error here, we might just test for types.isModuleNamespaceObject nevertheless.

jdalton · 2018-05-25T17:44:59Z

\cc @ljharb

On v8/issues/detail?id=7780 are you planning to file a spec bug?

Update:

I opened issue tc39/ecma262#1209.

jdalton · 2018-05-25T17:48:39Z

lib/util.js

+          types.isModuleNamespaceObject(value)) {
+        keys = Object.getOwnPropertyNames(value);
+      } else {
+        throw err;


~~☝️ the Object.getOwnPropertyNames(value) fallback should filter non-enumerable to be equiv to keys~~

Update:

Not required since this is now restricted to namespace objects.

I suppose you can't have non-enumerable keys on a module namespace?

@jdalton until/unless the spec changes we have no way to check which ones are enumerable, and what @targos said

i believe the only keys on a module namespace will ever (atm) be enumerable strings for the exports, and Symbol.toStringTag.

mscdex · 2018-05-25T19:17:02Z

If this is going to land on older branches, we should be sure to run benchmarks for those branches first. If there are performance regressions, we should try to see about minimizing the impact (for example the try-catch will probably have a noticeable impact in node v6.x).

devsnek · 2018-05-25T20:21:28Z

@mscdex luckily i don't think module namespace objects exist on 6.x

lib/util.js

-    keys = Object.keys(value);
+    // This might throw if `value` is a Module Namespace Object from an
+    // unevaluated module, but we don't want to perform the actual type
+    // check because it's expensive.


jdalton · 2018-05-28T16:28:34Z

lib/util.js

+      if (err instanceof ReferenceError &&
+          types.isModuleNamespaceObject(value)) {
+        keys = Object.getOwnPropertyNames(value);
+      } else {


\cc @bmeurer A pattern in much of Node's code is to extract try-catches into their own functions. In this case it would be named tryKeys(). Is this practice still beneficial in the V8 of today?

iirc it hasn't been needed since like v6

That is not necessary anymore. It used to prevent a function from optimizing and this is not the case anymore with turbofan.

mmarchini · 2018-05-29T12:51:26Z

Is there any strong -1 here or can we land this?

devsnek · 2018-05-29T12:54:43Z

@mmarchini ci still appears to have issues so I'm not running it yet.

mmarchini · 2018-05-29T13:15:14Z

@devsnek regarding our infrastructure I think the issues are fixed. There are only flaky tests we need to fix now.

targos · 2018-05-29T13:18:20Z

CI: https://ci.nodejs.org/job/node-test-pull-request/15140/

jdalton · 2018-05-29T13:40:40Z

lib/util.js

+    // check because it's expensive.
+    // TODO(devsnek): track https://github.com/tc39/ecma262/issues/1209
+    // and modify this logic as needed.
+    try {


Object.keys and Object.getOwnPropertyNames references should be cherry-picked above.

The instanceof ReferenceError check will fail from errors of a different realm (context).

for the first point, we purposely leave those as-is because some polyfills for things use them to mess with util.

for the second point, i guess types.isNativeError(err) && err.name === 'ReferenceError' would work?

for the first point, we purposely leave those as-is because some polyfills for things use them to mess with util.

Ah cool! Do you have any examples at hand?

for the second point, i guess types.isNativeError(err) && err.name === 'ReferenceError' would work?

Yep.

I don't know what package does it, but it was brought up during discussions about hardening core against tampering with builtin prototypes.

BridgeAR · 2018-05-29T19:06:27Z

CI https://ci.nodejs.org/job/node-test-pull-request/15151/

jdalton · 2018-05-29T23:36:17Z

lib/util.js

@@ -772,7 +788,7 @@ function formatNamespaceObject(ctx, value, recurseTimes, keys) {
    try {
      output[i] = formatProperty(ctx, value, recurseTimes, keys[i], 0);
    } catch (err) {
-      if (!(err instanceof ReferenceError)) {
+      if (!(types.isNativeError(err) && err.name === 'ReferenceError')) {
        throw err;


Could the error checks be simplified all up? If namespace objects can only error by throwing a reference error then we can assume that the error accessing it is because of the reference error and remove the error checks and rethrow code above. We could also simplify the initial check by doing:

try { keys = Object.keys(value); } catch (err) { if (types.isModuleNamespaceObject(value)) { keys = Object.getOwnPropertyNames(value); } else { throw err; } }

Here erroring should be rare and when it does error it is likely a namespace object so the check up-front on error should be fine.

i think its best to explicitly check if its a reference error, do you feel strongly about this?

I dig it if code could be simpler, and still correct, that it be simpler is all.

MylesBorins · 2018-05-30T15:38:30Z

This is blocking V8 6.7 landing

@jdalton is it ok for this to land as is and be simplified in a follow up?

MylesBorins · 2018-05-30T15:43:53Z

One more CI: https://ci.nodejs.org/job/node-test-pull-request/15167/

jdalton · 2018-05-30T16:10:56Z

Yep, yep.

mmarchini · 2018-05-30T18:05:34Z

FreeBSD failure is a flaky and I have no idea why it's saying OSX failed.

Running another one: https://ci.nodejs.org/job/node-test-pull-request/15169/

mmarchini · 2018-05-30T19:12:56Z

OSX should work now (thanks @refack!).

5th time's a charm! https://ci.nodejs.org/job/node-test-pull-request/15170/

mmarchini · 2018-05-30T20:06:26Z

CI is yellow. Landing!

mmarchini · 2018-05-30T20:08:41Z

Landed in a25730b

PR-URL: #20962 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: Matheus Marchini <matheus@sthima.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>

devsnek requested a review from BridgeAR May 25, 2018 13:37

nodejs-github-bot added the util Issues and PRs related to the built-in util module. label May 25, 2018

guybedford approved these changes May 25, 2018

View reviewed changes

devsnek force-pushed the fix/module-namespace-inspection branch from b3450e9 to e16a700 Compare May 25, 2018 13:50

BridgeAR reviewed May 25, 2018

View reviewed changes

mmarchini approved these changes May 25, 2018

View reviewed changes

BridgeAR reviewed May 25, 2018

View reviewed changes

jdalton reviewed May 25, 2018

View reviewed changes

mscdex added dont-land-on-v4.x labels May 25, 2018

jdalton mentioned this pull request May 25, 2018

Object.keys throws for uninitialized module namespace object tc39/ecma262#1209

Open

targos approved these changes May 26, 2018

View reviewed changes

targos mentioned this pull request May 26, 2018

deps: update V8 to 6.7 #19989

Closed

2 tasks

BridgeAR approved these changes May 28, 2018

View reviewed changes

lib/util.js

keys = Object.keys(value);

// This might throw if `value` is a Module Namespace Object from an

// unevaluated module, but we don't want to perform the actual type

// check because it's expensive.

This comment was marked as resolved.

Sign in to view

devsnek force-pushed the fix/module-namespace-inspection branch from 7d66a91 to f056724 Compare May 28, 2018 15:13

jdalton reviewed May 28, 2018

View reviewed changes

jdalton reviewed May 29, 2018

View reviewed changes

util: fix inspection of module namespaces

a9a0643

devsnek force-pushed the fix/module-namespace-inspection branch from f056724 to a9a0643 Compare May 29, 2018 14:25

BridgeAR added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label May 29, 2018

jdalton reviewed May 29, 2018

View reviewed changes

mmarchini mentioned this pull request May 30, 2018

Flaky parallel/test-trace-events-fs-sync #21038

Closed

mmarchini closed this May 30, 2018

devsnek deleted the fix/module-namespace-inspection branch May 30, 2018 20:34

devsnek removed the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label May 30, 2018

MylesBorins mentioned this pull request Jun 6, 2018

v10.4.0 proposal #21167

Merged

util: fix inspection of module namespaces #20962

util: fix inspection of module namespaces #20962

Conversation

devsnek commented May 25, 2018 • edited Loading

Checklist

BridgeAR left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BridgeAR commented May 25, 2018

devsnek commented May 25, 2018

mmarchini commented May 25, 2018

mmarchini commented May 25, 2018

BridgeAR commented May 25, 2018

devsnek commented May 25, 2018 • edited Loading

BridgeAR commented May 25, 2018

Choose a reason for hiding this comment

jdalton commented May 25, 2018 • edited Loading

jdalton May 25, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

devsnek May 25, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mscdex commented May 25, 2018 • edited Loading

devsnek commented May 25, 2018

This comment was marked as resolved.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mmarchini commented May 29, 2018

devsnek commented May 29, 2018

mmarchini commented May 29, 2018

targos commented May 29, 2018

jdalton May 29, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jdalton May 29, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BridgeAR commented May 29, 2018

jdalton May 29, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MylesBorins commented May 30, 2018

MylesBorins commented May 30, 2018

jdalton commented May 30, 2018

mmarchini commented May 30, 2018

mmarchini commented May 30, 2018

mmarchini commented May 30, 2018

mmarchini commented May 30, 2018

devsnek commented May 25, 2018 •

edited

Loading

devsnek commented May 25, 2018 •

edited

Loading

jdalton commented May 25, 2018 •

edited

Loading

jdalton May 25, 2018 •

edited

Loading

devsnek May 25, 2018 •

edited

Loading

mscdex commented May 25, 2018 •

edited

Loading

jdalton May 29, 2018 •

edited

Loading

jdalton May 29, 2018 •

edited

Loading

jdalton May 29, 2018 •

edited

Loading