WIP: initial attempt to run 'esm' against tc39/test262 #550

dnalborczyk · 2018-08-06T00:46:26Z

work in progress

notes and questions coming soon

just want to see some runs first 😋

update:

alrighty, cool. looking pretty good so far!

btw, the folder structure etc. where test262 is being cloned into and modified to etc. should probably change. also the way the tests are run (right now as part of the whole test suite). we can separate it out and make it a separate run, or a combination thereof...

couple things I'm noticing which are still to do:

I'll add more later ...

jdalton · 2018-08-06T05:57:49Z

test/tests.js

@@ -18,6 +18,7 @@ import "./main-hook-tests.mjs"
 import "./require-hook-tests.js"
 import "./repl-hook-tests.mjs"
 import "./scenario-tests.mjs"
+import "./_external/test262/index.js"



We call the external folder in src the vendor folder. We could call it that in test too.

jdalton · 2018-08-06T05:57:58Z

test/_external/test262/blacklist

@@ -0,0 +1,101 @@
+# export * from ns


Could you rename this list to skiplist

jdalton · 2018-08-07T01:26:26Z

test/_external/test262/skiplist

@@ -135,3 +135,6 @@ test/language/module-code/namespace/internals/own-property-keys-sort.js

 # NEEDS INVESTIGATION!
 test/language/module-code/namespace/internals/set.js
+
+# NEEDS INVESTIGATION! -> only fails on node v6.14.3
+test/language/module-code/namespace/internals/object-keys-binding-uninit.js


This is just a V8 tweak. In older Node they didn't error. In newer ones they do. We support erroring whenever Node does so this can be gated to whatever Node it's supported in. Related to https://bugs.chromium.org/p/v8/issues/detail?id=7780.

ah, ok. it seems to be on the v6 branch only I believe. I'll implement a condition for allowed versions or something like that (e.g. node6, harmony, etc.)

jdalton · 2018-08-07T01:28:26Z

.travis.yml

@@ -7,7 +7,7 @@ os:
 node_js:
  - 10
  - 8
-  - 6.2.0
+  - 6


we'll want to keep that low 6.2.0 is the lowest I can run the full suite at without random seg faults (due to issues in earlier 6.0 releases). It's fine as Node official support is only whatever the latest 6 is but for nice coverage I like to test as far back as possible.

yeah, that was just temporary. did you see my question regarding that toppic --> #550 (comment)

I'll extend the skiplist to be conditional. not: all or nothing

jdalton · 2018-08-07T01:30:24Z

test/_external/test262/skiplist

+test/language/module-code/namespace/internals/get-str-update.js
+
+# NEEDS INVESTIGATION!
+test/language/module-code/namespace/internals/own-property-keys-sort.js


We assign namespace keys in sort order so I'd be interested in that fail. 👆

jdalton · 2018-08-07T01:32:34Z

test/_external/test262/skiplist

+test/language/module-code/namespace/internals/own-property-keys-sort.js
+
+# NEEDS INVESTIGATION!
+test/language/module-code/namespace/internals/set.js


We throw here too but be careful as by default esm enables CJS interop with mutable namespace objects (which don't throw). This is configurable though cjs.mutableNamespace.

jdalton · 2018-08-07T01:33:22Z

test/_external/test262/skiplist

@@ -99,3 +105,33 @@ test/language/module-code/instn-star-props-nrml.js

 # NEEDS INVESTIGATION!
 test/language/module-code/parse-err-reference.js
+


Many of these fails look to be related to having cjs.mutableNamespace enabled 👇

ah. ok. that's good tho! what's the plan forward to make them pass? is that with mjs disabled as well? I would assume mjs and use module work the same (or similar)?

to avoid .mjs you could just include .esmrc or .esmrc.json in the test files folder or parent folder with specific options disabled.

jdalton · 2018-08-31T20:42:12Z

Big Big thanks @dnalborczyk!

I'm going to merge this in and then perform some cleanup on it.

dnalborczyk · 2018-09-01T01:27:02Z

you're welcome! thank you John! @jdalton

yeah, it probably needs some cleanup.

sorry for being a little inactive lately, just started a new job, and I'm knee deep occupied with onboarding and diving into their codebase. the company is using AWS Lambda, with serverless framework, and needless to say, I tried to bring in esm 😃 and it works great, although only in bridge mode (I believe AWS Lambda currently doesn't support the --require flag as far as I can tell). I noticed something regarding the cache behavior, but I think I better open a new issue for that. If I don't fall asleep I'll also try to open the other issues from some time ago.

you might notice that I'm creating a js folder as well as an mjs folder. I think I had to do some additional "manual" steps (commenting/uncomenting) in order to have it running locally, as a run against node --experimental modules, as well as against esm strict mode to compare the numbers and behavior.

if you want you can ignore that for now, and I'll add it later on again. I wanted to ask you some time ago how you wanted the tests to run ideally once merged in.

Let me know if you have any questions.

edit: everything in the skiplist is being ignored (skipped), except if it contains an additional @-flag, then it'll be only skipped if the condition applies, but runs otherwise.

dnalborczyk force-pushed the test262 branch from a1ac262 to 04b9b8a Compare August 6, 2018 00:55

jdalton added the enhancement label Aug 6, 2018

jdalton mentioned this pull request Aug 6, 2018

tc39/test262 #543

Closed

jdalton reviewed Aug 6, 2018

View reviewed changes

dnalborczyk force-pushed the test262 branch 6 times, most recently from 1d0f0cf to d904ebb Compare August 6, 2018 18:37

jdalton force-pushed the master branch from 57f1695 to 9adbd37 Compare August 6, 2018 18:54