Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement module execution #2922

Merged
merged 18 commits into from May 18, 2023
Merged

Implement module execution #2922

merged 18 commits into from May 18, 2023

Conversation

jedel1043
Copy link
Member

@jedel1043 jedel1043 commented May 10, 2023

Depends on #2921. Will rebase onto main when that PR is merged.

This Pull Request closes #64. We finally have a module system! 🎉

It changes the following:

  • Implements the module execution semantics from the spec.
  • Adds a Module struct API from which an user can progress the loading, linking and execution of a module.
  • Removes most of Context's module APIs in favour of methods in Module.
  • Adds a ModuleLoader trait from where a host can define how to load and cache modules.
  • Adds a ModuleEnvironment that can contain indirect references to other environment bindings.

There's still more work to do though, like adding support for the import() method, fix some parsing and linking bugs, and implementing JSON modules (Stage 3).

Marking it as draft since there's a lot of documentation missing (and apparently some bugs), but I wanted to put out the PR for reviews on the implementation.

@jedel1043 jedel1043 added enhancement New feature or request execution Issues or PRs related to code execution API labels May 10, 2023
@jedel1043 jedel1043 requested a review from a team May 10, 2023 00:00
@jedel1043 jedel1043 changed the base branch from main to declarative-kinds May 10, 2023 00:00
@jedel1043 jedel1043 added this to the v0.17.0 milestone May 10, 2023
@github-actions
Copy link

github-actions bot commented May 10, 2023

Test262 conformance changes

Test result main count PR count difference
Total 94,657 93,982 -675
Passed 73,931 73,694 -237
Ignored 17,505 17,489 -16
Failed 3,221 2,799 -422
Panics 0 0 0
Conformance 78.10% 78.41% +0.31%
Fixed tests (154):
test/language/module-code/instn-named-bndng-dflt-named.js (previously Failed)
test/language/module-code/instn-star-binding.js (previously Failed)
test/language/module-code/instn-named-bndng-const.js (previously Failed)
test/language/module-code/instn-named-bndng-gen.js (previously Failed)
test/language/module-code/eval-export-dflt-gen-named-semi.js (previously Failed)
test/language/module-code/instn-local-bndng-export-var.js (previously Failed)
test/language/module-code/instn-same-global.js (previously Failed)
test/language/module-code/export-expname-import-string-binding.js (previously Failed)
test/language/module-code/eval-export-dflt-expr-cls-named.js (previously Failed)
test/language/module-code/instn-named-bndng-fun.js (previously Failed)
test/language/module-code/eval-export-dflt-fun-anon-semi.js (previously Failed)
test/language/module-code/eval-export-dflt-gen-anon-semi.js (previously Failed)
test/language/module-code/parse-export-empty.js (previously Failed)
test/language/module-code/instn-named-bndng-dflt-star.js (previously Failed)
test/language/module-code/eval-export-fun-semi.js (previously Failed)
test/language/module-code/eval-export-dflt-expr-in.js (previously Failed)
test/language/module-code/export-default-asyncfunction-declaration-binding.js (previously Failed)
test/language/module-code/eval-export-dflt-expr-err-eval.js (previously Failed)
test/language/module-code/instn-named-bndng-var.js (previously Failed)
test/language/module-code/instn-local-bndng-export-fun.js (previously Failed)
test/language/module-code/eval-gtbndng-indirect-update.js (previously Failed)
test/language/module-code/instn-named-bndng-let.js (previously Failed)
test/language/module-code/instn-named-bndng-dflt-fun-anon.js (previously Failed)
test/language/module-code/eval-export-dflt-cls-named-semi.js (previously Failed)
test/language/module-code/eval-export-dflt-expr-cls-name-meth.js (previously Failed)
test/language/module-code/instn-named-bndng-dflt-gen-named.js (previously Failed)
test/language/module-code/instn-star-props-circular.js (previously Failed)
test/language/module-code/eval-gtbndng-indirect-update-dflt.js (previously Failed)
test/language/module-code/eval-export-dflt-cls-name-meth.js (previously Failed)
test/language/module-code/instn-named-bndng-dflt-cls.js (previously Failed)
test/language/module-code/eval-export-dflt-cls-anon.js (previously Failed)
test/language/module-code/export-expname-from-binding-string.js (previously Failed)
test/language/module-code/instn-named-bndng-dflt-gen-anon.js (previously Failed)
test/language/module-code/eval-export-dflt-expr-err-get-value.js (previously Failed)
test/language/module-code/export-default-generator-declaration-binding.js (previously Failed)
test/language/module-code/instn-once.js (previously Failed)
test/language/module-code/eval-export-dflt-expr-fn-named.js (previously Failed)
test/language/module-code/eval-export-dflt-cls-named.js (previously Failed)
test/language/module-code/eval-export-dflt-cls-anon-semi.js (previously Failed)
test/language/module-code/instn-uniq-env-rec.js (previously Failed)
test/language/module-code/instn-star-equality.js (previously Failed)
test/language/module-code/eval-gtbndng-indirect-faux-assertion.js (previously Failed)
test/language/module-code/instn-named-bndng-dflt-expr.js (previously Failed)
test/language/module-code/instn-named-bndng-trlng-comma.js (previously Failed)
test/language/module-code/instn-local-bndng-export-gen.js (previously Failed)
test/language/module-code/eval-self-once.js (previously Failed)
test/language/module-code/eval-this.js (previously Failed)
test/language/module-code/eval-export-dflt-expr-gen-named.js (previously Failed)
test/language/module-code/export-expname-binding-string.js (previously Failed)
test/language/module-code/eval-export-dflt-fun-named-semi.js (previously Failed)
test/language/module-code/instn-named-bndng-dflt-fun-named.js (previously Failed)
test/language/module-code/instn-local-bndng-export-let.js (previously Failed)
test/language/module-code/export-expname-from-star.js (previously Failed)
test/language/module-code/instn-local-bndng-export-const.js (previously Failed)
test/language/module-code/eval-gtbndng-indirect-trlng-comma.js (previously Failed)
test/language/module-code/export-default-asyncgenerator-declaration-binding.js (previously Failed)
test/language/module-code/instn-named-bndng-cls.js (previously Failed)
test/language/module-code/export-expname-from-star-string.js (previously Failed)
test/language/module-code/eval-export-cls-semi.js (previously Failed)
test/language/module-code/eval-gtbndng-indirect-update-as.js (previously Failed)
test/language/module-code/export-expname-from-string.js (previously Failed)
test/language/module-code/eval-export-gen-semi.js (previously Failed)
test/language/module-code/export-default-function-declaration-binding.js (previously Failed)
test/language/module-code/instn-star-ambiguous.js (previously Failed)
test/language/module-code/instn-local-bndng-export-cls.js (previously Failed)
test/language/module-code/top-level-await/module-async-import-async-resolution-ticks.js (previously Failed)
test/language/module-code/top-level-await/module-self-import-async-resolution-ticks.js (previously Failed)
test/language/module-code/top-level-await/module-import-unwrapped.js (previously Failed)
test/language/module-code/top-level-await/module-sync-import-async-resolution-ticks.js (previously Failed)
test/language/module-code/top-level-await/dfs-invariant.js (previously Failed)
test/language/module-code/top-level-await/module-import-resolution.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dft-class-decl-await-expr-new-expr.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-lex-decl-await-expr-literal-string.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dft-class-decl-await-expr-literal-string.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dflt-assign-expr-await-expr-null.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-lex-decl-await-expr-null.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-var-await-expr-obj-literal.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-lex-decl-await-expr-array-literal.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-lex-decl-await-expr-regexp.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dft-class-decl-await-expr-func-expression.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dft-class-decl-await-expr-this.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-lex-decl-await-expr-literal-number.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-lex-decl-await-expr-this.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-lex-decl-await-expr-identifier.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-class-decl-await-expr-obj-literal.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-var-await-expr-this.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-var-await-expr-literal-number.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dflt-assign-expr-await-expr-new-expr.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-var-await-expr-identifier.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-lex-decl-await-expr-nested.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-class-decl-await-expr-func-expression.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-class-decl-await-expr-literal-string.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-var-await-expr-regexp.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-class-decl-await-expr-new-expr.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dflt-assign-expr-await-expr-identifier.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dft-class-decl-await-expr-template-literal.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dft-class-decl-await-expr-identifier.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-class-decl-await-expr-array-literal.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dft-class-decl-await-expr-array-literal.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-class-decl-await-expr-this.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-var-await-expr-nested.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dflt-assign-expr-await-expr-array-literal.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-var-await-expr-new-expr.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-lex-decl-await-expr-new-expr.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-class-decl-await-expr-regexp.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dflt-assign-expr-await-expr-regexp.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dft-class-decl-await-expr-nested.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-var-await-expr-literal-string.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-class-decl-await-expr-nested.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dflt-assign-expr-await-expr-literal-number.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-class-decl-await-expr-null.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-var-await-expr-func-expression.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dflt-assign-expr-await-expr-obj-literal.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dft-class-decl-await-expr-null.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-var-await-expr-null.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-lex-decl-await-expr-func-expression.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-class-decl-await-expr-identifier.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-class-decl-await-expr-literal-number.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-var-await-expr-template-literal.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-lex-decl-await-expr-obj-literal.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-class-decl-await-expr-template-literal.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dft-class-decl-await-expr-regexp.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dft-class-decl-await-expr-literal-number.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dflt-assign-expr-await-expr-nested.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dflt-assign-expr-await-expr-template-literal.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dft-class-decl-await-expr-obj-literal.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-var-await-expr-array-literal.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dflt-assign-expr-await-expr-literal-string.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dflt-assign-expr-await-expr-this.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-lex-decl-await-expr-template-literal.js (previously Failed)
test/language/module-code/top-level-await/syntax/export-dflt-assign-expr-await-expr-func-expression.js (previously Failed)
test/language/module-code/namespace/Symbol.toStringTag.js (previously Failed)
test/language/module-code/namespace/Symbol.iterator.js (previously Failed)
test/language/module-code/namespace/internals/object-propertyIsEnumerable-binding-uninit.js (previously Failed)
test/language/module-code/namespace/internals/own-property-keys-sort.js (previously Failed)
test/language/module-code/namespace/internals/get-prototype-of.js (previously Failed)
test/language/module-code/namespace/internals/set-prototype-of.js (previously Failed)
test/language/module-code/namespace/internals/get-str-not-found.js (previously Failed)
test/language/module-code/namespace/internals/get-own-property-sym.js (previously Failed)
test/language/module-code/namespace/internals/get-sym-found.js (previously Failed)
test/language/module-code/namespace/internals/object-keys-binding-uninit.js (previously Failed)
test/language/module-code/namespace/internals/enumerate-binding-uninit.js (previously Failed)
test/language/module-code/namespace/internals/object-hasOwnProperty-binding-uninit.js (previously Failed)
test/language/module-code/namespace/internals/prevent-extensions.js (previously Failed)
test/language/module-code/namespace/internals/has-property-sym-found.js (previously Failed)
test/language/module-code/namespace/internals/has-property-sym-not-found.js (previously Failed)
test/language/module-code/namespace/internals/set-prototype-of-null.js (previously Failed)
test/language/module-code/namespace/internals/has-property-str-not-found.js (previously Failed)
test/language/module-code/namespace/internals/delete-non-exported.js (previously Failed)
test/language/module-code/namespace/internals/get-own-property-str-not-found.js (previously Failed)
test/language/module-code/namespace/internals/is-extensible.js (previously Failed)
test/language/module-code/namespace/internals/get-sym-not-found.js (previously Failed)
test/language/expressions/class/elements/class-name-static-initializer-default-export.js (previously Failed)
test/built-ins/Proxy/preventExtensions/trap-is-undefined-target-is-proxy.js (previously Failed)
Broken tests (8):
test/language/module-code/export-expname-string-binding.js (previously Passed)
test/language/module-code/parse-err-semi-name-space-export.js (previously Passed)
test/language/module-code/early-import-eval.js (previously Passed)
test/language/module-code/parse-err-semi-export-star.js (previously Passed)
test/language/module-code/parse-err-semi-named-export.js (previously Passed)
test/language/module-code/early-import-arguments.js (previously Passed)
test/language/module-code/parse-err-semi-dflt-expr.js (previously Passed)
test/language/module-code/parse-err-semi-named-export-from.js (previously Passed)

@jedel1043
Copy link
Member Author

By the way, the test count reduction is because we were running module tests twice: on no strict and strict mode. However, this is unnecessary for modules because they're strict by default, so we can just run them once.

@jedel1043
Copy link
Member Author

Ok, I checked the regressed tests and all of them have the $DONOTEVALUATE() function, meaning those were false positives before!

Base automatically changed from declarative-kinds to main May 11, 2023 03:56
@codecov
Copy link

codecov bot commented May 11, 2023

Codecov Report

Merging #2922 (b2ec6ad) into main (ab87b2f) will decrease coverage by 1.70%.
The diff coverage is 1.52%.

@@            Coverage Diff             @@
##             main    #2922      +/-   ##
==========================================
- Coverage   52.15%   50.45%   -1.70%     
==========================================
  Files         436      442       +6     
  Lines       43777    45267    +1490     
==========================================
+ Hits        22831    22840       +9     
- Misses      20946    22427    +1481     
Impacted Files Coverage Δ
boa_ast/src/declaration/export.rs 0.00% <0.00%> (ø)
boa_ast/src/declaration/import.rs 0.00% <0.00%> (ø)
boa_ast/src/module_item_list/mod.rs 0.00% <0.00%> (ø)
boa_cli/src/main.rs 0.66% <0.00%> (-0.11%) ⬇️
boa_engine/src/builtins/promise/mod.rs 29.92% <ø> (ø)
boa_engine/src/bytecompiler/module.rs 0.00% <0.00%> (ø)
...engine/src/environments/runtime/declarative/mod.rs 85.05% <0.00%> (-8.62%) ⬇️
...ine/src/environments/runtime/declarative/module.rs 0.00% <0.00%> (ø)
boa_engine/src/lib.rs 78.20% <ø> (ø)
boa_engine/src/module/source.rs 0.00% <0.00%> (ø)
... and 22 more

@jedel1043 jedel1043 marked this pull request as ready for review May 12, 2023 01:09
@jedel1043
Copy link
Member Author

All documented and added an example, this is ready for full reviews now!

Copy link
Member

@raskad raskad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still reviewing the execution parts. The ast and parser changes look very good, I just found a few copy-pasted docs.

boa_ast/src/module_item_list/mod.rs Outdated Show resolved Hide resolved
boa_ast/src/module_item_list/mod.rs Outdated Show resolved Hide resolved
Copy link
Member

@HalidOdat HalidOdat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks really good!

boa_engine/src/module/mod.rs Show resolved Hide resolved
boa_engine/src/module/mod.rs Outdated Show resolved Hide resolved
boa_engine/src/module/mod.rs Show resolved Hide resolved
boa_engine/src/module/mod.rs Show resolved Hide resolved
@jedel1043 jedel1043 requested review from HalidOdat, raskad and a team May 14, 2023 07:31
boa_interner/src/sym.rs Show resolved Hide resolved
boa_engine/src/object/internal_methods/namespace.rs Outdated Show resolved Hide resolved
boa_engine/src/object/internal_methods/namespace.rs Outdated Show resolved Hide resolved
boa_engine/src/object/internal_methods/immutable.rs Outdated Show resolved Hide resolved
boa_engine/src/object/internal_methods/immutable.rs Outdated Show resolved Hide resolved
boa_engine/src/object/internal_methods/immutable.rs Outdated Show resolved Hide resolved
@jedel1043 jedel1043 requested review from HalidOdat and a team May 17, 2023 04:53
Copy link
Member

@HalidOdat HalidOdat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some small nitpicks, the rest of the code looks great!

boa_engine/src/module/source.rs Outdated Show resolved Hide resolved
boa_engine/src/module/source.rs Outdated Show resolved Hide resolved
@jedel1043 jedel1043 requested a review from HalidOdat May 17, 2023 05:25
@jedel1043 jedel1043 requested a review from nekevss May 17, 2023 17:45
@jedel1043 jedel1043 requested a review from raskad May 17, 2023 19:56
Copy link
Member

@raskad raskad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just very awesome work on this @jedel1043. I checked out #2932 locally and it seems like with these huge steps we are on a very good path for getting most of the language features done.

I also like the optimizations built into the module environment bindings and the safe design on the Status transitions.

I'm all for merging this.

Copy link
Member

@HalidOdat HalidOdat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks perfect to me! Great work!

@jedel1043 jedel1043 enabled auto-merge May 18, 2023 00:40
@jedel1043 jedel1043 added this pull request to the merge queue May 18, 2023
Merged via the queue into main with commit 06161ed May 18, 2023
9 checks passed
@jedel1043 jedel1043 deleted the module-exec branch May 18, 2023 01:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API enhancement New feature or request execution Issues or PRs related to code execution
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement the ECMAScript module system
4 participants