Layering: track execution contexts' script/module #242

domenic · 2015-12-10T22:14:16Z

Fixes #78. Introduces the concept of a Script Record as a counterpart to a Module Record, and adds [[HostDefined]] fields to both of them. Every execution environment now has ScriptOrModule component pointing back to its "creator" script or module:

Top level script/module execution contexts point to the appropriate script/module.
Function execution contexts for functions declared at top level point to that of the containing script/module.
Function execution contexts for functions declared inside other functions point to that of the containing function.
Eval execution contexts point to that of their containing function.
Job execution contexts (which are largely a spec artifact) point to the script/module that originally enqueued the job.

In the course of doing this, factored out ParseScript and ScriptEvaluation abstract operations, which now better parallel modules.

I think this is pretty good but review would be appreciated, especially where I converted ScriptEvaluation from Runtime Semantics into an abstract operation. The grammar interactions aren't 100% clear to me there.

anba · 2015-12-11T15:11:20Z

spec.html

+        1. Assert: _sourceText_ is an ECMAScript source text (see clause <emu-xref href="#sec-ecmascript-language-source-code"></emu-xref>).
+        1. Parse _sourceText_ using |Script| as the goal symbol and analyze the parse result for any Early Error conditions. If the parse was successful and no early errors were found, let _body_ be the resulting parse tree. Otherwise, let _body_ be an indication of one or more parsing errors and/or early errors. Parsing and early error detection may be interweaved in an implementation dependent manner. If more than one parse or early error is present, the number and ordering of reported errors is implementation dependent but at least one error must be reported.
+        1. If _body_ is an abrupt completion or error indication, then
+          1. Throw a SyntaxError exception.


The sentence "If more than one parse or early error is present, the number and ordering of reported errors is implementation dependent but at least one error must be reported." allows to report multiple errors at once, but the next step "Throw a SyntaxError exception." limits this to a single error.

Also "Throw a SyntaxError exception." is not correct for early ReferenceErrors. (Me shakes fist at early reference errors.) The current 15.2.1.16.1 ParseModule abstract operation needs to be fixed, too.

Interesting. Note that I am essentially copying ParseModule here, so ParseModule will also have both these problems.

I don't believe your first paragraph is correct though. You can report multiple errors independently of throwing a SyntaxError. That step is just requiring that at least one SyntaxError be thrown.

Do you have suggested replacement text for both here and ParseModule? Maybe it would suffice to just replace "Throw a SyntaxError exception" with "Throw a SyntaxError or ReferenceError exception, according to the error indicated"?

If there are early syntax and reference errors, a single error kind is no longer sufficient.

anba · 2015-12-11T15:19:06Z

Functions created in 19.2.1.1.1 CreateDynamicFunction or indirect evals won't be associated with a "Script or Module". Does this pose a problem?

And ECMAScript needs a better term for "Script or Module", because repeating "Script or Module" every time a script or module is referenced is kind of awkward. (That's not a criticism on this PR, but rather a general issue.)

domenic · 2015-12-11T16:36:48Z

Functions created in 19.2.1.1.1 CreateDynamicFunction or indirect evals won't be associated with a "Script or Module". Does this pose a problem?

Yes, I think it could; I will fix that. Thanks for finding it.

And ECMAScript needs a better term for "Script or Module"

I agree. I couldn't come up with anything on the spot. (I thought "code unit", but Unicode already took that one...)

anba · 2015-12-11T17:31:59Z

I agree. I couldn't come up with anything on the spot. (I thought "code unit", but Unicode already took that one...)

FWIW I'm currently using "Program" (based on ES5's Program parser production). But "ProgramUnit" or "ApplicationUnit" are probably better names, especially because "program" is already used in the spec to describe the running ECMAScript program itself.

domenic · 2015-12-11T23:19:10Z

New version uploaded, should be better now! Thanks @anba.

domenic · 2015-12-12T00:16:39Z

Functions created in 19.2.1.1.1 CreateDynamicFunction or indirect evals won't be associated with a "Script or Module".

I took another look at this and I can't see what you're talking about actually. CreateDynamicFunction calls FunctionInitialize, which should have no problem setting up the association.

anba · 2015-12-12T10:21:59Z

In FunctionInitialize:

If the active function object is null, set the [[ScriptOrModule]] internal slot of F to the running execution context's ScriptOrModule.
Otherwise, set the [[ScriptOrModule]] internal slot of F to the value of the active function object's [[ScriptOrModule]] internal slot.

The active function object during CreateDynamicFunction is the intrinsic %Function%, and %Function%.[[ScriptOrModule]] is undefined ([[ScriptOrModule]] is initialized with undefined per 6.1.7.2).

The same issue is present for indirect eval calls:

Set the evalCxt's ScriptOrModule to ctx's ScriptOrModule.

ctx's ScriptOrModule component is undefined.

domenic · 2015-12-14T16:59:54Z

Actually, all built-in functions must have a [[ScriptOrModule]] value. We just don't say what it is.

But, this is probably sub-optimal. I will try to work out something so that built-in functions have undefined [[ScriptOrModule]], and are skipped over when calculating an execution context's ScriptOrModule.

anba · 2015-12-14T17:06:31Z

Actually, all built-in functions must have a [[ScriptOrModule]] value. We just don't say what it is.

But, this is probably sub-optimal. I will try to work out something so that built-in functions have undefined [[ScriptOrModule]]

As I said above, [[ScriptOrModule]] is initialized with undefined per 6.1.7.2: "Unless specified otherwise, the initial value of an internal slot is the value undefined."

domenic · 2015-12-14T17:07:31Z

Yes, and as I said above, [[ScriptOrModule]] is not initialized with undefined per that clause; it is initialized with whatever the host environment says:

If a built-in function object is implemented as an exotic object it must have the ordinary object behaviour specified in 9.1. All such exotic function objects also have [[Prototype]], [[Extensible]], [[Realm]], and [[ScriptOrModule]] internal slots.

anba · 2015-12-14T17:23:48Z

6.1.7.2 applies to all objects, it's not limited to a specific kind of object. So if the specification adds [[ScriptOrModule]] to built-in function objects, but does not assign a specific value to [[ScriptOrModule]], the default value for [[ScriptOrModule]] is undefined. Obviously it's also possible to explicitly set [[ScriptOrModule]] to undefined when creating a built-in function object during 9.3.3 CreateBuiltinFunction.

domenic · 2015-12-14T17:31:03Z

I see. In any case, fixed; built-in functions are now explicitly skipped in this calculation. Let me know if I got it right?

@bterlson we should decide on whether to adopt "program" as a name for "script or module" before merging, especially now that I've introduced an abstract op (and thus a link ID) with that name.

This brings the script execution parts of the spec up to date with the latest changes in ES. Notable changes include: - Removed of some generalization to allow non-JavaScript scripting languages (including XMl-based ones). - Let ES track the execution context stack, and the corresponding stack of scripts; see tc39/ecma262#242. This allows us to stop tracking the stack of script settings objects, instead defining entry and incumbent settings objects with reference to ES. This is especially important since our current mechanism, of monkey-patching the SourceElement grammatical construction, no longer works since that grammar production has disappeared. Fixes #155. - Established a direct correspondence between HTML's "script" concept and ES's Script Record/Module Record concepts. The former is stored in the [[HostDefined]] slot of the latter. - The process of parsing and executing scripts, including handling any resulting errors, is now formalized and appropriately calls out to ES's ParseScript and ScriptEvaluation abstract operations. Note that we do *not* use the ECMAScript ScriptEvaluationJob or job queue for our script parsing/evaluation. That setup is overengineered and does not serve the needs of HTML; we hope to remove it from ES eventually. Instead we simply use ParseScript and EvaluateScript directly. This almost completes https://www.w3.org/Bugs/Public/show_bug.cgi?id=25981, with the remaining work item being to integrate promise jobs and ensure they are run as microtasks.

This brings the script execution parts of the spec up to date with the latest changes in ES. Notable changes include: - Removed of some generalization to allow non-JavaScript scripting languages (including XML-based ones). - Let ES track the execution context stack, and the corresponding stack of scripts; see tc39/ecma262#242. This allows us to stop tracking the stack of script settings objects, instead defining entry and incumbent settings objects with reference to ES. This is especially important since our current mechanism, of monkey-patching the SourceElement grammatical construction, no longer works since that grammar production has disappeared. Fixes #155. - Established a direct correspondence between HTML's "script" concept and ES's Script Record/Module Record concepts. The former is stored in the [[HostDefined]] slot of the latter. - The process of parsing and executing scripts, including handling any resulting errors, is now formalized and appropriately calls out to ES's ParseScript and ScriptEvaluation abstract operations. Note that we do *not* use the ECMAScript ScriptEvaluationJob or job queue for our script parsing/evaluation. That setup is overengineered and does not serve the needs of HTML; we hope to remove it from ES eventually. (See tc39/ecma262#240 (comment) for details.) Instead we simply use ParseScript and EvaluateScript directly. This almost completes https://www.w3.org/Bugs/Public/show_bug.cgi?id=25981, with the remaining work item being to integrate promise jobs and ensure they are run as microtasks.

bterlson · 2015-12-15T00:56:01Z

I find program to be more confusing (I consider a program to be composed of multiple libraries and such) so have a slight preference for scriptOrModule but am fine either way really.

annevk · 2015-12-15T09:44:39Z

(Suggestion on the naming based on <script> and <script type=module>. Rename Script to DefaultScript and Module to ModuleScript. Then use Script to mean either.)

This brings the script execution parts of the spec up to date with the latest changes in ES. Notable changes include: - Removed of some generalization to allow non-JavaScript scripting languages (including XML-based ones). - Let ES track the execution context stack, and the corresponding stack of scripts; see tc39/ecma262#242. This allows us to stop tracking the stack of script settings objects, instead defining entry and incumbent settings objects with reference to ES. This is especially important since our current mechanism, of monkey-patching the SourceElement grammatical construction, no longer works since that grammar production has disappeared. Fixes #155. - Established a direct correspondence between HTML's "script" concept and ES's Script Record/Module Record concepts. The former is stored in the [[HostDefined]] slot of the latter. - The process of parsing and executing scripts, including handling any resulting errors, is now formalized and appropriately calls out to ES's ParseScript and ScriptEvaluation abstract operations. Note that we do *not* use the ECMAScript ScriptEvaluationJob or job queue for our script parsing/evaluation. That setup is overengineered and does not serve the needs of HTML; we hope to remove it from ES eventually. (See tc39/ecma262#240 (comment) for details.) Instead we simply use ParseScript and EvaluateScript directly. This almost completes https://www.w3.org/Bugs/Public/show_bug.cgi?id=25981, with the remaining work item being to integrate promise jobs and ensure they are run as microtasks.

anba · 2015-12-15T16:33:26Z

The ScriptEvaluation change for empty scripts also fixes https://bugs.ecmascript.org/show_bug.cgi?id=4345 .

bterlson · 2015-12-15T18:30:35Z

spec.html

@@ -5935,7 +5935,8 @@
  <!-- es6num="8.3" -->
  <emu-clause id="sec-execution-contexts">
    <h1>Execution Contexts</h1>
-    <p>An <dfn>execution context</dfn> is a specification device that is used to track the runtime evaluation of code by an ECMAScript implementation. At any point in time, there is at most one execution context that is actually executing code. This is known as the <em>running</em> execution context. A stack is used to track execution contexts. The running execution context is always the top element of this stack. A new execution context is created whenever control is transferred from the executable code associated with the currently running execution context to executable code that is not associated with that execution context. The newly created execution context is pushed onto the stack and becomes the running execution context.</p>
+    <p>An <dfn>execution context</dfn> is a specification device that is used to track the runtime evaluation of code by an ECMAScript implementation. At any point in time, there is at most one execution context that is actually executing code. This is known as the <dfn id="running-execution-context">running execution context</dfn>.</p>
+    <p>An <dfn id="execution-context-stack">execution context stack</dfn> is used to track execution contexts. The running execution context is always the top element of this stack. A new execution context is created whenever control is transferred from the executable code associated with the currently running execution context to executable code that is not associated with that execution context. The newly created execution context is pushed onto the stack and becomes the running execution context.</p>


Perhaps "The" instead of "An" to suggest that there is only one of these (and soon to be one-per-continent once that language lands)?

This was really just splitting up into two paragraphs what was previously one long paragraph but happy to make that change while I'm here.

Done and pushed.

This brings the script execution parts of the spec up to date with the latest changes in ES. Notable changes include: - Removed of some generalization to allow non-JavaScript scripting languages (including XML-based ones). - Let ES track the execution context stack, and the corresponding stack of scripts; see tc39/ecma262#242. This allows us to stop tracking the stack of script settings objects, instead defining entry and incumbent settings objects with reference to ES. This is especially important since our current mechanism, of monkey-patching the SourceElement grammatical construction, no longer works since that grammar production has disappeared. Fixes #155. - Established a direct correspondence between HTML's "script" concept and ES's Script Record/Module Record concepts. The former is stored in the [[HostDefined]] slot of the latter. - The process of parsing and executing scripts, including handling any resulting errors, is now formalized and appropriately calls out to ES's ParseScript and ScriptEvaluation abstract operations. Note that we do *not* use the ECMAScript ScriptEvaluationJob or job queue for our script parsing/evaluation. That setup is overengineered and does not serve the needs of HTML; we hope to remove it from ES eventually. (See tc39/ecma262#240 (comment) for details.) Instead we simply use ParseScript and EvaluateScript directly. This almost completes https://www.w3.org/Bugs/Public/show_bug.cgi?id=25981, with the remaining work item being to integrate promise jobs and ensure they are run as microtasks.

bterlson · 2015-12-15T19:18:12Z

Seems good to go! @domenic you ready? :-P

domenic · 2015-12-15T19:20:01Z

Yes, do it! :D

UltCombo · 2015-12-15T19:27:47Z

Looks like there is a full copy of the spec in the PR, which most likely originated from a merge conflict. I guess @bterlson can take care of it while rebasing.

This brings the script execution parts of the spec up to date with the latest changes in ES. Notable changes include: - Removed of some generalization to allow non-JavaScript scripting languages (including XML-based ones). - Let ES track the execution context stack, and the corresponding stack of scripts; see tc39/ecma262#242. This allows us to stop tracking the stack of script settings objects, instead defining entry and incumbent settings objects with reference to ES. This is especially important since our current mechanism, of monkey-patching the SourceElement grammatical construction, no longer works since that grammar production has disappeared. Fixes #155. - Established a direct correspondence between HTML's "script" concept and ES's Script Record/Module Record concepts. The former is stored in the [[HostDefined]] slot of the latter. - The process of parsing and executing scripts, including handling any resulting errors, is now formalized and appropriately calls out to ES's ParseScript and ScriptEvaluation abstract operations. Note that we do *not* use the ECMAScript ScriptEvaluationJob or job queue for our script parsing/evaluation. That setup is overengineered and does not serve the needs of HTML; we hope to remove it from ES eventually. (See tc39/ecma262#240 (comment) for details.) Instead we simply use ParseScript and EvaluateScript directly. This almost completes https://www.w3.org/Bugs/Public/show_bug.cgi?id=25981, with the remaining work item being to integrate promise jobs and ensure they are run as microtasks.

bterlson · 2015-12-15T19:31:08Z

Yes I have discovered this and am fixing presently.

domenic · 2015-12-15T19:31:10Z

Arghleblargle, will fix, one sec.

Fixes tc39#78. Introduces the concept of a Script Record as a counterpart to a Module Record, and adds [[HostDefined]] fields to both of them. Every execution environment now has ScriptOrModule component pointing back to its "creator" script or module: - Top level script/module execution contexts point to the appropriate script/module. - Function execution contexts for functions declared at top level point to that of the containing script/module. - Function execution contexts for functions declared inside other functions point to that of the containing function, skipping built-in functions. - Eval execution contexts point to that of their containing function (skipping built-in functions, including `eval` itself). - Job execution contexts (which are largely a spec artifact) point to the script/module that originally enqueued the job. In the course of doing this, a couple other substantial changes were introduced: - Factored out ParseScript and ScriptEvaluation abstract operations, so that script parsing now better parallels module parsing. - Added HostReportErrors for reporting both parsing errors and runtime errors, and wired it in to parsing and NextJob as appropriate.

bterlson · 2015-12-15T19:32:03Z

ok will wait for @domenic :-P

domenic · 2015-12-15T19:32:44Z

done

This brings the script execution parts of the spec up to date with the latest changes in ES. Notable changes include: - Removed of some generalization to allow non-JavaScript scripting languages (including XML-based ones). - Let ES track the execution context stack, and the corresponding stack of scripts; see tc39/ecma262#242. This allows us to stop tracking the stack of script settings objects, instead defining entry and incumbent settings objects with reference to ES. This is especially important since our current mechanism, of monkey-patching the SourceElement grammatical construction, no longer works since that grammar production has disappeared. Fixes #155. - Established a direct correspondence between HTML's "script" concept and ES's Script Record/Module Record concepts. The former is stored in the [[HostDefined]] slot of the latter. - The process of parsing and executing scripts, including handling any resulting errors, is now formalized and appropriately calls out to ES's ParseScript and ScriptEvaluation abstract operations. Note that we do *not* use the ECMAScript ScriptEvaluationJob or job queue for our script parsing/evaluation. That setup is overengineered and does not serve the needs of HTML; we hope to remove it from ES eventually. (See tc39/ecma262#240 (comment) for details.) Instead we simply use ParseScript and EvaluateScript directly. This almost completes https://www.w3.org/Bugs/Public/show_bug.cgi?id=25981, with the remaining work item being to integrate promise jobs and ensure they are run as microtasks.

bterlson · 2015-12-15T19:56:17Z

Committed as 698819c (added a note to the commit message about the bugzilla bug fixed :-P)

This brings the script execution parts of the spec up to date with the latest changes in ES. Notable changes include: - Removed of some generalization to allow non-JavaScript scripting languages (including XML-based ones). - Let ES track the execution context stack, and the corresponding stack of scripts; see tc39/ecma262#242. This allows us to stop tracking the stack of script settings objects, instead defining entry and incumbent settings objects with reference to ES. This is especially important since our current mechanism, of monkey-patching the SourceElement grammatical construction, no longer works since that grammar production has disappeared. Fixes #155. - Established a direct correspondence between HTML's "script" concept and ES's Script Record/Module Record concepts. The former is stored in the [[HostDefined]] slot of the latter. - The process of parsing and executing scripts, including handling any resulting errors, is now formalized and appropriately calls out to ES's ParseScript and ScriptEvaluation abstract operations. Note that we do *not* use the ECMAScript ScriptEvaluationJob or job queue for our script parsing/evaluation. That setup is overengineered and does not serve the needs of HTML; we hope to remove it from ES eventually. (See tc39/ecma262#240 (comment) for details.) Instead we simply use ParseScript and EvaluateScript directly. This almost completes https://www.w3.org/Bugs/Public/show_bug.cgi?id=25981, with the remaining work item being to integrate promise jobs and ensure they are run as microtasks.

anba reviewed Dec 11, 2015
View reviewed changes

domenic force-pushed the track-script-module branch from 4e50c90 to dfd046a Compare December 11, 2015 23:18

domenic force-pushed the track-script-module branch from dfd046a to 4201e30 Compare December 14, 2015 17:30

domenic force-pushed the track-script-module branch 4 times, most recently from 15dc665 to a61144c Compare December 14, 2015 22:28

domenic mentioned this pull request Dec 14, 2015

Rewrite script execution on top of ES whatwg/html#401

Merged

bterlson reviewed Dec 15, 2015
View reviewed changes

domenic force-pushed the track-script-module branch from a61144c to 2c818bd Compare December 15, 2015 18:49

domenic force-pushed the track-script-module branch from 2c818bd to 90842e1 Compare December 15, 2015 19:32

bterlson closed this Dec 15, 2015

littledan mentioned this pull request Apr 8, 2019

Layering: Add GetRequestedModules() to Module Record #1501

Open

snyk-bot mentioned this pull request Sep 23, 2020

[Snyk] Upgrade ecmarkup from 4.0.0 to 4.1.0 dbalde/ecma262#53

Open

jmdyck mentioned this pull request Mar 20, 2021

GlobalDeclarationInstantiation breaks on empty Script #2354

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Layering: track execution contexts' script/module #242

Layering: track execution contexts' script/module #242

domenic commented Dec 10, 2015

anba Dec 11, 2015

domenic Dec 11, 2015

anba Dec 11, 2015

anba commented Dec 11, 2015

domenic commented Dec 11, 2015

anba commented Dec 11, 2015

domenic commented Dec 11, 2015

domenic commented Dec 12, 2015

anba commented Dec 12, 2015

domenic commented Dec 14, 2015

anba commented Dec 14, 2015

domenic commented Dec 14, 2015

anba commented Dec 14, 2015

domenic commented Dec 14, 2015

bterlson commented Dec 15, 2015

annevk commented Dec 15, 2015

anba commented Dec 15, 2015

bterlson Dec 15, 2015

domenic Dec 15, 2015

domenic Dec 15, 2015

bterlson commented Dec 15, 2015

domenic commented Dec 15, 2015

UltCombo commented Dec 15, 2015

bterlson commented Dec 15, 2015

domenic commented Dec 15, 2015

bterlson commented Dec 15, 2015

domenic commented Dec 15, 2015

bterlson commented Dec 15, 2015

Layering: track execution contexts' script/module #242

Layering: track execution contexts' script/module #242

Conversation

domenic commented Dec 10, 2015

anba Dec 11, 2015

Choose a reason for hiding this comment

domenic Dec 11, 2015

Choose a reason for hiding this comment

anba Dec 11, 2015

Choose a reason for hiding this comment

anba commented Dec 11, 2015

domenic commented Dec 11, 2015

anba commented Dec 11, 2015

domenic commented Dec 11, 2015

domenic commented Dec 12, 2015

anba commented Dec 12, 2015

domenic commented Dec 14, 2015

anba commented Dec 14, 2015

domenic commented Dec 14, 2015

anba commented Dec 14, 2015

domenic commented Dec 14, 2015

bterlson commented Dec 15, 2015

annevk commented Dec 15, 2015

anba commented Dec 15, 2015

bterlson Dec 15, 2015

Choose a reason for hiding this comment

domenic Dec 15, 2015

Choose a reason for hiding this comment

domenic Dec 15, 2015

Choose a reason for hiding this comment

bterlson commented Dec 15, 2015

domenic commented Dec 15, 2015

UltCombo commented Dec 15, 2015

bterlson commented Dec 15, 2015

domenic commented Dec 15, 2015

bterlson commented Dec 15, 2015

domenic commented Dec 15, 2015

bterlson commented Dec 15, 2015