Normative: phrase tail calls as discarding resources rather than popping execution context stack

[bakkot]

In #2409, #2400, #2429, #2413, and other issues, we've been trying to clarify how execution contexts work within the spec. I think I have a good handle on how to clarify all of the sources of confusion with some editorial tweaks, with the exception of PrepareForTailCall.

PrepareForTailCall is particularly strange. Unlike everywhere else in the spec (as of #2413), it pops the execution context stack without immediately resuming the execution context which is now at the top of the stack, leaving the spec's abstract machine in an odd state where the "running execution context" (defined to be the top of the stack) is not actually running in the sense of "not suspended".

I talked to @allenwb about the intended interpretation of PrepareForTailCall (thanks Allen!). He explained that the actual goal of the AO is just to require engines discard the resources associated with the current call - popping the execution context stack and discarding the former top-of-stack is intended to signify that. But manipulating the execution context stack has other consequences with regards to the transfer of control within the specification's abstract machine, and I don't have a good way to make that coherent for PrepareForTailCall as currently phrased.

Instead, this PR replaces PrepareForTailCall's pop of the execution context stack with an explicit instruction to discard the resources associated with the running execution context, without actually removing it from the execution context stack in the specification's abstract machine.

Unfortunately, this has observable consequences for the realm of the TypeError which results when tail-calling a revoked proxy (although no other observable consequences, that I can tell), which is why this is Normative. Concretely, in

let F = evalInNewRealm(`
  (function F() {
    let { proxy, revoke } = Proxy.revocable(() => {}, {});
    revoke();
    return proxy();
  })
`);

try {
  F();
} catch (e) {
  console.log(e instanceof TypeError);
}

prior to this PR the above snippet would return true (because the TypeError is created in the realm of the code which invokes F, since F's execution context is discarded from the stack before the TypeError is thrown), and with this PR it would return false (because F's execution context is still on top of the stack at the time the TypeError is thrown).

Of the engines I tested, only JavaScriptCore and XS implemented tail calls. I couldn't figure out how to observe the realm for error objects in XS, and in JSC the error is always created in the realm of the Proxy object itself, which is wrong both before and after this PR. So I don't think there's any web-reality issues with this PR.

This PR is an alternative to #2430.

[devsnek]

this feels more confusing in the context of communicating to someone how to implement this language.

[bakkot]

this feels more confusing in the context of communicating to someone how to implement this language.

Hm. I very, very strongly disagree.

[jmdyck]

this feels more confusing in the context of communicating to someone how to implement this language.

More confusing than the status quo?

[devsnek]

More confusing than the status quo?

yes, i don't find the idea that the spec requires an execution context to perform logic to be very useful. this also came up with the duplicated execution context when async functions are called.

[bakkot]

i don't find the idea that the spec requires an execution context to perform logic to be very useful

This PR doesn't really do anything to entrench that idea. Rather, it's attempting to address the thing where changing the top-of-stack execution context is sometimes used as a transfer of control, but in an incoherent way - see the removed note about "the above call will not return here", which implies that PrepareForTailCall in its current state affects the control of the VM in some unspecified and difficult-to-reason-about way, and also see the various notes about transfers of control in generators.

The goal of this PR is to make the transfers of control consistent with the execution stack manipulation, by removing this one execution stack manipulation that is said to cause a transfer of control (eventually) in a different (and unspecified) way than all others.

[bakkot]

I got consensus for the normative change implied by this PR at TC39 today.

[michaelficarra]

This LGTM but I'd like to discuss the "Discard all resources associated with the current execution context" phrasing at the next editor call before merging. I think we can choose a phrasing that better reflects the committee's desired constraints for PTCs.

[jugglinmike]

Test under review here: tc39/test262#3181

[jmdyck]

I got consensus for the normative change implied by this PR at TC39 today.

Am I right in thinking that #2430 doesn't involve this normative change, and so is effectively dead?

[bakkot]

Almost certainly, yes. I'll confirm with the other editors at our next call what approach we want to take.

[bakkot]

@jmdyck Yes, we're going to go with this approach rather than #2430. Thanks for writing it up, though; it and our conversations elsewhere have been very helpful in thinking through this, particularly your very detailed PR message.