Extending "Forbidden Extensions"

[littledan]

This nice idea from @bakkot may deserve further investigation in TC39. What do you all think?

I think the "forbidden extensions" thing only really makes sense in the historical context of browsers experimenting with language extensions without consulting with each other or the broader community (e.g.). Since they don't seem to be inclined to do that sort of thing anymore (thankfully), I don't think we necessarily need to worry about people shipping a syntax before we have consensus on it.

Personally I'd be happiest if we could add a "everything not permitted is forbidden" part to the spec, but that might be hard to get through & would require some careful wording to allow browsers to ship stage 3 proposals (since the current spec makes no reference to the proposal process at all).

I'd be happiest with the strictest practical definition of what JS is, and I see nothing impractical about @bakkot 's proposal. This proposal could enable us to share more tests between implementations through test262, for one benefit.

[bakkot]

FWIW, test262-parser-tests already maintains a list (slightly outdated, but updating it is on my todo list) of illegal program texts, where "illegal" means "not currently permitted by the specification" rather than "explicitly not permitted by the specification". Part of the reason it's split out as a separate project is because of @leobalter's very valid concern that it is difficult to figure out which illegal program texts to include, and how to organize them; that project takes a more scattershot approach of "we'll include any test anyone thought worth writing, and we're not going to worry too much about coverage or duplication".

[bakkot]

FWIW, test262-parser-tests already maintains a list (slightly outdated, but updating it is on my todo list) of illegal program texts, where "illegal" means "not currently permitted by the specification" rather than "explicitly not permitted by the specification". Part of the reason it's split out as a separate project is because of @leobalter's very valid concern that it is difficult to figure out which illegal program texts to include, and how to organize them; that project takes a more scattershot approach of "we'll include any test anyone thought worth writing, and we're not going to worry too much about coverage or duplication".

[erights]

Ignoring extensions, JavaScript has many useful inabilities; most obviously memory safety. However, I see nothing in the current spec that would forbid an implementation adding a non-config, non-write data property with builtin peek and poke methods, for directly accessing raw memory addresses. However, any theorem about what JS programs cannot do relies on such inabilities. As long as such extensions are not forbidden, such theorems are not sound.

I solicit help from formalists to help answer the following question:

What seem to be the intended inabilities that should enable useful theorems to proceed soundly? How should we specify them?

[erights]

Ignoring extensions, JavaScript has many useful inabilities; most obviously memory safety. However, I see nothing in the current spec that would forbid an implementation adding a non-config, non-write data property with builtin peek and poke methods, for directly accessing raw memory addresses. However, any theorem about what JS programs cannot do relies on such inabilities. As long as such extensions are not forbidden, such theorems are not sound.

I solicit help from formalists to help answer the following question:

What seem to be the intended inabilities that should enable useful theorems to proceed soundly? How should we specify them?

[littledan]

@erights What if we start with forbidding syntactic extensions, and then move on to considering semantic things to forbid as the next step?

[littledan]

@erights What if we start with forbidding syntactic extensions, and then move on to considering semantic things to forbid as the next step?

[erights]

@littledan Anything that gets us started, makes progress, would be great!

[erights]

@littledan Anything that gets us started, makes progress, would be great!

[allenwb]

What if we start with forbidding syntactic extensions, and then move on to considering semantic things to forbid as the next step?

So, you would try to forbid implementations from implementing experimental syntactic features or even stage 4 features that are not yet in an actual Ecma GA approved edition of ECMA-262? Or host/environment specific features such as JSX?

In practice, such a probation would be ignored and that fact that TC39 is trying to enforce unrealistic demands would diminish TC39's "moral authority" to make important realistic demands.

The current "forbidden extensions" were all carefully selected to address very specific concerns. By being limited and targeted they convey that TC39 is serious about the importance of keeping the specific identified behaviors out of the language. Expending it to say something like "no new syntax" means that TC39 is neither serious or realistic when it talks about restricting extensions and makes it more likely that the actual important restriction could be ignored.

[allenwb]

What if we start with forbidding syntactic extensions, and then move on to considering semantic things to forbid as the next step?

So, you would try to forbid implementations from implementing experimental syntactic features or even stage 4 features that are not yet in an actual Ecma GA approved edition of ECMA-262? Or host/environment specific features such as JSX?

In practice, such a probation would be ignored and that fact that TC39 is trying to enforce unrealistic demands would diminish TC39's "moral authority" to make important realistic demands.

The current "forbidden extensions" were all carefully selected to address very specific concerns. By being limited and targeted they convey that TC39 is serious about the importance of keeping the specific identified behaviors out of the language. Expending it to say something like "no new syntax" means that TC39 is neither serious or realistic when it talks about restricting extensions and makes it more likely that the actual important restriction could be ignored.

[zenparsing]

The current "forbidden extensions" were all carefully selected to address very specific concerns.

Also note that the specific concerns addressed by each restriction vary. The ":" restriction is interesting, as I think it can be interpreted two ways. On one hand, it may indicate the desire to reserve syntactic space for type annotations in JS. On the other hand, it may indicate the desire to reserve that space for compile-to-JS supersets.

@allenwb Do you remember what motivated the ":" restriction?

[zenparsing]

The current "forbidden extensions" were all carefully selected to address very specific concerns.

Also note that the specific concerns addressed by each restriction vary. The ":" restriction is interesting, as I think it can be interpreted two ways. On one hand, it may indicate the desire to reserve syntactic space for type annotations in JS. On the other hand, it may indicate the desire to reserve that space for compile-to-JS supersets.

@allenwb Do you remember what motivated the ":" restriction?

[allenwb]

Do you remember what motivated the ":" restriction?

type annotations

[allenwb]

Do you remember what motivated the ":" restriction?

type annotations

[bakkot]

@allenwb

So, you would try to forbid implementations from implementing experimental syntactic features or even stage 4 features that are not yet in an actual Ecma GA approved edition of ECMA-262?

per the OP: this "would require some careful wording to allow browsers to ship stage 3 proposals". I expect such wording could be written. We of course would not try to prevent implementations shipping stage 3 proposals which had not yet made it into the spec.

In practice, the such a probation would be ignored

Can you say more about why you think so? I'm not aware of any instance this decade in which someone intending to be an implementation of ECMAScript shipped any syntax which was neither proposed nor in the spec.

[bakkot]

@allenwb

So, you would try to forbid implementations from implementing experimental syntactic features or even stage 4 features that are not yet in an actual Ecma GA approved edition of ECMA-262?

per the OP: this "would require some careful wording to allow browsers to ship stage 3 proposals". I expect such wording could be written. We of course would not try to prevent implementations shipping stage 3 proposals which had not yet made it into the spec.

In practice, the such a probation would be ignored

Can you say more about why you think so? I'm not aware of any instance this decade in which someone intending to be an implementation of ECMAScript shipped any syntax which was neither proposed nor in the spec.

[leobalter]

I believe that generally forbidding syntax extensions is weak and too loose. The examples I'm aware so far can be specifically annotated and described. The prevented extensions for : on type annotations and the one proposed for dynamic imports are a good signal we can do a better job specifying what we want to secure for the web compat in the future. We can do the same work for other parts / proposals.

[leobalter]

I believe that generally forbidding syntax extensions is weak and too loose. The examples I'm aware so far can be specifically annotated and described. The prevented extensions for : on type annotations and the one proposed for dynamic imports are a good signal we can do a better job specifying what we want to secure for the web compat in the future. We can do the same work for other parts / proposals.

[allenwb]

Can you say more about why you think so?

The 70 year history of programming languages and how they evolve. Including the relative lack of success of previous attempts by various languages to restrict extensions. The times when it has worked (to some degree) have generally been situations where there were IP based contractual restrictions that limited extensions.

Standards are followed as long as they are useful to a business or user community. As soon as a potential competitive advantage or environmental requirement conflicts with a standard the standard goes out the window.

I'm not aware of any instance this decade in which someone intending to be an implementation of ECMAScript shipped any syntax which was neither proposed nor in the spec.

Ah... Babel; JSX; Node, non-standard top level semantics ...

Regardless, my most important point was:

that fact that TC39 is trying to enforce unrealistic demands would diminish TC39's "moral authority" to make important realistic demands.

Power can be lost if it isn't used carefully.

[allenwb]

Can you say more about why you think so?

The 70 year history of programming languages and how they evolve. Including the relative lack of success of previous attempts by various languages to restrict extensions. The times when it has worked (to some degree) have generally been situations where there were IP based contractual restrictions that limited extensions.

Standards are followed as long as they are useful to a business or user community. As soon as a potential competitive advantage or environmental requirement conflicts with a standard the standard goes out the window.

I'm not aware of any instance this decade in which someone intending to be an implementation of ECMAScript shipped any syntax which was neither proposed nor in the spec.

Ah... Babel; JSX; Node, non-standard top level semantics ...

Regardless, my most important point was:

that fact that TC39 is trying to enforce unrealistic demands would diminish TC39's "moral authority" to make important realistic demands.

Power can be lost if it isn't used carefully.

[ljharb]

Which implementations shipped runtime support for jsx? Babel is a build tool and is unconstrained by the spec in this regard, and thus not relevant to this thread.

[ljharb]

Which implementations shipped runtime support for jsx? Babel is a build tool and is unconstrained by the spec in this regard, and thus not relevant to this thread.

[devsnek]

The only thing I can think of is V8's natives syntax feature, which allows embedders and V8 itself to inline calls to V8 internals from JS. (const target = %JSProxyGetTarget(someProxy);)

[devsnek]

The only thing I can think of is V8's natives syntax feature, which allows embedders and V8 itself to inline calls to V8 internals from JS. (const target = %JSProxyGetTarget(someProxy);)

[bathos]

@ljharb I think the line gets fuzzy sometimes:

require('@babel/register');

[bathos]

@ljharb I think the line gets fuzzy sometimes:

require('@babel/register');

[ljharb]

@bathos sure but that’s runtime stuff, not shipped by the implementation, nor syntax. If anything that’s an argument for forbidding any extensions, because it can always be enabled by a runtime mechanism.

[ljharb]

@bathos sure but that’s runtime stuff, not shipped by the implementation, nor syntax. If anything that’s an argument for forbidding any extensions, because it can always be enabled by a runtime mechanism.

[allenwb]

Babel is a build tool and is unconstrained by the spec in this regard, and thus not relevant to this thread.

Of course Babel is an implementation of ECMA-262 and historically one that has tended to have lots of deviations and extensions to the specification.

There is (intentionally) nothing in ECAM-262 that says what technologies or techniques must be used to implement the specified language. Source to source translation (regardless of the target language) is a perfectly valid implementation technique.

[allenwb]

Babel is a build tool and is unconstrained by the spec in this regard, and thus not relevant to this thread.

Of course Babel is an implementation of ECMA-262 and historically one that has tended to have lots of deviations and extensions to the specification.

There is (intentionally) nothing in ECAM-262 that says what technologies or techniques must be used to implement the specified language. Source to source translation (regardless of the target language) is a perfectly valid implementation technique.

[bakkot]

Babel does not by default include any plugins which extend the spec, and plugins can and do change the syntax and semantics of things which are covered by the spec, rather than merely adding non-forbidden extensions. I don't think babel would have a problem with this, at least any more than it has a problem with the current spec - it is already the case the current spec does not carve out nearly enough space in what extensions it permits to fit all of babel's plugins. (I am technically a member of the babel team, but am not speaking on behalf of that team.)

Regardless, my most important point was:

that fact that TC39 is trying to enforce unrealistic demands would diminish TC39's "moral authority" to make important realistic demands.

Sure, but the question is whether this is actually unrealistic. To me, it looks like people who try to follow the ECMAScript standard have not, in recent years, wanted to ship non-standard non-proposed syntax. If we expect that to continue - and perhaps you're right that we shouldn't, but I'm not totally convinced - then this would not be an unrealistic demand.

[bakkot]

Babel does not by default include any plugins which extend the spec, and plugins can and do change the syntax and semantics of things which are covered by the spec, rather than merely adding non-forbidden extensions. I don't think babel would have a problem with this, at least any more than it has a problem with the current spec - it is already the case the current spec does not carve out nearly enough space in what extensions it permits to fit all of babel's plugins. (I am technically a member of the babel team, but am not speaking on behalf of that team.)

Regardless, my most important point was:

that fact that TC39 is trying to enforce unrealistic demands would diminish TC39's "moral authority" to make important realistic demands.

Sure, but the question is whether this is actually unrealistic. To me, it looks like people who try to follow the ECMAScript standard have not, in recent years, wanted to ship non-standard non-proposed syntax. If we expect that to continue - and perhaps you're right that we shouldn't, but I'm not totally convinced - then this would not be an unrealistic demand.

[allenwb]

To me, it looks like people who try to follow the ECMAScript standard have not, in recent years, wanted to ship non-standard non-proposed syntax.

JSX?

If we expect that to continue - and perhaps you're right that we shouldn't, but I'm not totally convinced - then this would not be an unrealistic demand.

I expect change, in everything. Expecting that a status quo will continue is unrealistic.

[allenwb]

To me, it looks like people who try to follow the ECMAScript standard have not, in recent years, wanted to ship non-standard non-proposed syntax.

JSX?

If we expect that to continue - and perhaps you're right that we shouldn't, but I'm not totally convinced - then this would not be an unrealistic demand.

I expect change, in everything. Expecting that a status quo will continue is unrealistic.

[zenparsing]

I think @allenwb 's point is that ECMA-262's authority to prevent things is meager to begin with and that expanding this section may tend to dilute that authority rather than increase it.

[zenparsing]

I think @allenwb 's point is that ECMA-262's authority to prevent things is meager to begin with and that expanding this section may tend to dilute that authority rather than increase it.

[bakkot]

JSX?

To my knowledge most consumers of JSX also make use of :-style type annotations, which are currently explicitly forbidden by the spec. I don't think this has created problems for anyone.

I don't think those two things are meaningfully different. Either the spec should forbid both, and people wishing to use build tools to add non-standard syntax which compiles to spec-compliant JS can continue to ignore the part of spec forbidding such extensions just as they do today, or the spec should permit both.

I think @allenwb 's point is that ECMA-262's authority to prevent things is meager to begin with and that expanding this section may tend to dilute that authority rather than increase it.

Yeah; I think this depends on whether we think browsers will be sufficiently motivated to ship non-standard non-proposed syntax that they're willing to ignore that authority. Personally, I don't think they will. If the committee thinks otherwise, certainly we should not pursue this.

[bakkot]

JSX?

To my knowledge most consumers of JSX also make use of :-style type annotations, which are currently explicitly forbidden by the spec. I don't think this has created problems for anyone.

I don't think those two things are meaningfully different. Either the spec should forbid both, and people wishing to use build tools to add non-standard syntax which compiles to spec-compliant JS can continue to ignore the part of spec forbidding such extensions just as they do today, or the spec should permit both.

I think @allenwb 's point is that ECMA-262's authority to prevent things is meager to begin with and that expanding this section may tend to dilute that authority rather than increase it.

Yeah; I think this depends on whether we think browsers will be sufficiently motivated to ship non-standard non-proposed syntax that they're willing to ignore that authority. Personally, I don't think they will. If the committee thinks otherwise, certainly we should not pursue this.

[ljharb]

Browsers and all major engines are on the committee - if the committee thinks that forbidding syntax extensions is tenable, how is that not an explicit indicator that the spec indeed does have the authority to do so, granted by the implementations themselves?

[ljharb]

Browsers and all major engines are on the committee - if the committee thinks that forbidding syntax extensions is tenable, how is that not an explicit indicator that the spec indeed does have the authority to do so, granted by the implementations themselves?