Skip to content
This repository has been archived by the owner on Jan 25, 2022. It is now read-only.

Example of the utility of static private methods #4

Closed
domenic opened this issue Dec 21, 2017 · 70 comments
Closed

Example of the utility of static private methods #4

domenic opened this issue Dec 21, 2017 · 70 comments

Comments

@domenic
Copy link
Member

domenic commented Dec 21, 2017

This is loosely based on existing code in jsdom, with some additions to illustrate further points.

I am working on a class, call it JSDOM, which has some static factory methods:

export class JSDOM {
  // ... elided ...
  
  static fromURL(url, options) {
    if (options.referrer === undefined) {
      throw new TypeError();
    }
    if (options.url === undefined) {
      throw new TypeError();
    }
    options.url = (new URL(options.url)).href; // normalize
    
    // ... TODO: actual interesting stuff
  }
  
  static fromFile(filename, options) {
    if (options.url === undefined) {
      throw new TypeError();
    }
    options.url = (new URL(options.url)).href; // normalize
    
    // ... TODO: actual interesting stuff
  }
}

I notice two things:

  • The options preprocessing is distracting from the main important part of the method.
  • Some of my options preprocesing is shared between both classes.

Both of these are things that are addressed well by the extract method refactoring.

I have two choices on how to do this:

  1. Extract into actual static methods of the class.
  2. Extract into functions at module-level.

Although it is a matter of opinion, I find (1) better than (2), because it keeps the code inside the class that it's designed for, and near the methods that use it, instead of somewhere later in the file. Today, (1) has the disadvantage of polluting my public API, so I usually choose (2). But if we had static private methods, I could do (1) without that drawback.

Still, let's assume that arguments from #1 prevail and we don't get static private methods. It's not too bad:

export class JSDOM {
  // ... elided ...
  
  static fromURL(url, options = {}) {
    normalizeFromURLOptions(options);
    normalizeOptions(options);
    
    // ... TODO: actual interesting stuff
  }
  
  static fromFile(filename, options = {}) {
    normalizeOptions(options);
    
    // ... TODO: actual interesting stuff
  }
}

function normalizeFromURLOptions(options) {
  if (options.referrer === undefined) {
    throw new TypeError();
  }
}

function normalizeOptions(options) {
  if (options.url === undefined) {
    throw new TypeError();
  }
  options.url = (new URL(options.url)).href;
}

But as I continue my work on implementing these functions, the lack of static private ends up felt more deeply. I get to the following point:

export const registry = new JSDOMRegistry();

export class JSDOM {
  #createdBy;
  
  #registerWithRegistry() {
    // ... elided ...
  }
 
  async static fromURL(url, options = {}) {
    normalizeFromURLOptions(options);
    normalizeOptions(options);
    
    const body = await getBodyFromURL(url);
    const jsdom = new JSDOM(body, options);
    jsdom.#createdBy = "fromURL";
    jsdom.#registerWithRegistry(registry);
    return jsdom;
  }
  
  static fromFile(filename, options = {}) {
    normalizeOptions(options);
    
    const body = await getBodyFromFilename(filename);
    const jsdom = new JSDOM(body, options);
    jsdom.#createdBy = "fromFile";
    jsdom.#registerWithRegistry(registry);
    return jsdom;
  }
}

Again I notice I have some duplicated code. Here is the code I want to write to clean this up:

export const registry = new JSDOMRegistry();

export class JSDOM {
  #createdBy;
  
  #registerWithRegistry() {
    // ... elided ...
  }
 
  async static fromURL(url, options = {}) {
    normalizeFromURLOptions(options);
    normalizeOptions(options);
    
    const body = await getBodyFromURL(url);
    return JSDOM.#finalizeFactoryCreated(new JSDOM(body, options), "fromURL");
  }
  
  static fromFile(filename, options = {}) {
    normalizeOptions(options);
    
    const body = await getBodyFromFilename(filename);
    return JSDOM.#finalizeFactoryCreated(new JSDOM(body, options), "fromFile");
  }
  
  static #finalizeFactoryCreated(jsdom, factoryName) {
    jsdom.#createdBy = factoryName;
    jsdom.#registerWithRegistry(registry);
    return jsdom;
  }
}

But I can't, because static private methods don't exist. Here is the code I have to write to work around that:

export const registry = new JSDOMRegistry();

export class JSDOM {
  #createdBy;
  
  #registerWithRegistry() {
    // ... elided ...
  }
 
  async static fromURL(url, options = {}) {
    normalizeFromURLOptions(options);
    normalizeOptions(options);
    
    const body = await getBodyFromURL(url);
    return finalizeFactoryCreated(
      new JSDOM(body, options), "fromURL",
      (jsdom, value) => jsdom.#createdBy = value,
      (jsdom, ...args) => jsdom.#registerWithRegistry(...args)
    );
  }
  
  static fromFile(filename, options = {}) {
    normalizeOptions(options);
    
    const body = await getBodyFromFilename(filename);
    return finalizeFactoryCreated(
      new JSDOM(body, options), "fromFile",
      (jsdom, value) => jsdom.#createdBy = value,
      (jsdom, ...args) => jsdom.#registerWithRegistry(...args)
    );
  }
}

function finalizeFactoryCreated(jsdom, factoryName, createdBySetter, registerCaller) {
  createdBySetter(jsdom, factoryName);
  registerCaller(jsdom, registry);
  return jsdom;
}

This code is basically worse than the original code with duplication. The lack of static private, and the workarounds I have been forced to employ to get around it, have made persisting with duplication a better choice than attempting to apply extract method---because extract method has basically been broken.


This is why I think static private methods are quite important. Without it you cannot apply basic refactoring patterns, which users will expect to be feasible, to JS classes.

@littledan
Copy link
Member

@zkat @ljharb Do you find these scenarios relevant or useful?

@ljharb
Copy link
Member

ljharb commented Dec 27, 2017

@littledan not to me personally - i'd use "Extract into functions at module-level" every time - but I think the OP is a well-motivated justification for static private (the first I've seen).

@domenic
Copy link
Member Author

domenic commented Dec 27, 2017

"Extract into functions at module-level"

As shown in the second half, that doesn't really work if you need access to private state.

@ljharb
Copy link
Member

ljharb commented Dec 27, 2017

Very true; i was trying to come up with an example I’d prefer, but the access to instance private state is the kicker.

@littledan
Copy link
Member

Seems like a key aspect here, which I missed in explaining the feature in the past, is that it's especially likely to come up with factory methods. I think there's a not quite exactly so bad way to split it up, by using instance private methods for the "second half":

export const registry = new JSDOMRegistry();

export class JSDOM {
  #createdBy;
  
  #registerWithRegistry(registry) {
    // ... elided ...
  }

  #init(factoryName) {
    this.#createdBy = factoryName;
    this.#registerWithRegistry(registry);
    return this;
  }
 
  async static fromURL(url, options = {}) {
    normalizeFromURLOptions(options);
    normalizeOptions(options);
    
    const body = await getBodyFromURL(url);
    return new JSDOM(body, options).#init("fromURL");
  }
  
  static fromFile(filename, options = {}) {
    normalizeOptions(options);
    
    const body = await getBodyFromFilename(filename);
    return new JSDOM(body, options).#init("fromFile");
  }
}

Yes, this is splitting in two, and if you're refactoring between public and private static factory methods, it would be not so much fun. Maybe there are other cases which split up even worse, and maybe it's unintuitive how to do the refactoring I did above, but any case I can think of divides up into two pieces more or less--before you create the instance (function outside the class) and after (instance private method). The final code, in this particular case, doesn't look so bad to me.

Does anyone have any use cases for private static fields?

@domenic
Copy link
Member Author

domenic commented Dec 27, 2017

Does anyone have any use cases for private static fields?

Only personal-preference ones, analogous to the (1) vs. (2) from the first half of my post.

@zkat
Copy link

zkat commented Dec 28, 2017

I agree that this is a compelling reason to support them! Thanks, domenic :)

@littledan
Copy link
Member

@allenwb @erights What do you think about this motivation for static private methods?

@erights
Copy link

erights commented Dec 29, 2017

I favor the variant of (2) that @allenwb suggested: support lexical declarations at the class level. This is like extracting to a function at module level, except that it is in the scope of the private instance field names, and can therefore access that private state on its arguments.

What ever happened to that suggestion? Did anyone ever turn it into a proposal? Did it ever get disqualified for some reason? Why?

@domenic
Copy link
Member Author

domenic commented Dec 29, 2017

I would be happy with such a variant. Especially if the way you declared such lexical declarations was by using static #foo() { } and static #bar = baz; ;).

@ljharb
Copy link
Member

ljharb commented Dec 29, 2017

@erights to clarify: static publics, and instance publics and privates, would continue as proposed, but static privates would be covered by lexical declarations (presumably with var/let/const) at the class level?

@ljharb
Copy link
Member

ljharb commented Dec 29, 2017

Wasn't the declaration static what creates the inheritance confusion in the first place that demoted statics down to stage 2?

@littledan
Copy link
Member

littledan commented Dec 30, 2017

@erights My concern (@wycats convinced me of this) with @allenwb 's proposal: Keywords like function and let don't exactly scream, "unlike the other things in this list, this declaration is not exported and it's also not specific to the instance". I think something that is more the form of a field or method declaration would be a better fit for a class body and more intuitive.

I like what @domenic is suggesting: using me of syntax for a lyrically scoped function or variable. I presented something like this initially as the semantics for both instance and static private methods (not my idea; it was derived from earlier comments from @allenwb and others. See this patch which switched to doing type checks). Several committee members said in issues (#1 #3) that would prefer type checks on the receiver rather than the lexically scoped semantics.

If we did that sort of change, there are some semantic details that I'm unsure of. Should we revisit these semantics for just static private fields and methods, but leave static private methods actually on the instance? What would we do with the receiver--just pass it as the receiver of the function, and entirely ignore it for static private fields?

@bakkot
Copy link

bakkot commented Jan 2, 2018

@domenic, strictly speaking you can still do what you want without requiring all that overhead:

export const registry = new JSDOMRegistry();

let finalizeFactoryCreated;
export class JSDOM {
  #createdBy;
  
  #registerWithRegistry() {
    // ... elided ...
  }
 
  async static fromURL(url, options = {}) {
    normalizeFromURLOptions(options);
    normalizeOptions(options);
    
    const body = await getBodyFromURL(url);
    return finalizeFactoryCreated(new JSDOM(body, options), "fromURL");
  }
  
  static fromFile(filename, options = {}) {
    normalizeOptions(options);
    
    const body = await getBodyFromFilename(filename);
    return finalizeFactoryCreated(new JSDOM(body, options), "fromFile");
  }
  
  static finalizeFactoryCreated(jsdom, factoryName) {
    jsdom.#createdBy = factoryName;
    jsdom.#registerWithRegistry(registry);
    return jsdom;
  }
}
finalizeFactoryCreated = JSDOM.finalizeFactoryCreated;
delete JSDOM.finalizeFactoryCreated;

But I agree this is significantly less clean.

On the other hand, the pattern that I've given works even if you're using fromURL on a subclass of JSDOM, whereas the static private solution does not, and which I think people will expect to work.

Edit: sorry, that that bit is false of course, since you've written JSDOM rather than this. Still, I don't know how well we'll be able to convince people not to use this, given that it is the overwhelmingly common way of writing it in languages Java, in my experience.

Edit 2: the second part of the edit was also false. :| Note to self: I should not try to remember how languages work when running a fever.

None of our options seem all that great.

@domenic
Copy link
Member Author

domenic commented Jan 2, 2018

(Replying to the edit.)

I'd be very surprised if people used this instead of the class name. I didn't even know you could do that in Java; when I was taught static methods it was always either unqualified or using the class name. At least one StackOverflow answer seems to back me up there.

@bakkot
Copy link

bakkot commented Jan 2, 2018

Now that I look, in fact you can't. Don't know what I was thinking, sorry.

@littledan
Copy link
Member

@bakkot I don't think we should be thinking about delete as a viable option of a replacement. Without that, do you have other thoughts on whether private static methods are significantly motivated?

@littledan
Copy link
Member

littledan commented Jan 2, 2018

I've updated the explainer to advocate for the original solution, which does include static private methods, based on the feedback in this thread that they are important. Please keep the feedback coming!

@bakkot
Copy link

bakkot commented Jan 2, 2018

@littledan I expect that pattern could be captured by a decorator without requiring any deletes, but it would still be awkward, yes.

Without that, do you have other thoughts on whether private static methods are significantly motivated?

This issue gives a good motivation, I think, but I'm still worried about the subclass footgun. I think it actually is pretty likely to come up in real code if we go with the current semantics.

I'll riff off @domenic's example above (the 'code I want to write' option). Suppose that we the implementors of the JSDOM class furthermore decide we'd like consumers to be able to customize the behavior of the class further, say by providing their own implementations of getBodyFromURL, fromFile, or instance methods. Here's how I'd want to write that:

export const registry = new JSDOMRegistry();

export class JSDOM {
  #createdBy;
  
  #registerWithRegistry() {
    // ... elided ...
  }

  async static getBodyFromURL(url) {
    // ... elided ...
  }

  async static getBodyFromFilename(filename) {
    // ... elided ...
  }
 
  async static fromURL(url, options = {}) {
    normalizeFromURLOptions(options);
    normalizeOptions(options);
    
    const body = await this.getBodyFromURL(url);
    return this.#finalizeFactoryCreated(new this(body, options), "fromURL");
  }
  
  static fromFile(filename, options = {}) {
    normalizeOptions(options);
    
    const body = await this.getBodyFromFilename(filename);
    return this.#finalizeFactoryCreated(new this(body, options), "fromFile");
  }
  
  static #finalizeFactoryCreated(jsdom, factoryName) {
    jsdom.#createdBy = factoryName;
    jsdom.#registerWithRegistry(registry);
    return jsdom;
  }
}
// elsewhere:

import { JSDOM } from "jsdom";

class SpecializedJSDOM extends JSDOM {
  static getBodyFromFilename(filename) {
    // ... some different implementation ...
  }

  static fromFile(filename, options = { something }) {
    return super.fromFile(filename, options);
  }

  serialize() {
    // ... some different implementation ...
  }
}
(diff)

--- domenic.js
+++ me.js
@@ -6,20 +6,28 @@
   #registerWithRegistry() {
     // ... elided ...
   }
+
+  async static getBodyFromURL(url) {
+    // ... elided ...
+  }
+
+  async static getBodyFromFilename(filename) {
+    // ... elided ...
+  }
  
   async static fromURL(url, options = {}) {
     normalizeFromURLOptions(options);
     normalizeOptions(options);
     
-    const body = await getBodyFromURL(url);
-    return JSDOM.#finalizeFactoryCreated(new JSDOM(body, options), "fromURL");
+    const body = await this.getBodyFromURL(url);
+    return this.#finalizeFactoryCreated(new this(body, options), "fromURL");
   }
   
   static fromFile(filename, options = {}) {
     normalizeOptions(options);
     
-    const body = await getBodyFromFilename(filename);
-    return JSDOM.#finalizeFactoryCreated(new JSDOM(body, options), "fromFile");
+    const body = await this.getBodyFromFilename(filename);
+    return this.#finalizeFactoryCreated(new this(body, options), "fromFile");
   }
   
   static #finalizeFactoryCreated(jsdom, factoryName) {
@@ -28,3 +36,24 @@
     return jsdom;
   }
 }

Unfortunately, this doesn't work: trying to call SpecializedJSDOM.fromFile('file') throws an error at this.#finalizeFactoryCreated because this is SpecializedJSDOM, which lacks the #finalizeFactoryCreated private slot. It could be made to work by using JSDOM.#finalizeFactoryCreated instead of this.#finalizeFactoryCreated, but since the preceding line needs to be this.getBodyFromURL (if it hardcodes JSDOM here, subclasses can't overwrite that method), I think it's going to be very easy to use the this form for the static private method as well as the static public method. Moreover, it feels really awkward to me to be forced to avoid it.

So while it's true that, as the readme explains, that linters and good error messages and so on might be able to discourage the use of this.#staticMethod(), I think it would be better if they didn't have to.

@littledan
Copy link
Member

As an alternative @bakkot and I have discussed installing private static methods on subclasses, but omitting private static fields. Do you all have any thoughts on that approach?

@bakkot
Copy link

bakkot commented Jan 2, 2018

To add on to the above: come to think of it, there's a case where using JSDOM.#finalizeFactoryCreated just isn't good enough. (In fact @erights pointed this out a while ago.)

Suppose that #finalizeFactoryCreated wants to provide an extension point for subclasses, such as

export class JSDOM {
  // ...

  static #finalizeFactoryCreated(jsdom, factoryName) {
    if (typeof this.finalize === 'function') {
      this.finalize(jsdom, factoryName);
    }
    // ...
  }
}

If you can use this.#finalizeFactoryCreated, everything works out: the receiver for #finalizeFactoryCreated is the subclass, and so the implementation of #finalizeFactoryCreated can see the subclass's this.finalize. If you're forced to write JSDOM.#finalizeFactoryCreated, that simply doesn't work and cannot be made to work.


This sort of thing is why I am in favor of the approach @littledan mentions.

@domenic
Copy link
Member Author

domenic commented Jan 2, 2018

I am very skeptical of this hypothesizing about desiring some sort of polymorphic dispatch on static members, and especially private static members. I don't believe any other language supports that, and I don't think we should be adding machinery to JS (such as copying the methods to subclasses) in order to support that use case.

My mental model of statics is essentially that they are lexical declarations at class level. I realize for public static you can do polymorphic dispatch, but I am fine saying that is not true for private statics, especially given that private instance fields are already very anti-polymorphic by virtue of not participating in the prototype chain.

@littledan
Copy link
Member

@domenic I'm not so convinced by the "code grouping" requirement. There's lots of things that we don't allow grouped, e.g., class declarations in the top level of a class (you could do this through static private fields but it would look weird). I think it should be OK to put things that are not exported outside the class declaration; I don't think initializer expressions are likely to contain something that needs to reference private fields or methods.

One reason that supporting private static methods on subclasses isn't all that ad-hoc is that it's actually sort of analogous to instance private methods--it's installed on all the objects that it would've otherwise be found on the prototype chain. Does this make @bakkot 's proposal any less excessive-feeling to you?

About polymorphic dispatch on static members: in another thread (having trouble finding it now), @allenwb explained that ES6's practice of assembling a prototype chain of constructors is deliberate, useful and matches other dynamic object-oriented languages. About the complexity of private fields in JS and the implications of that: It's true that this is relatively new ground, largely because other dynamic languages do not attempt to enforce privacy to this extent (but we have good reasons for making the choice we did).

If the mental model of statics is that they're lexical function declarations, then this would allow for the genericity in @bakkot 's comment. If you really want lexical semantics, we could just straight-up say that private static fields are just lexically scoped variables and private static methods are just lexically scoped functions (all with funny method/field-like syntax). Would anyone prefer that?

@bakkot
Copy link

bakkot commented Jan 2, 2018

@littledan

About polymorphic dispatch on static members: in another thread (having trouble finding it now)

I'd guessing you're thinking of this comment.

@littledan
Copy link
Member

littledan commented Jan 3, 2018

tldr: I'll change the semantics to be, you can use this.#foo() on subclasses for static private methods, but for static private fields, you have to use ClassName.#bar or otherwise risk getting a TypeError

Following some discussion on #tc39 with @bakkot, @domenic and @ljharb , I'm leaning towards a new intermediate option: Private static methods are installed on subclasses, but private static fields are not and still have a TypeError when read or written to subclasses. I'll edit the explainer to give more detailed reasoning and write spec text specifying this option, but to summarize:

  • Adding private static methods to subclasses will be useful and also follow similar logic to instance private methods: although these methods are specified as own fields of the instance/constructor, they are logically as if they are shared based on the prototype chain, since they are immutable and set on each object that would have been able to access the method through the prototype chain.
  • For private static fields, because they are mutable and may be shadowed in the prototype chain, there's no good, analogous semantics for them to have in subclasses. It would be fairly crazy to replicate the semantics of ordinary properties and prototype chains, and partial workarounds (e.g., allowing reads but not writes from subclasses) are very complicated. These would be included but throw a TypeError when read from or written to in a subclass.

@wycats
Copy link

wycats commented Jan 3, 2018

@littledan I quite like this outcome.

It aligns class initialization more closely with instance initialization where it can be done without confusing side effects, and disallows cases where side-effects make all of the options compromised.

At the same time, the use-cases for static private fields can virtually always be accomplished using a lexical variable, which was not the case for static private methods.

TLDR: Between really needing static private methods semantically and the fact that they aren't confusing due to side effects, there's a strong argument for making them work. Inversely, between not really needing static private fields semantically and the fact that they are confusing due to side effects, there's a strong argument for disallowing them. I like this conclusion.

@ljharb
Copy link
Member

ljharb commented Jan 3, 2018

@littledan to clarify, for static private methods you can use (where #foo is a static private method on a superclass) this.#foo() on subclasses, but not in them, correct?

@littledan
Copy link
Member

@ljharb Exactly, thanks for the clarification. I fixed the wording as you suggested.

@bakkot
Copy link

bakkot commented Jan 3, 2018

@wycats To make sure we're on the same page, the proposal is not to disallow private static fields, but rather to allow them with the semantics that referring to this.#staticField within a static method invoked on a subclass would throw (even though this.#staticMethod() would not). Is that your interpretation, and if not is it still something you're on board with?

@wycats
Copy link

wycats commented Jan 3, 2018

Ah I didn't understand that, so thanks for the clarification.

I think, for me, the consequence of this decision is that I would avoid static private fields and teach people to avoid them.

Was there a strong argument for static private fields that made us want to give them these semantics rather than defer them?

Either way, I could live with this conclusion.

@gibson042
Copy link

static private methods and fields in the forms that have been discussed are highly problematic , essentially the straw that breaks the camels back.

I agree.

private instance methods are problematic because they violate the lexically-resolved/object-resolved distinction I described. They syntactically manifested as if they were object-resolved methods when they are really lexically-resolved functions. They start us on the slippery-slope of complexity and confusion.

I strongly agree.

there are no issues with "static public fields". They are just initialized data properties of a constructor.

I wouldn't say no issues, but everyone (including me) seems to be comfortable with normal Javascript prototype behavior (including copy-on-write semantics) if static private fields are taken off the table.

No issues with private and public instance "fields".

Strange situations are certainly possible, but I agree that they don't really feel like new issues.

class A { #private = "A"; getter() { return this.#private; } }
class B { #private = "B"; getter() { return this.#private; } }
var traitor = Object.setPrototypeOf(new A(), B.prototype);
A.prototype.getter.call(traitor); // => "A"
traitor.getter(); // => TypeError

So, I'd be relatively happy if we rolled out just public/private instance fields and won't have a problem if we also included public static "fields".

I think if we include class body lexical 'function` declaration we will have covered all the key class capabilities and won't have to do additional work (except for decorators) on class definitions for a long while.

I've been persuaded. It's extremely unfortunate that the class fields proposal introduces a new "private" concept that doesn't apply to any other class feature, but not technically fatal. The table would end up like this:

function/method field
instance ES2015

proposal-class-fields

static ES2015

proposal-static-class-features

private functionality "class body lexical declarations"

"class body lexical declarations", also proposal-class-fields #private one-off

@littledan
Copy link
Member

littledan commented Jan 9, 2018

@allenwb I agree with @bakkot's argument in #4 (comment) .

To clarify where we are in committee discussion: I brought the question about whether it's appropriate to pursue private methods given the lexically scoped function alternative possibility, to TC39 twice: in the July 2017 and September 2017 TC39 meetings. We discussed multiple alternatives, including function declarations in class bodies, private method syntax with lexical function declaration semantics, and private method syntax with private field semantics. The committee reached consensus in July on the third alternative. With this option, private methods reached Stage 3 in the September 2017 meeting. Later, in the November 2017 meeting, we confirmed that private instance methods are at Stage 3.

I believe Allen's arguments in favor of lexically scoped function declarations instead of private methods were made well-known to the committee. I tried to present them in my private methods presentations, but Allen had also made them directly to the committee in the past. So, I'm not sure what new information we have to revisit these decisions.

@gibson042 About using lexical scoping rather than # for private fields: see this FAQ entry for why we decided against that option.

@littledan
Copy link
Member

The difficulty I keep having with @allenwb 's lexical scoping proposal is that it seems odd to me to have a bare function declaration or let declaration in the middle of a class body. I was discussing this with @ljharb and he had an interesting idea for what the syntax could be:

class C {
    static function x(v) { return v+y }
    static let y = 1;
    method() { return x(1); }
}
new C().method();  // => 2

What do you think? We could use a different keyword instead of static if anyone has any other ideas.

@ajklein
Copy link

ajklein commented Jan 10, 2018

@littledan At first glance I like this direction! And we already have precedent to prefix declarations with a keyword, for module exports.

@ljharb
Copy link
Member

ljharb commented Jan 10, 2018

Note as well; my suggestion wouldn't necessarily preclude static #foo() {} or static #bar, it could complement it - the # syntax could be installed on subclasses, and the lexical ones wouldn't be. That might help accommodate the two mental models @allenwb's describing.

@allenwb
Copy link
Member

allenwb commented Jan 11, 2018

We could use a different keyword instead of static if anyone has any other ideas.

class C {
    class function x(v) { return v+y }
    class let y = 1;
    method() { return x(1); }
}
new C().method();  // => 2

It explicitly says: I am defining a class (scoped) function or variable. static doesn't seem quite right as the current usage of static basically means "associated with the constructor object" and that isn't what these definitions do.

The difficulty I keep having with @allenwb 's lexical scoping proposal is that it seems odd to me to have a bare function declaration or let declaration in the middle of a class body.

This seems symmetric with the odd feeling I get when I see what looks like an assignment statement (ie, a instance field declaration with an initializer) in the middle of a class body. I suspect we would both would get over those feelings after enough usage.

One of the arguments for the keywordless field definitions was that the keyword is unnecessary noise and that developers will be annoyed by the extra typing. The same argument seems applicable here.

But, I even with a redundant keyword, I think this would be much better than what has been kicked around for private methods and static private fields.

@allenwb
Copy link
Member

allenwb commented Jan 11, 2018

@ljharb

Note as well; my suggestion wouldn't necessarily preclude static #foo() {} or static #bar, it could complement it - the # syntax could be installed on subclasses, and the lexical ones wouldn't be.

Let me make a suggestion for a better way to think about static # inheritance. Unlike class instances, class constructors are singleton objects. Static fields (properties or private slots) are defined directly upon the constructor objects and constructor inheritance chains go directly from subclass constructor to superclass constructor without going through intermediate prototype objects. Constructor inheritance is pure prototypal inheritance.

There is one other context in JS where we declarative define single objects without also defining an associated prototype -- Object literals. To me, the semantics of static field definitions seem most like the semantics of object literals properties than the semantics of instance fields. So, when reasoning about what would be reasonable behavior for static # fields I always try to think about what I would expect if there was a way to defining #fields within object literals.

For example:

let obj1 = {
   #ctr: 0,
   get counter() {return this.#ctr},
   increment() {++this.#ctr}
};

let sub = {
   __proto__: obj1
};

console.log(obj1.counter); //0
sub.increment();
console.log(obj1.counter); //1

I really think we should steer away from private static fields (at least for now) but if you want think about them, think about object literal singletons.

@ljharb
Copy link
Member

ljharb commented Jan 11, 2018

The semantics you describe make sense to me, ftr.

@ljharb
Copy link
Member

ljharb commented Jan 11, 2018

Also i think class does make more sense as a keyword than static, for the lexical variations, because that’s the scope they have.

@wycats
Copy link

wycats commented Jan 11, 2018

@littledan I really love this direction.

One thing that's been gnawing at me this entire time (and which @allenwb has repeatedly pointed out) is that all static private features are easily modellable using normal closures. And since private static methods are lexical, they're really just a convoluted way to describe a lexically scoped function.

The only reason that isn't sufficient to make that the end of the story here is that we need the functions to be on the inside of the class body in order to have access to instance state defined inside the class body.

@allenwb is correct that allowing lexically scoped functions inside the class body addresses the use cases for static private fields with reasonable ergonomics and a clear programming model. The issue with putting a bare function inside the class body (for some of us, at least) is that adding bare functions to a class could confuse people about what the semantics of a class body are.

@littledan's proposal to use a prefix modifier like static or class eliminates that concern for me (at least at first glance), which means that the use-cases for static/private would be fully covered by class functions! We might eventually want static/private for some reason, and I wouldn't want to completely foreclose the possibility, but we would have a coherent system with the addition of static/public fields + instance/private fields/methods/accessors + class functions even if we never decide to do static/private fields/methods/accessors.

TL;DR I would love to see us decouple static/private stuff from this proposal and focus our efforts on class functions for those use cases for now.

@littledan
Copy link
Member

littledan commented Jan 11, 2018

Bikeshedding: One thing I liked about @ljharb's suggestion of static is that this semantically implies that the declaration is not related to the instance (even if it's technically a mismatch since it doesn't really have to do with the constructor either). class seems a little odd to me because:

  • We might want to have inner class declarations, which would be class class
  • It seems a little funny to repeat the outer keyword in an inner context, like, "JavaScript, I just told you that I'm inside a class; why do I have to keep saying class?"

Note that we're not restricted to existing keywords here and can use basically anything as a contextual keyword (as long as we're ok with the no-newline-after limitation).

@littledan
Copy link
Member

littledan commented Jan 11, 2018

Next, in JS we have basically two kinds1 of procedure calls baked deep into the call semantics: lexically-resolved calls and object-resolved calls. As should be obvious from the names I choose, the primarily difference is how the actual invoked function is determined. But the two different call forms also have an impact on how programmers conceptualize and use procedural decomposition.

I'm not sure I agree entirely with this phrasing of the problem space. I think there are two reasons programmers may put code in a method:

  • In order to dispatch on the receiver ("ad-hoc polymorphism")
  • For namespacing, chaining, and other more "superficial" aspects of methods.

The private instance methods proposal opts to use method syntax, rather than lexically scoped function syntax, to retain consistency and parallelism for the second case. Where a method uses this (even if many JS programmers find this to be a mis-feature), private method syntax facilitates easy refactoring between public and private ways to factor out behavior. These benefits are even despite the fact that there is only ever one value for the method, so it can't be said to really "dispatch".

I really think we should steer away from private static fields (at least for now) but if you want think about them, think about object literal singletons.

This seems like a really useful lens; thanks for bringing it up. The original proposal for class fields would throw a TypeError in both cases and be consistent with respect to subclassing. (I actually wrote out a sketch of what object literal semantics would be in an attempt to create consistency here; however, I went back and removed it because it seemed too limiting to have object private fields scoped only to a single literal.)

However, the current draft spec, due to inheriting private static methods and accessors, loses this property: Private static methods are inherited (copied) by extends, but not by prototype chains of instances. We could try to recreate this property with changes to Object.create, __proto__ in literals, etc, but this could get very complicated.

@littledan
Copy link
Member

@bakkot and @rbuckton both separately brought up another possible syntax: using a static { } block in a class, with declarations inside that block having the semantics that they are available in the whole class body ("hoisted"):

class C {
    static {
        function x(v) { return v+y }
        let y = 1;
    }
    method() { return x(1); }
}
new C().method();  // => 2

What do you think of this? (emoji reacts welcome)

Advantages:

  • Being in a separate block provides a sort of syntactic break that makes it more clear that these are not method or field declarations. In general, classes have method/field declarations at the top level, and statement lists within the next level of square brackets/after the = tokens.
  • Very small surface area, which could make it easy to learn.
  • Static blocks are more expressive in general. It can include code which is not a declaration.

Disadvantages:

  • This breaks the property that curly braces always create a new lexical scope. These particular curly braces would continue to be in the enclosing scope, and just indicate a mode switch (from class element declarations to ordinary statement lists).
  • Because classes can be expressions, this is a way to include statements in expression context, meaning that we need to solve many of the open issues with do expressions to determine semantics (namely the semantics of abrupt completions like break, and the scoping of var). Not really a disadvantage but a larger design problem.

@jridgewell
Copy link
Member

This breaks the property that curly braces always create a new lexical scope

I think this is a pretty large con. Being able to reference declarations that are defined inside an inner curly brace feels really bad.

@bakkot
Copy link

bakkot commented Jan 11, 2018

A bit behind here...

@allenwb

Is this conclusion data based?

Not precisely. As you know, it's quite hard to get good data here, which is why we lean on educators and other people with exposure to broad parts of the community. On this particular question, though, I think every single person I've ever talked to about it had the expectation that they'd be per-instance. Here's an example from the private fields discussion, which a lot of people apparently felt was reasonable despite there being no way it could possibly work without fundamentally changing what closures are; later someone says they tried using lexical declarations in class bodies in exactly this way (because an older JS pattern) and were disappointed when it didn't work.

It seems to me that much of the difficulty we've had with extending class declaration functionality has been dealing with expectations that programmers (and us) may carry over from other languages.

I actually don't think this is attributable to experience in other languages. One reason is that there's been a JavaScript pattern taught for many years which suggests using variable declarations in constructors to create (per instance) "private members": here's Crockford giving it, for example. And yes, that's not the situation here because the methods in a class body are shared, rather than being created every time the constructor runs, but especially given the examples I've linked I'm not convinced the instinctive mental model of lay programmers makes that distinction.

private instance methods are problematic because they violate the lexically-resolved/object-resolved distinction I described. They syntactically manifested as if they were object-resolved methods when they are really lexically-resolved functions. They start us on the slippery-slope of complexity and confusion.

I'm not sold on this particular claim. Because of the type check, the semantics are equivalent to "define immutable field on objects which are instances of this class, with a name which no other objects can share or reference", and not equivalent to "define a lexically-resolved function".

I also agree with @littledan's reading of how people tend to think about functions vs methods. As an example, if I have an object foo which is an array, and I call foo.map(something), I am 100% of the time expecting to this to be Array.prototype.map. This isn't the case for all methods - sometimes I really am intending to do dynamic dispatch - but it is for some of them.

class function

This would be confusing for me personally, but I think I like it better than static. local or inner would also do (as in "inner class"). Though I'm also ok with bare function declarations, just not bare let/const declarations.

I think of these my preference is for inner-prefixed general lexical declarations or bare function declarations.

@littledan
Copy link
Member

littledan commented Jan 12, 2018

OK, it seems like we have the shape of a possible solution here:

  • Static private methods and fields should be removed, and replaced with @allenwb 's lexical declarations in class bodies, with @ljharb 's suggested syntax tweak that these declarations be preceded by a keyword.
  • It seems likely that we can bikeshed a good keyword here, though the two keywords which have been discussed the most (class and static) have certain disadvantages.
  • Public static fields can proceed as originally proposed--own properties of the constructor where they are declared, with no particular behavior in subclassing.
  • The Stage 3 private instance methods proposal has come up on this thread as something which might be subsumed by lexically scoped functions in class bodies, but I believe they are still justified for the reasons I tried to present in the July 2017 TC39 meeting: private methods and accessors provide good ergonomics for behavior in classes that has to do with instances, providing easy refactoring from public instance methods and features like this and super property access.

The next steps from here would be:

  • Update the explainer to explain the above solution in some more detail with examples.
  • Write draft spec text for lexically scoped declarations, using a placeholder for the actual token.
  • Open another bug to discuss what the token should be. (EDIT: see Which token is best for lexical declarations in a class declaration? #9)
  • Present the new version of the proposal at TC39 in January as a Stage 2 update, asking for Stage 3 reviewers.

Although the above proposal is significantly different in semantics, it's an iteration on the same problem space, so I don't think it requires the process overhead of a new staged proposal.

@littledan
Copy link
Member

I'm developing an alternative explainer and spec text based on lexical declarations in #10 .

@wycats
Copy link

wycats commented Jan 15, 2018

@littledan I agree with your analysis, but especially with the point that private instance methods are still justified.

@littledan
Copy link
Member

Draft spec text for adding lexically scoped declarations; see discussion of semantics in the commit message. Better explainer text coming soon. Any feedback would be very appreciated. 4a2d164

@littledan
Copy link
Member

We have concluded that private static will be part of this proposal, and it has advanced to Stage 3.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests