Scoped Class Extensions

[eernstg]

In response to #40, this is a proposal for a mechanism, scoped class extensions, that allows class type hierarchies to be extended with new instance methods. This proposal is intended to be compatible with the scoped static extension method mechanism (#41), in the sense that those two proposals can be combined into a single one, or they can be adopted one after the other. Scoped static extension methods can do things that are not possible with scoped class extensions, and vice versa, so it makes sense to have both.

[Edit Jan 22 2019: Generalized this mechanism to apply to all sets of target classes whose subtyping is a tree, rather than only a set of targets which are related by the subclass relation. Changed eval to be a method, based on bitter complaints about using the name eval for a getter. But they are right, that is a bad name for a getter. ;-]

[Edit Mar 26 2019: Adjusted the description of generic extensions, the constraints previously mentioned are more strict than they have to be.]

[Edit Apr 1 2019: Clarified that an implements relationship must hold from case to case.]

Motivation

Like #41, this is a proposal for a mechanism that allows developers to add new methods to existing receivers without editing the corresponding existing declarations.

The special advantage of this proposal is the ability to enhance an existing class type hierarchy with new methods that are subject to object-oriented dispatch. In other words, whereas scoped static extension methods are similar to static methods, this proposal enables something which is similar to adding instance methods to the target class type hierarchy, without editing it.

The visitor design pattern is a well-known software engineering idiom for which a main selling point is that it allows developers to, sort of, add a new instance method to an existing class hierarchy. This is considerably less convenient than a real instance method, however:

• With a visitor, the target hierarchy (that is, the classes whose instances we wish to visit) must implement special support for double dispatch. That is, if we wish to visit instances of a class C with a visitor of type Visitor, then the class C must declare an accept(Visitor v) method that invokes the method in the visitor that corresponds to C.
• With a visitor, extra ceremony must be applied in order to specify a behavior which is similar to object-oriented overriding (e.g., every visit method must have a default implementation that makes an emulated "superinvocation", and care must be taken for repeated overrides).
• With a visitor, it is necessary for clients to use a different syntax in order to "invoke the added method": (they typically need to do myVisitor.accept(myReceiver)), and with a given visitor type, it is impossible to directly specify different signatures, so the approach taken could often be to make the visitor generic and use that to specify the return type, and then to store arguments to the invocation in the visitor itself, etc.
• Finally, it is necessary to edit the visitor itself in order to broaden the support such that the visitor can be used on any new kinds of objects that weren't taken into account (and maybe weren't even written) when the visitor was written.

In other words, we already have a design pattern that is well-known for being able to "add a new method" to a given class type hierarchy, but it is quite inconvenient to use.

This proposal offers a more smooth mechanism: The target hierarchy need not be edited at all in order to allow for adding an extension instance method (so there's nothing like an accept method); invocation of the extension instance methods (each one corresponding to a visitor) uses the same syntax as invocation of ordinary instance methods; extension instance methods can receive arguments and return results of any expressible type, using the familiar syntax and semantics of instance member declarations; and inheritance and overriding work in the same way for extension instance methods as it does for ordinary instance methods.

We will discuss the non-generic case first, but scoped class extensions allow for extending generic classes and capturing the actual value of the type arguments of the receiver.

Example

Here is an example of a target class type hierarchy that we will later extend with some extension instance methods:

// This is 'expr.dart'.

abstract class Expr {}

class Literal extends Expr {
  final int value;
  Literal(this.value);
  toString() => "$value";
}

class Sum implements Expr {
  final Expr leftOperand, rightOperand;
  Sum(this.leftOperand, this.rightOperand);
  toString() => "($leftOperand + $rightOperand)";
}

This is a Dart version of the standard example for the topic area known as the expression problem, using toString() to express a pretty-printing method just because that's a rather natural choice for Dart.

Now we want to extend that class hierarchy with an evaluation method, eval(). This is a method, but the treatment of getters, operators, etc. follows naturally from that. Here is such an example extension:

// This is 'eval-extension.dart'.

import 'expr.dart';

extension Eval on Expr {
  int eval() {
    if (this is EvalThirdParty) return this.evalThirdParty();
    throw "Unsupported subtype";
  }
}
on Literal {
  eval() => value;
}
on Sum {
  eval() => leftOperand.eval() + rightOperand.eval();
}

abstract class EvalThirdParty implements Expr {
  int evalThirdParty();
}

In general, a given class type hierarchy (like {Expr, Literal, Sum}) can be extended by a sequence of extension blocks, so the declaration of the extension above is one declaration. This ensures that the static class extension mechanism can be subject to separate compilation. We use the phrase 'an extension' to refer to the entity corresponding to each of these syntactic extension blocks, just like we'd use the phrase 'a class' to refer to the entity declared by a class declaration.

The class EvalThirdParty is not necessary, but it is used to illustrate how we can add support for "third party" classes that are not known by the developer who writes the extension. It is possible to import a scoped class extension that covers a set of "standard" classes, and then you can write your own additional ("third party") classes implementing or extending some of the standard classes, and you can write them in such a way that they support the extension. This would not be possible with a visitor: You would need to edit the visitor in order to make it visit a larger set of types.

In order to run an extension method, an invocation of the form e.m(arguments) is checked and compiled as follows:

• The static type of e does not have a member m (statically known regular instance members always win over extension instance members).
• A scoped class extension Ext targeting the static type of e is in scope.
• Ext declares a member m; it is a compile-time error if the arguments passed to m do not conform to the declaration Ext.m, and otherwise we have now decided that this is an invocation of m on the extension Ext.
• At run time, e is evaluated, let o be the resulting value, then the dynamic type of o is used to look up the corresponding extension object oExt, and the invocation is performed as oExt.m(o, arguments).

The notion of corresponding extension objects is crucial; it is explained below in a separate section.

For now, we just consider the simple case illustrated by the Eval example above. So here is a snippet of code where that extension is being used:

import 'expr.dart';
import 'eval-extension.dart';

// The `Eval` extension doesn't know about `Subtraction`, but we can still
// add a new receiver type to that extension: We just need to implement
// `EvalThirdParty`, which means that we need to implement an `evalThirdParty()`
// method.
class Subtraction implements EvalThirdParty {
  final Expr leftOperand, rightOperand;
  Subtraction(this.leftOperand, this.rightOperand);
  prettyPrint() => "($leftOperand - $rightOperand)";
  evalThirdParty() => leftOperand.eval() - rightOperand.eval();
}

main() {
  Expr e = Sum(Literal(3), Literal(4));
  print(e); // Prints '(3 + 4)'.
  print(e.eval()); // Prints '7'.
  Expr e2 = Subtraction(Literal(8), e);
  print(e2); // Prints '(8 - (3 + 4))'.
  print(e2.eval()); // Prints '1'.
}

This illustrates that we can add a new class to the set of target classes of the extension Eval (here: Subtraction), even though that new class is not in scope at the declaration of the extension, so the developer who wrote Eval had no way of knowing about that new class. We can mix and match the "standard" class instances with the "added" class instances, because they are all of type Expr, and evalThirdParty() will transparently be invoked via the extension method eval when Eval does not have an implementation.

With that example in place, here's a "desugared" version of the code, which shows how it works. Note that the extension and call sites get desugared, but expr.dart remains unchanged (which is true in general, because we do not change a class hierarchy in any way when we extend it). Some comments were added to the desugared code, in order to explain what is going on.

// Desugared version of 'eval-extension.dart'.

import 'expr.dart';

// ----------------------------------------------------------------------
// Code corresponding to the scoped class extension `Eval`.

// For each block in the scoped extension declaration, implicitly generate
// a class which will be used to hold the desugared extension methods.

class Eval_Expr {
  // Generated code to support the mechanism.
  const Eval_Expr();
  static Eval_Expr extensionObject(Expr e) {
    // Handle types that are addressed directly.
    var result = eval_extensionObject[e];
    if (result != null) return result;
    // Other types are resolved according to the
    // declaration order: Last match wins.
    if (e is Sum) return eval_Sum;
    if (e is Literal) return eval_Literal;
    return eval_Expr;
  }

  // Transformed user-code.
  int eval(covariant Expr _this) {
    if (_this is EvalThirdParty) return _this.evalThirdParty();
    throw "Unsupported subtype";
  }
}

class Eval_Literal extends Eval_Expr {
  const Eval_Literal();
  static Eval_Literal extensionObject(Literal e) =>
      Eval_Expr.extensionObject(e);
  eval(Literal _this) => _this.value;
}

class Eval_Sum extends Eval_Expr {
  const Eval_Sum();
  static Eval_Sum extensionObject(Sum e) => Eval_Expr.extensionObject(e);
  eval(Sum _this) =>
      Eval_Expr.extensionObject(_this.leftOperand).eval(_this.leftOperand) +
      Eval_Expr.extensionObject(_this.rightOperand).eval(_this.rightOperand);
}

// Implicitly generate a dispatch mapping from types to extension objects:
// For every concrete class `G` known at the declaration of the extension which
// is a subtype of the first target (here: `Expr`) add a mapping from `G`
// to the corresponding extension object to this map.
const Map<Type, Eval_Expr> eval_extensionObject = {
  Literal: eval_Literal,
  Sum: eval_Sum,
};

// Implicitly generate constant extension objects.
const eval_Expr = const Eval_Expr();
const eval_Literal = const Eval_Literal();
const eval_Sum = const Eval_Sum();

// No desugaring needed for `EvalThirdParty`.
abstract class EvalThirdParty implements Expr {
  int evalThirdParty();
}

One thing to note is that there is a static method extensionObject in each class which is the desugaring of an an extension block in the original scoped class extension declaration. These static methods all do the same, but the ones that are associated with extensions targeting a subclass have a more specific argument type and return type. This is needed in order to allow the call site on an instance of, say, Literal to statically know that all extension methods declared for or inherited by the Literal target are available, and not just the ones which are declared for the target Expr. The associated downcasts are safe (so a compiler can omit them), because of the design of the mapping eval_extensionObject.

The mapping eval_extensionObject is a "dispatch map" which maps every receiver type to the corresponding extension object. It is described in the next section how to create it.

The main implementation of the static methods extensionObject is the one in the first extension block (here: the one for Expr), and it uses the mapping eval_extensionObject to dispatch directly to the extension object for each known target class.

If the given receiver o is not a direct instance of any of these classes, we check, in reverse order, whether o is an instance of each target type (using is, that is, allowing for proper subtypes). This means that if a third party class implements Sum or Literal, an instance thereof just get the implementation which is written for Sum respectively Literal (and that would presumably work, because said class actually promises to work like a Sum respectively a Literal); in the ambiguous case where the third party class implements both Sum and Literal, the developer who wrote the extension made the choice to put Sum after Literal, and this is used to disambiguate: Sum wins because it is "more specific".

Finally, if the dynamic receiver type does not implement anything more specific than Expr, we may choose to say that there is no meaningful implementation of eval() for such an object; but in this case we can actually push the task back onto the receiver by means of the EvalThirdParty supertype: Whoever implements EvalThirdParty has made a commitment to support this extension by implementing some instance methods. This fits with the situation where the receiver class C was actually written by a "third party", and the writer of the extension has no idea that C exists.

Here is the desugared version of the main library:

import 'expr.dart';
import 'eval-extension.dart';

// Only extension method invocations need desugaring here.
class Subtraction implements EvalThirdParty {
  final Expr leftOperand, rightOperand;
  Subtraction(this.leftOperand, this.rightOperand);
  toString() => "($leftOperand - $rightOperand)";
  // Being lazy, we evaluate `this.leftOperand` twice; real desugaring
  // will use local variables to ensure that such expressions are only
  // evaluated once; same for `rightOperand`.
  get evalThirdParty =>
      Eval_Expr.extensionObject(this.leftOperand).eval(this.leftOperand) -
      Eval_Expr.extensionObject(this.rightOperand).eval(this.rightOperand);
}

main() {
  Expr e = Sum(Literal(3), Literal(4));
  print(e);
  print(Eval_Expr.extensionObject(e).eval(e));
  Expr e2 = Subtraction(Literal(8), e);
  print(e2);
  print(Eval_Expr.extensionObject(e2).eval(e2));
}

This example illustrates the core ideas: A target class type hierarchy is supplemented by an extension class hierarchy, and extension instance method invocation proceeds in two steps: (1) compute the receiver o and find the corresponding extension object oExt; (2) invoke the extension method as oExt.m(o, ...).

Corresponding Extension Objects

A scoped class extension can be declared for a set of types whose subtype relation is a tree, and it introduces an ordering on this type hierarchy which is not a contradiction of the subtype relationship. This means a few things:

It is a compile-time error to declare a scoped class extension with a target which is dynamic or void. It is a compile-time error to declare a scoped class extension whose initial target is a type T, if a subsequent target D is not a subtype of T. It is a compile-time error for a scoped class extension to have two targets D1 and D2 in that order (but not necessarily consecutively) if D1 <: D2.

Finally, assume that a scoped class extension with initial target class C and subsequent targets D1 and D2 and D3 (where D1 can be C, but D1, D2 and D3 are distinct) is such that D3 <: D1 and D3 <: D1; it is then a compile-time error if D2 <: D1 does not hold. (This ensures that the subtype relationships among all targets is a tree.)

In other words, a scoped class extension must have a list of targets which is a topologically sorted enumeration of a subset of the subtypes of the first target, and the set of target types must be a tree according to the subtype order.

The point is that this makes it easy to see that we can create a shadow hierarchy of extension classes corresponding to the given hierarchy of target classes; this would not be so straightforward if we had allowed the subtyping structure on the targets to be a general directed acyclic graph, rather than a tree.

So, during desugaring we will create a class for each extension target, and the inheritance structure among the extension classes is a "coarsened" version of the inheritance structure among the target classes.

The approach taken is: For each extension class Ce with target Ct, let S be the minimal proper supertype of Ce among all targets of the extension, and let Ce2 be the corresponding extension class (S is guaranteed to exist because the target types is a tree.). Then Ce2 is the superclass of Ce.

Moreover, each extension class after the first one implements the one that is associated with the previous case, except when that previous case is already its direct superclass.

This approach guarantees that whenever a target has static type T and dynamically matches a target S, the corresponding extension types are such that when Ce corresponds to T and Ce2 corresponds to S, it is guaranteed that Ce2 <: Ce. This ensures that if static analysis predicts that a receiver can have a method m invoked on an instance of Ce, such an invocation will also be possible on an instance of Ce2, and the usual override rules ensure that the invocation has the same soundness guarantees as we have for ordinary instance method invocations. For instance, parameter passing is guaranteed to be statically safe, except when the invoked method has one or more parameters which are covariant.

On Static Type Safety

Scoped class extensions allow for adding a new instance method to some or all classes in a class hierarchy (that is, a set of classes where each pair has a subclass relationship to each other, direct or indirect). It does not require the target classes to be modified in any way in order to allow this.

Consequently, it needs to have some treatment of the case where the dynamic type of the receiver is a subtype of the initial target class (so in the example it is a subtype of Expr), but not a subtype of any of the types (Literal and Sum) for which there is an implementation.

We could make this a compile-time error (so if you only know that e is an Expr, you cannot call its eval, you have to know statically that it's a Literal or a Sum). This would allow the initial target (and in desugared code: the class Eval_Expr) to be abstract, and the developer could declare eval as abstract. This would be safe, but quite inconvenient.

So we have chosen to say that no extension blocks are abstract, and every extension block hence needs to implement every method that it supports. In some cases that's possible; In fact, it isn't worse than it would be to write a static scoped extension method, because such a method always relies on the statically known receiver type.

However, just like eval on a receiver which is an Expr and not a Literal nor a Sum, there will be cases where it is just not possible to come up with a reasonable implementation.

The crucial point here is that we are actually defining a method, with implementations, for all subtypes of the initial target (here: Expr), not just for a subclass hierarchy, and there is no way we can get this kind of concept without a certain trade-off. The trade-off is that it may be necessary for an extension method like eval in the initial extension block of Eval to throw at some point: We just don't know what to do for this particular receiver. The other side of this trade-off is that it is a more powerful concept than an ordinary instance method to cover all subtypes.

However, the use of EvalThirdParty in the example shows that it is actually quite easy to come up with a programming idiom that allows all those third parties to write their classes in such a way that they will be supported by a given extension like Eval: Just implement EvalThirdParty.

Generics

When one or more of the target classes is generic, a scoped class extension can use a type pattern (#170) to declare type parameters for the extension, which provides access to the actual type arguments of the run-time type of the receiver, in the body of the extension.

These type patterns can be irrefutable, which means that they are guaranteed to match for any given instance of the underlying type.

*For instance, List<var X> is an irrefutable type pattern, because every instance of type List<T> for any T will match that pattern. Similarly, C<var X extends num> is an irrefutable type pattern in the case where C declares a type parameter with bound num.

// Target class declaration.
class A<X, Y extends int, Z extends B<Y>> {...}

// Examples of irrefutable type patterns for `A`.
A<var X, var Y extends int, var Z extends B<Y>>
A<var U1, var U2, var U3> // Renaming and omission of bounds is OK.
A // Using a raw type means "I don't care about the actual type arguments".

The first pattern may be the most useful one, because it allows the extension to get access to the dynamic value of all the type arguments of the target, and it equips each of them with the best possible bound. However, the last one may yield better performance at run time (because there is no need to perform matching and binding any type parameters to a value, it's just a plain subtype test).

However, we may also use refutable type patterns (that is, patterns which are not irrefutable). In this case we require that the greatest closure of the type patterns (that is, the transformation which erases var X to Object and var X extends B to B) form a tree with respect to the subtype relation.

Here is an example:

// Greatest closure is a topologically sorted traversal of a subtype tree: OK.
extension E1 on List<var X extends Object> {...}
on List<var X extends num> {...}
on List<int> {...}

It is worth noting that is allowed for multiple cases to match the same type, based on the type pattern. So there is nothing wrong with having a case for List<int> which is applied if we actually have a List<int>, and also a case for List<var X extends num> which will be applied for an instance of List<Null>, List<double>, or List<num>. There is no ambiguity for the List<int> because the later case is considered more specific, so the case List<int> gets to work with the instance of List<int>.

At run time, matching proceeds from the most specific end (the one at the end, textually), and upwards through less and less specific type patterns, until there is one that matches. For instance, a List<String> gets dispatched to the first extension (the one with List<var X extends Object>), because the others do not match.

It does matter whether a scoped class extension uses irrefutable type patterns or not, because the ones that have one or more refutable type patterns are likely to require a compilation strategy which is more costly (in terms of run-time performance). So we'd expect irrefutably type patterns to be the most common approach, and refutable ones are used when we want something more fancy, and are willing to pay for it.

In particular, every scoped class extension which has no type parameters at all will have irrefutable type patterns.

As an example of a case that admits optimal performance, any scoped class extension which has irrefutable type patterns for a tree-shaped subset of a subclass hierarchy allows for a direct mapping from each receiver type to the corresponding extension object. So, whenever the receiver is an instance of one of the classes in that hierarchy (that is, the ones for which we have a case, plus the intermediate ones that we have no case for, but which are in scope at the declaration of the extension), we can directly map from the receiver type to the extension object, like a vtable. Let's say that the classes in this "extension vtable" are called well-known classes. For classes that are not well-known, e.g., a third-party class that implements a well-known class, we still have to perform a linear search in order to find the most specific case of the extension that matches, but we are likely to get the fast (vtable-ish) dispatch in the majority of cases.

Note that it is important that we can determine at compile-time that a given type pattern will match the actual receiver:

extension E on List<Object> { Object foo() {...}}
on List<var X extends num> { num foo() {...}}

main() {
  List<int> xs = [1];
  num n = xs.foo(); // We get the type from the second case of the extension.
}

This is a sound typing because a receiver of static type List<int> is guaranteed to match the type pattern List<var X extends num> at run time.

Here is another example:

abstract class Expr<X extends num> {}

class Literal<X extends num> extends Expr<X> {
  final X value;
  Literal(this.value);
  toString() => "$value";
}

class Sum<X extends num> extends Expr<X> {
  final Expr leftOperand, rightOperand;
  Sum(this.leftOperand, this.rightOperand);
  toString() => "($leftOperand + $rightOperand)";
}

// Extension.

extension Eval on Expr<var X extends num> {
  int eval() { ... } // Nothing new about this.
  bool get isEven {
    X x = eval();
    if (x is int) return x.isEven;
    return false;
  }
}
on Literal<var X extends num> {
  eval() => value;
  int intValue() => value is int ? value : value.round();
}
on Sum<var X extends num> {
  eval() => leftOperand.eval() + rightOperand.eval();
  List<X> get operands => <X>[leftOperand.eval(), rightOperand.eval()];
}

The semantics of this kind of extension could be specified in terms of generic extension objects, or it could use non-generic extension objects.

With generic extension objects, an instance of Sum<double> would have an instance of Eval_Sum<double> as its corresponding extension object, and an instance of Literal<num> would have an instance of Eval_Literal<num>. This means that it would be necessary for the implementation to deliver these instances of generic classes upon dispatch on an instance of a target class (because it is not known statically which actual type arguments we will have ... except of course when it can only be num, int, double, and Null, but it is true in general ;-). With compiler support, it may be an operation with rather good performance to create these instances (of Eval_...<T> for some T), because they have no state, and the type argument list may be shared.

With non-generic extension methods, we would pass all the type arguments that are made accessible to the body of the extension as (additional) type arguments to the extension method itself. This approach is suitable for static extension methods because it allows us to avoid allocating "an extension object" at all, but with a scoped class extension the extension object is crucial, and it may or may not be a good implementation strategy to make it generic.

The reason why we can use both of these strategies is that there is no state in an extension object, and user-written code can never get explicit access to an extension object. So there is no way for user-written code to detect whether any given mechanism that offers access to these type variables is based on method type parameters or on class type parameters.

[leafpetersen]

Just capturing some offline discussion: it's possible that this can be treated as just a use of static extension methods. This provides either an alternative implementation/de-sugaring mechanism, or possibly an argument against adding a separate mechanism. For the example above, the de-sugared code would look something like the following (where static extension is sample syntax for the feature described in #41):

// This is 'eval-extension.dart'.

import 'expr.dart';

static extension Eval on Expr {
  int get eval {
    if (this is Sum) return this.eval; // Dispatches to the Sum specialization
    if (this is Literal) return this.eval; // Dispatches to the Literal specialization
    if (this is EvalThirdParty) return this.evalThirdParty;
    throw "Unsupported subtype";
  }
}

static extension EvalLiteral on Literal 
  get eval => value;
}
static extension EvalSum on Sum {
  get eval => leftOperand.eval + rightOperand.eval;
}

abstract class EvalThirdParty implements Expr {
  int get evalThirdParty;
}

It's plausible that we could add some syntax to allow the dispatch method in Expr to be generated automatically, making this feature a sub-feature of static extension methods.

[eernstg]

@leafpetersen wrote

the de-sugared code would look something like the following

The implied desugaring would need to have some extra elements in order to get a similar semantics. For instance, the approach that I describe needs to have one implementation of the dispatch. That's different with the desugaring that you mention.

Dispatch at all receiver types: If we have a receiver with a static type which is not the top of the hierarchy which is the target of the extension (here: Expr) then we won't get any dispatching (say, the static type could be Literal and the dynamic type could be some subclass SpecialLiteral which has its own implementation of the target method), we would just call the implementation for the statically known type. So, in general, there must be a dispatching method for each extension which can be the static type of the receiver, except of course in the case where said dispatch would only have one implementation to choose among.

Dispatch at all methods: We would need dispatching code for each method—the one which is shown is only for eval.

So if we don't want to have any support for something like scoped class extensions, and just tell developers to use a specific design pattern like the one you mention, then they'd have to write many dispatch methods. They wouldn't be identical (so we can't just write one and forward to that):

Different dispatch signatures: If we wish to emulate a set of method implementations where there is a non-trivial overriding relationship then we need to declare the dispatch methods with the corresponding signatures. For instance, an extension method m for a class A could accept one argument, and an override of m for B could accept that plus some optionals. This means that the dispatch method for m for A would also accept one argument, and the one for B would accept that plus those optionals. So we couldn't just let the one for B be a forwarder to the one for A. This would imply that we can't just have one dispatch implementation and let all the others forward to it, we'd need to write a lot of repetitive code (and get the dispatch ordering and everything correct every time).

Different names: So we'd need to use different names, the dispatchers would have the "official" name (such as eval) and all the implementations would have a different name (maybe eval_impl), because some extensions need to have both. It may or may not be possible to use the official name (eval) for the implementation methods in the case where there is no dispatch (because there is only one method implementation for all subtypes of the current target type).

But I think we would have to solve some other problems as well:

Overriding relations: If each extension is an island that adds the behavior needed for a specific target class, how would we know whether any particular method signature is a correct override? It may be possible for a developer to see in the dispatching methods that "this is an emulation of a scoped class extension on the classes A, B, ..., K" then we may inspect those classes and verify that some of them are subclasses of others, but I can't see how a compiler would be able to tell that "you have declared m in this extension, and this declaration of m is intended to override that one in that other extension, but they don't have a correct override relationship". Presumably, we would get some compile-time errors in the dispatching code because we are trying to call a method (B.m) from another one (a dispatcher for A.m), but that wouldn't be a particularly developer-friendly way to say "incorrect override of m for B", and we would probably miss out on problems like void m(int i, int j) is not a correct override of void m(int i, [int j]), because the dispatcher would pass both arguments.

Another overriding relationship that would be difficult to support is variation on types: If we have an A.m(int i) and a contravariant override B.m(num n) then the dispatcher for A cannot be used for B, because myB.m(3.0) would hit a dynamic check for int if we just treat this like an invocation of A.m.

Disambiguation: The scoped class extensions mechanism is designed to enforce that each extension extends a class hierarchy (ok, any tree structure would do, so we can relax that if we want), and the declaration order is used to enforce and enable a deterministic and explicit choice by developers: If the receiver turns out to be a subtype of both Literal and Sum, we invoke the implementation for Sum, because the developer made the choice to put it later in the list than Literal. With a set of (in principle unordered) static extensions, how would we obtain a well-defined disambiguation in such cases? I think it would be an implicit property of all those dispatch methods: They make the choice to call the Sum implementation and would only test for Literal after having rejected Sum. It would be necessary for developers to maintain a consistent ordering of all the cases in the dispatch methods. If they disagree then we'll get funny disambiguation when methods are invoked on receivers that implement multiple of the target classes, which might be somewhat tricky to debug.

Grouping: I suspect that it will be hugely helpful to use a mechanism whereby the target set of classes is explicitly enumerated and prioritized. But if we do that then we are certainly beyond an approach where we just say "here's a design pattern using static extensions, use that".

Superinvocations: Even with a notion of a group ("this is the set of target classes"), it is not obvious to me that the emulation of scoped class extensions using static extensions will establish a well-defined notion of a subclass relationship. Basically, it is implied by the dispatching code: If we get to run the code for B.m if possible, and only consider A.m if B.m can't be used, then we have effectively specified that B methods override A methods. This is rather implicit, however. With my proposal about scoped class extensions it is guaranteed that the extended hierarchy is a refinement (considered as a partial order) of the hierarchy of extension classes (that is, the desugaring of each block in the extension declaration). This means that we have a simple guarantee: If the receiver is a T respectively an U where U <: T then the corresponding extension object is an instance of Tx respectively Ux and Ux <: Tx. So extension methods are guaranteed to be organized in a way that respects subtyping for receivers.

This means that, in a scoped class extension, when we execute a method m that is added to B and we invoke super.m, then it is simply desugared to a superinvocation which will call the implementation of m on the extension object for a superclass of B. The dynamic semantics and the static checks can be derived directly from the fact that a super invocation in an extension is actually a superinvocation in the usual sense. We could of course define a superinvocation in an extension to mean something else, but I happen to think that developers will be quite comfortable if it means essentially the same thing as usual.

Tear-offs: With scoped class extensions, an extension method is simply an instance method in the class which is the desugaring of one block in the extension declaration, i.e., an instance method of the extension object. So tear-offs need special treatment, but not much: A normal instance method m taking parameters corresponding to arguments can be torn off a receiver object o as follows: (parameters) => o.m(arguments), with just small adjustments (the function literal has no explicit return type, but we must use the one from the declaration of m; and the resulting function object must have an implementation of == that takes the selector m and the receiver o into account). A user-written function can't do this, but it is not difficult for a compiler to generate code for the creation of such a function object. With an extension method we have the extension object xo and the receiver o, and the tear-off would then yield (parameters) => xo.m(o, arguments), with the same small adjustments. I suspect that it would take some extra effort (and introduce some extra bugs) if a design pattern using static extensions were to be extended such that tear-offs will work nicely.

Performance: There are lots of potential performance optimizations with scoped class extensions. In particular, when one extension method calls another method of the same extension on the same receiver, it's just a plain method invocation on the extension object (no added dispatch at all). If instances of a given class has many methods of the same extension executed over time, it might me helpful to cache the extension object at the class (it is always the correct extension object for all direct instances of that class, so we only need to run the dispatch code once). This is also applicable to "tricky cases" like a third party class that implements both Literal and Sum, etc.

My desugaring had a static extensionObject method for each extension class, and all but one is a simple forwarder of the form Eval_T1 extensionObject(T1 o) => Eval_Expr.extensionObject(o);. We can give the extensionObject method a special type which reflects the fact that it will always deliver an instance of Eval_T for an argument of the corresponding type T, because that method is always generated and the compiler can generate code that satisfies this. This is a special kind of type of the form f(T) Function(T) where f is the type-level function that maps Expr to Eval_Expr, Literal to Eval_Literal, etc, and there's no way we would add such a type to the Dart type system in general, but it can be introduced in order to capture the actual guarantees provided by extensionObject. With generics we actually have an even stronger type: The extension object for T<exactly U1, exactly U2, ...> is Eval_T<exactly U1, exactly U2, ...>.

Similarly, the receiver argument is covariant (eval(covariant Expr e)), but extension methods are never receiving the first argument from user code, it is always provided by the compiler in an expression of the form MyExtensionClass.extensionObject(v).m(v, ...), and we already gave extensionObject a special type that allows the compiler to conclude that it is safe to pass v as the first actual argument to m.

In other words, I really don't think there is a very easy design pattern using static extensions that we can ask people to use, and then they wouldn't need anything like scoped class extensions. I think they need a specific and explicit mechanism which will get all these things right, if we want the ability to extend class hierarchies with new instance methods, in any reasonable sense.

If we do add such a mechanism then I suspect that we will be able to get a rather straightforward and comprehensible semantics, and potentially a nice set of optimizations, by having something which is similar to scoped class extensions as proposed. Or, of course, we could replace the syntax I mentioned by all kinds of other syntaxes.

But I do think that it will be helpful to have the crucial properties:

• The mechanism specifies a group of classes organized as a subtype tree, and declares extensions for them which are organized in a similar tree (maybe the target has more nodes, but it must be a refinement of the extension hierarchy).

• The extension hierarchy supports all the normal instance member concepts, such as overriding, correct overrides (including well-defined support for parameters that are covariant by declaration), superinvocations, tear-offs, and so on, with the usual and well-known semantics, except for the small differences that follow from the nature of the mechanism (e.g., we need to generate the code slightly differently for tear-offs).

• The dynamic semantics of extension method invocation will select the method implementation which is most specific for the given receiver, for all the possible static types of said receiver.

• At call sites, the static analysis is able to detect properties of an instance extension method invocation just like it detects properties of regular instance method invocations; in particular, myB.m(...) will statically have the return type declared for m in the extension of B and not anything else (e.g., not the return type of m in an extension of a superclass A), and the parameter list shape and the parameter types for B.

With those things in place, I think it would be possible for developers to use this mechanism effectively, because there are so many things which are true for ordinary instance methods which are just the same for instance extension methods, and that's probably quite useful in the daily work —where application and domain complexity is rarely a scarce resource. ;-)

[eernstg]

The proposal for scoped class extensions has been optimized for simplicity: It has a well-defined semantics, including the choice of method implementation in the case where the given receiver implements two or more of the target classes that are not subtypes of each other.

But it could of course be extended in various ways.

For instance, we could easily allow a scoped class extension to be extended by another one, potentially in some other library:

extension MoreEval extends Eval on Subtraction {
  get eval => left.value - right.value;
} ... // More cases, as needed.

This could be taken to mean that the startup semantics of the enclosing Dart program would cause the basic "dispatch table" for Eval to be extended with an entry Subtraction: eval_Subtraction.

The reason why I did not do this is that we would need to have a well-defined prioritized dispatch which would kick in, e.g., for the case where the dynamic type of the receiver implements both Sum and Subtraction (obviously, someone will write such a thing ;-).

In that case, and in particular when we have 25 extensions like this all over the program, we would not have an obvious criterion for the ordering which would disambiguate such an invocation. In terms of the desugaring I mentioned, we would need to merge a new element into the list of dispatch tests (if (e is Subtraction) return eval_Subtraction;), and there may be many positions for the new element that satisfy the constraints (of not contradicting the subclass tree).

Similarly, we could have conflicting declarations: Different implementations for the same target type, possibly with different signatures, or declarations of an overriding implementation of an extension method m which is a correct override of all the declarations which are known (that is, in scope) at this declaration of m, but where some other extension makes this declaration an overriding declaration of some other declaration (which is not in scope), and they have an incorrect override relation.

// Assume class hierarchy D <: C <: B <: A.

// Basic declaration of extension `Foo`.
extension Foo on A {
  void m() {}
}
on B {
  void m() { print("B!"); }
}

// Extension of `Foo` in Library L1.
extension Foo1 extends Foo on D {
  void m() {}
}

// Another extension of `Foo` in L2, not imported by L1.
extension Foo2 extends Foo on C {
  void m([int i]) { print("C $i"); }
}

If a program does not include L2 then Foo1 is OK, but if we add an import of L2 then Foo1 will override void m([int]) by void m(), and that's an error. But it is not possible to detect this error during compilation of L1, it only arises when L1 and L2 are brought together.

Presumably, it would not be very convenient to manage a large scale software system if that kind of conflicts arise en masse as soon as we add one more dependency to a package.

Robust and comprehensible solutions to this problem would be accepted with loud expressions of joy! 🥇

[ds84182]

@eernstg This reminds me of Rust's traits. In Rust, you can't implement an external trait (one from another crate) on an external type (one from another crate). You can implement your trait on external types, but you can only implement an external trait on your types. There are some more rules when it comes to type parameters (iirc you can implement an external trait on an external type if you use internal types as a type parameter, no clue how this will work with Dart's covariant generics)

A similar design in Dart could feel odd because of the definition of a library, but it could scale well in a larger application (for example, you could take advantage of parts to separate out extension declarations for large files like Kernel's ast.dart)

In your example, C and D have to be defined in L2 or L3 in this model, which makes L2 have to import L3, which makes the program valid. This also simplifies finding all extensions for a particular class, since the extension declarations are next to the classes.

[eernstg]

Interesting, @ds84182!

My understanding is that Rust enforces that each trait is implemented at most once for any given type, and it is the orphan rule ("can't implement an external trait for an external type"), and this constraint would help making that property locally enforceable. (I think, when it comes to Rust there aren't that many things that I know for sure).

Another thing is that object-oriented dispatch arises only with trait objects, and they carry something similar to a vtable around, such that all the trait methods around, along with the underlying entity of some type that implements the given trait(s).

This means that we never have anything that could be called an implementation override relation, and every method invocation (which isn't monomorphized and compiled as a static call) can be performed using a vtable. So we're in a context which is so different from Dart that it's getting difficult for me to understand what it would even mean to say "let's try the same thing for Dart". ;-)

However, you could say that a scoped class extension is similar to a trait in the sense that it is capable of adding support to a class hierarchy (thereby a type hierarchy, and all subtypes) for an additional group of member signatures. So maybe the initial example I have could be said to "add support for the Eval trait to the Expr types".

Given that I've made the scoped class extension a single declaration, it is enforced that we do this in a single library, and the extension declaration specifies both the interface and the implementations, so in that sense we are requiring that all trait implementations (the ones that are scoped class extensions) must occur together with the trait itself.

Conversely, we can implement the same added methods by editing the target classes directly, in which case you could say that we specify the interface and implementations together with the 'types' (the target class declarations).

If we were to allow extensions of extensions then that could presumably be described as "implementing an external trait for an external type", in which case we could say that this proposal actually includes the orphan rule. ;-)

[eernstg]

Here is an example that visits a number of issues with a rewriting approach where a scoped class extension is implemented by rewriting it to some static extensions, and in particular whether it would be realistic to perform this rewrite by hand.

I gave a bunch of reasons for this, and the example below illustrates some of them:

I believe that such a rewrite-to-static approach would have to introduce multiple dispatch methods (each method m1 and m2 would have its own dispatcher; each extension where m1/m2 needs dispatch would have its own dispatcher; so that's "M times N" dispatchers); and they'd need to be distinct (in particular, they would have different signatures), and they could not be one implementation and a lot of forwarders to the implementation (because some of them would accept parameters that others should reject).

Finally, with the rewrite-to-static approach, the actual implementation methods would need to have different names than the dispatcher methods, because they both need to exist in the same scope. So we would have an m1 dispatcher and an m1_impl method containing the actual implementation, and dispatch code would need to select the m1_impl which is declared by the most specific static extension which has one (alternatively, an m1_impl would need to be added to every static extension which is the result of desugaring the given scoped class extension, and the ones that emulate an inherited m1 would forward to an m1_impl of the static extension targeting the relevant superclass).

Also, I think developers would not receive a very good service with respect to overriding method declarations (because it would remain undetected in a lot of situations where the emulation of an overriding declaration is not correct, and in other situations it would be difficult for developers to understand that this is actually "an extension method override error", because the rewriting obfuscates that fact).

Also, using a static rewrite approach I believe that a super-invocation would be rewritten to a receiver cast (to the superclass that declares the desired member) followed by an invocation of the given extension method. But that would also be hard to read and maintain: It looks just like any other receiver cast, and the target of the cast would be the target declared for one of the other static extensions, and there's no support for detecting if it does not happen to be the right one; finally, the invocation would need to invoke the actual implementation method of the relevant static extension (m1_impl), not the dispatcher (m1).

Here is the example:

// Library 'abc.dart'. We cannot (or do not wish to) edit this.
class A {}
class B extends A {}
class C extends B {}

// Library 'ext.dart'. This is the scoped class extension.
import 'abc.dart';

extension Ext on A {
  Object m1() {
    print("Ext_A.m1");
    return null;
  }
}

on B {
  num m1([int i]) {
    print("B.m1");
    return i;
  }
  void m2(covariant num n) {
    print("B.m2: ${m1(n.floor())}");
  }
}

on C {
  int m1([num _]) {
    print("C.m1");
    return super.m1(42) + 1;
  }
  Object m2(int i) {
    print("C.m2: ${m1(i + 0.1)}");
    return null;
  }
}

// Library 'main.dart'.
import 'abc.dart';
import 'ext.dart';

void main() {
  C c = C();
  B b = c;
  A a = b;

  Expect.equals(c.m1(1), 43);
  Expect.equals(c.m1(1).isEven, false);
  Expect.equals(c.m1(1.1), 43);
  Expect.equals(c.m1(1.1).isEven, false);
  var _1 = c.m2(1);
  c.m2(1.1); //# int_expected: compile-time error

  Expect.equals(b.m1(1), 43);
  b.m1(1).isEven; //# isEven_not_defined: compile-time error
  Expect.equals(b.m1(1).floor(), 43);
  b.m1(1.1); //# int_expected_2: compile-time error
  b.m2(1);
  // var _2 = b.m2(1); // # cannot_use_void: compile-time error
  b.m2(1.1); //# int_expected_3: runtime error

  a.m1(1); //# too_many_arguments: compile-time error
  a.m1(1.1); //# too_many_arguments_2: compile-time error
  a.m2(1); //# m2_not_defined: compile-time error
  a.m2(1.1); //# m2_not_defined_2: compile-time error

  c.m1();
  c.m2(); //# too_few_arguments: compile-time error
  Expect.equals(b.m1(), 43);
  b.m1().isEven; //# isEven_not_defined_2: compile-time error
  Expect.equals(b.m1().floor(), 43);
  b.m2(); //# too_few_arguments_2: compile-time error
  Expect.equals(a.m1(), 43);
  a.m1().isEven; //# isEven_not_defined_3: compile-time error
  a.m1().floor(); //# floor_not_defined: compile-time error
  a.m2(); //# m2_not_defined_3: compile-time error

  b = B();
  a = b;

  Expect.equals(b.m1(1), 1);
  b.m1(1).isEven; //# isEven_not_defined_4: compile-time error
  Expect.equals(b.m1(1).floor(), 1);
  b.m1(1.1); //# int_expected_4: compile-time error
  b.m2(1);
  b.m2(1.1);

  a.m1(1); //# too_many_arguments_3: compile-time error
  a.m1(1.1); //# too_many_arguments_4: compile-time error
  a.m2(1); //# m2_not_defined_4: compile-time error
  a.m2(1.1); //# m2_not_defined_5: compile-time error

  Expect.equals(b.m1(), null);
  b.m1().isEven; //# isEven_not_defined_5: compile-time error
  b.m1().floor(); //# called_on_null: runtime error
  b.m2(); //# too_few_arguments_3: compile-time error
  Expect.equals(a.m1(), null);
  a.m1().isEven; //# isEven_not_defined_6: compile-time error
  a.m1().floor(); //# floor_not_defined_2: compile-time error
  a.m2(); //# m2_not_defined_6: compile-time error

  a = A();

  a.m1(1); //# too_many_arguments_5: compile-time error
  a.m1(1.1); //# too_many_arguments_6: compile-time error
  a.m2(1); //# m2_not_defined_7: compile-time error
  a.m2(1.1); //# m2_not_defined_8: compile-time error

  Expect.equals(a.m1(), null); //
  a.m1().isEven; //# isEven_not_defined_7: compile-time error
  a.m1().floor(); //# floor_not_defined_3: compile-time error
  a.m2(); //# m2_not_defined_9: compile-time error
}

Here is a Gerrit CL which adds a desugared version of the above program elements, which runs with the current tools (except for a single bug which is not related to this topic).

[eernstg]

The language team has had some requests for "virtual constructors". For instance, there was a proposal about extending the instances of type Type that the platform delivers as a reification of a type such that they would have instance methods for each static method and constructor that the corresponding class had:

// THIS WAS NEVER IMPLEMENTED, and it's not going to happen.

class C {
  static void staticMethod() {}
  C.named();
}

main() {
  dynamic d = C(); // We've forgotten that this is a `C`.
  Type t = d.runtimeType; // Get hold of its reified type.

  // The reified type would then have an instance method for each "class method" of `C`.
  t.staticMethod(); // Calls `C.staticMethod()`, but is a dynamic invocation.
  dynamic other = t.named(); // Creates an instance like `C.named()`, but dynamically.
}

This feature does not fit very well in a statically typed context (so it was dropped before we even decided on what 'Dart 2' would mean), because it would require the type Type to have all those members: There should be a member named named such that we know we can call t.named(), and similarly for staticMethod, but the set of members for any given variable of type Type would differ from one moment to the next, depending on which class reification is its value right now (if we do t = String then it should suddenly have members fromCharCode and fromEnvironment, etc.). So that's an inherently dynamic mechanism.

But if we can live with a solution that requires us to write an explicit list of types that we will create instances of then we can express a clone operation as follows:

// Here are the target classes, e.g., "everything in Flutter materials".
abstract class A1 ... {} // Assume `A1` is a supertype of all of these.
class A2 ... {}
...
class Ak<X> ... {} // Target classes can be generic, no problem.

// Here is the Clone extension.
extension CloneA on A1 {
  // We included an abstract class in order to show that it needs special treatment.
  // This case is only reached if some third-party class `B` (not among 
  // A1 .. Ak) implements `A1` and `someB.clone()` is invoked. We don't know `B`,
  // so we can't clone it: throw.
  A1 clone() => throw "Trying to clone a class outside A1..Ak";
}
on A2 {
  A2 clone() {
    // We can use `this` here, if we want a clone with the same state,
    // or we could use some default values if we want an "empty" clone.
    return A2(some, constructor, arguments, forA2);
  }
}
...
on Ak<var X> { Ak<X> clone() => Ak(arguments, forAk); }

// Usage example.
main() {
  A1 a = e; // An instance of `Aj` for some `j`.
  A1 otherA = a.clone(); // One more instance of `Aj`.
  A2 a2 = A2(); // Maybe we know a subtype statically.
  A2 otherA2 = a2.clone(); // Then we also know that the clone is an `A2`.
}

If we extend Type to have a type argument (such that the reification of num is an instance of Type<num> and the reification of String is an instance of Type<String>, etc), then we can also make reified type objects work in a similar manner:

extension NewA on Type<A1> {
  A1 newA() => throw "Trying to create an instance of a class outside A1..Ak";
}
on Type<A2> { A2 newA() => A2(some, constructor, arguments, forA2); }
...
on Type<Ak<var X>> { Ak<X> newA() => Ak(arguments, forAk); }

// Usage example.
main() {
  A1 a = e; // An instance of `Aj` for some `j`.
  Type<A1> t = a.runtimeType;
  A1 otherA = t.newA(); // One more instance of `Aj`.
  if (t is Type<A2>) {
    A2 otherA2 = t.newA(); // Statically safe.
  }
}

Note that we can only call newA on a receiver with static type Type<T> such that T is a subtype of A1 — otherwise it won't match any of the cases in the extension NewA, and Type does not have a newA method. We then know statically that the returned object has type T. (OK, we can be a bit more precise here: if T <: Aj then we know statically that t.newA() has type Aj).

So the situation where we have confirmed that t is a Type<A2> allows us to safely assign the returned object to a variable of type A2 (where "safely" means "also with --no-implicit-casts").

We can also use a type parameter to create a new object:

A1 foo<X extends A1>() => X.newA();

This works because X evaluated as an expression yields an instance of Type<X>, and it is statically known that X <: A1 (such that the extension is applicable).

This won't create an instance of X in all cases (for instance, newA will not return null in the case where X is Null), but it will return an instance of the "best" type Aj among A1 .. Ak, namely the one where X <: Aj, and no i > j satisfies X <: Ai. Based on the rules for scoped class extensions, this is the most special class we can get.

Of course, foo<A1>() will throw (because that's how the extension method newA for Type<A1> is implemented, because A1 is abstract), but we could choose to let it return an Aj for some j, effectively emulating a factory constructor on A1 that returns an instance of some "default" among the possible types A2 .. Ak.

[shinayser]

I am sorry if this is a dumb question, but this proposal allows us to pass an Extension Function as parameter to another function?

For example, imagine a class Person and a function that loads it from a database:

class Person {
  String name;
  int id;

  Person(this.name, this.id);

}

Future<Person> loadPersonFromDatabase(int id) => personDatabase.queryById(id);

Now, imagine a extension function toJsonon the Person class:

extension Jsonize on Person {

 String toJson() => {"name" : name, "id" : id};

}

Will we be able to pass the toJson as parameter to the then method of a Future object, like that below?

String json = await loadPersonFromDatabase(1).then(toJson);

If the answer is not, pehaps it's time to reconsider that proposal?

(I am not a languge designer, just a standard programmer, so this would be very helpful and handy to use on day to day.)

[morisk]

Extension are in essence mixin on already existing types.

mixin on String {
  newFunc() {}
}

We might add conditions or conformance:

 // Already defined in other library. I want to extend it only when T = String
class Transformer<T> {
}

mixin on Transformer<T> where T == String {
  newFunc() {}
}

So now we also can have regular mixin conditions, mixin signature will match class signature it extends.

mixin Tansforming<T> where T == String { }

class Transformer<T> with Transforming {} 

class Base {}

mixin Tansforming<T> where Self == Base, T == String { }

class Transformer<T> extends Base with Transforming {} 

Absrtact mixin. Since mixin cant be used by them self we can imply 'abastractness' from context.

mixin Tansforming<T> where T == String {
  someFunc() { print('') }
  reFunc() // must be implemented later
 }

class Transformer<T> with Transforming {
  otherFunc() {}
  reFunc() { } // must be implemented 
} 

[eernstg]

@morisk wrote:

Extension are in essence mixin on already existing types

It is true that scoped class extensions are intended to have a similar effect, and it might be quite useful to think about scoped class extensions as mixing in some mixin on an existing type, in order to get a good first approximation of their semantics.

However, they differ in other ways: Scoped class extensions don't rely on actually changing any classes. This is crucial in cases where the target classes are platform controlled like int or String, because compilers and other tools can give such objects special treatment, and that would not work if an arbitrary user-written library could force changes to the representation of these objects. It's also a serious source of complexity (if at all possible) to allow an arbitrary library to change the layout of all instances of certain types (or all of them, if we are adding things to Object), and unlikely to be compatible with separate compilation.

Next, the fact that scoped class extensions are applicable only when in scope ensures that conflicts can be handled locally. Otherwise you could have a library L1 adding a getter m to a given class C via a scoped class extension, and another library L2 adding a method m. But no single class in Dart can have such two members at the same time, so we couldn't ever desugar that into anything that somehow adds two new mixins to C. So this makes it hard to create a program where both L1 and L2 are present, especially if you want to ensure consistency: If both m members are proper instance members then we should be able to invoke each of them dynamically (o.m would tear off the method and call the getter), but there is no way we could disambiguate this invocation in the case where the current library imports both L1 and L2, or none of them.

With scoped class extensions, if you do want to import two different ones that have conflicting members (in particular, members with the same name) then you can use all the mechanisms that we are currently introducing for static extension methods in order to resolve any ambiguous invocations.

Conditions (like "a Transformer<T> has a newFunc member if and only if T <: String) is an interesting idea. We have discussed variants of that idea previously; at this point I believe that they would be covered in a quite useful manner by putting the condition on the member itself. The member declaration could then be located in any class-like declaration, be it a class, a mixin, a static extension method declaration, or a scoped class extension, and the only support that it needs is (1) compile-time errors for statically checked invocations where the condition is not satisfied, and (2) dynamic checks on the condition for dynamic invocations. So conditions could certainly be useful, but I don't think they're necessarily associated with any particular kind of class-like entities, they could go anywhere.

In summary, I agree that there are lots of interesting connections and similarities, but also maintain that scoped class extensions are not the same thing as mixins applied from a distance.

[eernstg]

@shinayser wrote:

this proposal allows us to pass an Extension Function as parameter to another function?

Sorry about the late reply!

The main issue here is whether it's possible to tear off a member of a scoped class extension:

extension Jsonize on Person {
  String toJson() => '{"name" : $name, "id" : $id}';
}
on Employee { ... } // etc.

main() {
  Person p = ...;
  String Function() f = p.toJson;
}

That would certainly be supported, following the same kind of rules as those of static extension methods.

However, a tear-off in Dart captures both the receiver and the method implementation, and the resulting function object will always invoke the given method on that receiver, you can't provide the receiver at the call site and invoke "the same method" on some other object.

It would be possible to tear off just the "identity" of the method (one way to say that is that we're tearing off the selector of the method), which would be a mechanism similar to a C++ pointer to member. The resulting entity would run different code for invocations with different types of receivers, because they are instances of different classes (and late binding is applicable). In C++, this mechanism could be implemented basically as an offset into a vtable: Provide the receiver, look up the function point stored at that location in the vtable, and call it with the given receiver and arguments.

But this is not difficult to emulate such an entity in Dart: You just need to write a function which does these things explicitly:

main() {
  String Function(Person) f = (p) => p.toJson;
}

This is also what you'd need for the example:

String json = await loadPersonFromDatabase(1).then((p) => p.toJson);

In this respect, methods in scoped class extensions and static extension methods are just like regular instance methods, a torn off function will always capture the receiver. And it wouldn't work to pass a plain toJson to then, with any of those kinds of methods.

However, there are many proposals for how to express a function like (p) => p.toJson more concisely. For instance, it could be written as => it.toJson with the approach proposed in #265.

[morisk]

@eernstg wrote:

Scoped class extensions don't rely on actually changing any classes. This is crucial in cases where the target classes are platform controlled like int or String, because compilers and other tools can give such objects special treatment, and that would not work if an arbitrary user-written library could force changes to the representation of these objects.

Would be great if it was possible to extend any object, including String. Other languages do that, like Swift, Kotlin, JavaScript.

@eernstg wrote:

...applicable only when in scope ensures that conflicts can be handled locally.
Does one can write library that (using scoped class extension) extends other library?

"Prototypical based inheritance" is very flexible and convenient paradigm. I think Swift nailed it. Maybe there are parallels worth considering.

[eernstg]

@morisk wrote:

Would be great if it was possible to extend any object

That's no problem for mechanisms like static extension methods and scoped class extensions, and they can also add methods to other types (like function types), but you cannot add state and you cannot invoke the added methods when the receiver has type dynamic.

In JavaScript you can change whatever you want, but it may be costly in terms of performance.

But Kotlin extension are resolved statically, so they can't be virtual (as opposed to methods in scoped class extensions). Swift protocol extensions have a similar restriction: 'Protocol extensions can add implementations to conforming types but can’t make a protocol extend or inherit from another protocol. Protocol inheritance is always specified in the protocol declaration itself' here.

So the ability to extend any object comes in many shades. ;-)

.. conflicts can be handled locally

.. library extends other library?

That would involve a notion of first class libraries (similar to functors in SML, or objects used as libraries in Self or Newspeak), and that's a substantial generalization in a statically typed context. Similarly, prototypes and delegation is indeed very flexible, but hard to analyze statically. So Dart won't get these things any time soon..

[morisk]

@eernstg

...you cannot add state and you cannot invoke the added methods when the receiver has type dynamic.

Personally, I think it worth sacrificing "adding state", and "usage of dynamic" in favor proper extension as long as I have access to underling objects state from a mixin/extension.

Protocol extensions can add implementations to conforming types but can’t make a protocol extend or inherit from another protocol. Protocol inheritance is always specified in the protocol declaration itself.

"Protocol extension" is a term to describe adding functionality to a protocol, an "abstract class extension" in Swift. Protocol extension is a subset of Extension, where you can extend any object type.
Protocol can conform to other protocols, extending (not 'inherite') other protocols.

In Swift it is very common to extend classes.

extension Int {
  func toEnglish() -> String {...}
}
say(42.toEnglish())

And it can have awesome feature in RxDart (for example)

  final button = FlatButton(...);
  button.rx.change
      .map((event) => event.keyCode)
      .bufferCount(10, 1)
      .listen((_) => result.innerHtml = 'KONAMI!');

[eernstg]

"Protocol extension" is a term to describe adding functionality to a protocol

Yes, I mentioned protocol extensions because (regular) extensions in Swift are similar to Kotlin extensions in that they cannot support virtual methods, and that's the core difference between static extension methods in Dart and scoped class extensions.

In any case, there's a large language design space around these constructs, and it will always be a delicate matter to get the best balance between the expressive power and the cost in terms of complexity/performance/etc.

[laczoka]

Hi,

is this language feature still being considered?

[eernstg]

Right now the non-nullable types are taking up almost all resources. This proposal is not on any short lists, but also not dropped. I think it would be a very useful generalization of the static extension methods that we already have.

[junaid1460]

I think this should be prioritised, developer happiness is something dart lacks, which should not be the case, to me honestly I am writing a production application

• I feel lazy thinking about writing another class, it's too much boiler plate to maintain, external build runners which I almost all the time forget to run etc
• front end code mostly would have builder pattern, scoped extension is something that would make code readable.
• data classes make big things look smaller (Kotlin, it helps me mentally).
• please try to promote functions over classes, they have smaller foot-print.

examples:

{required .., required..., required...} why required all the time, when we know it's going to be common.
what is the matter with that final and const when compiler can tell me that it can't be a const why can't it just fix it by itself.

[eernstg]

Thanks, @junaid1460!

However, note that the parts of your comment which are arguably concerned with other topics would be more effective if they were given as comments on the relevant issues. For instance, #878 and #938 contain a long discussion about the choice to make required explicit rather than, say, inferring it based on the parameter type. The recommendation to promote functions over classes would be a long discussion, and it would certainly require some more supporting material. Data classes are expected to be handled mostly by static metaprogramming.