Should non-virtual methods be generalized to classes?

[sigmundch]

In the proposal for #2360, we talk about extension structs having statically dispatched methods. As hinted in #2352 (comment), I'd like to separately consider whether nonvirtual method dispatch should be it's own feature.

Part of my goal is to split what we mean by views/extension structs into smaller building blocks (similar to how "extension structs" are a specialization of "structs") - the notion of non-virtual dispatch and how we compose and create forwarding stubs can be managed as a separate feature, then views are mostly adding the notion of an erasable type.

As @leafpetersen, this exists in other languages like C#.

Some examples for illustration:

class A1 {
  int m1() => 1;
  nonvirtual int m2() => 2;
}

We could consider extends also exposing nonvirtual methods (just like extension methods do):

class A2 extends A1 {
   // inherits both m1 and m2, but m2 is dispatched statically
}

And we could consider allowing overrides or not:

class A2 extends A1 {
   nonvirtual int m2() => 3; // TBD if we allow it
}

// If we do, nonvirtual methods are dispatched statically:
// (A2() as A1).m2() returns 2

As @leafpetersen pointed out, there are special considerations to be made to handle generics:

class A<T> {
  nonvirtual void show() { print(T); };
}

void main() {
  A<num> a = A<int>();
  a.show(); // prints "num" not "int"?
}

cc @leafpetersen @mit-mit @lrhn @eernstg @chloestefantsova @johnniwinther @munificent @stereotype441 @natebosch @jakemac53 @rakudrama @srujzs @rileyporter @mraleph

[lrhn]

If we allow non-virtual methods on classes, we need people to be able to step back and understand that they are completely different beast than virtual methods.

For example, the "override" example above:

class A2 extends A1 {
   nonvirtual int m2() => 3; // TBD if we allow it
}

has a method with the same signature as A1.m2.
It's not actually an override. The subclass method could just as well have been:

class A2 extends A1 {
   nonvirtual String m2(int x, String y) => y * x;
}

There is no relation between the two methods, except that the subclass method shadows the superclass methods when invoked on an expression with a subclass type.

We can also choose to allow:

class A2 extends A1 {
   String m2() => "I'm virtual now";
}

that is, adding a virtual method with the same name as the superclass non-virtual method. That's no more or less problematic than if the superclass method had been a static method with the same name, the two are still completely unrelated.
If you then do:

  A2 a2 = A2();
  A1 a1 = a2;
  print(a1.m2()); // prints 2
  print(a2.m2()); // prints "I'm virtual now"

For generics, it's not a given how to handle those (but after writing these paragraphs, I do have a preference).

I can see a.show() printing int. After all, the T variable is not non-virtual. The print(T) method is statically resolved, but it is still invoked on a this object with runtime type A<int>, so maybe T will be resolved to the int type, just as:

Consider:

abstract class X<T> {
  abstract final T _value;
  nonvirtual T get value => _value; // Can a non-virtual method refer to `T`?
}
class Y extends X<int> {
  final int _value;
  Y(this._value);
}
void main() {
  var x = Y(42).value;
  print(x);
}

I see no reason that code cannot work and print 42.
The _value field does not exist in the X class, but the get value member can still access it virtually.
For get value to be valid, it also needs to have access to the T type, which every X must have. Which shouldn't be a problem since it can also access _value which has that type.

Also consider:

class C<X> {
  X v1;
  X v2;
  C(this.v1, this.v2);
  void setBoth(List<X> values) {
    v1 = values.first;
    v2 = values.last;
  }
  nonvirtual void swap() {
     setBoth(<X>[v2, v1]);
  }
}
C<num> c = C<int>(37, 42);
c.swap();

Is this code allowed? I think it should be. The swap operation is general and works on all Cs.
Is it safe? It is, if X is actually the X of the surrounding instance, not from the static type.
Otherwise the List<num> would make setBoth choke on the covariant generic parameter check.

I'm thinking that type parameters of a generic class are just as virtual and instance-based as virtual instance members. They're just private to the class hiererchy and not exposed by the public API. (So we do have protected, yey!)

(For the record, C#'s classes are more complicated than just being "non-virtual". Their virtual methods can exist in multiple independent variants on the same object, even with the same signature, and which virtual method you invoke depends on the static type that you call it on - there are overrides if you need another one.
It's also simpler in some ways than Dart, because a C# class does not introduce an interface, so the non-virtual methods can still assume that they apply to an object extending the declaring object, because all subtypes of a class extends the class. There are no non-virtual methods on interfaces. I guess that's called interface default methods instead.)

[leafpetersen]

I filed a somewhat (but possibly not entirely) redundant issue on the same basic idea here.

[leafpetersen]

On the question of generics, we can definitely implement these either way: that is, given this example:

class A<T> {
  nonvirtual void show() { print(T); };
}

void main() {
  A<num> a = A<int>();
  a.show(); // prints "num" not "int"?
}

We could make it print num or int. The former is consistent with viewing these as extension methods, whereas the latter is consistent with viewing these as methods are that in some sense private to the class. [Update: to avoid confusion, I don't mean "private" in the sense of only callable from the class, but rather that they are "name mangled" to the class such that they are only reachable through the specific interface of the class.]

Note though that for the "views" use case, I think we would be pretty much forced into the static type based approach, in which this would print "num", since there is no reified type there to use.

This actually doesn't seem like a terrible story to me, if we simply present these as a way of defining inline extension methods on classes, which "go with" the type.

[mraleph]

I think if we make them to behave as extensions (as in: look at the static type of the receiver rather than actual reified runtime type), then nonvirtual is a somewhat confusing name. To me nonvirtual simply implies direct dispatch. Maybe it should be extension then (indicating then this method behaves equivalent to declaring it in an extension type):

class A<T> {
  extension void show() { print(T); }
}

An interesting question here is whether or not we could apply this to fields:

class A<T> {
  nonvirtual int foo;  // extension does not fit here.
}

Semantics would be that this declares a nonvirtual get foo => this.:foo; allowing to introduce fields which can't be overriden with getters.

[lrhn]

The former is consistent with viewing these as extension methods,

Extension methods are limited to the public API of the type they are working on. We don't currently have a way to access the type variable from the outside.

Also, the way extensions are declared, the type variable they have access to comes from the extension declaration:

extension F<T> on Future<T> {
  /// T refers to extension declaration, not future declaration.
}

If we get type argument patters along with pattern matching, then you can access the runtime type variable from the outside the class.

You'd then do

extension F<T> on Future<T> {
  Future<List<T>> get singleton async {
     Future<var R extends T> future = this; // Extracts actual runtime value type of future.
     return <R>[await future];
  }
}

and actually return a list of the same type as the future's value type.

I'd even expect the following to be valid:

extension F on Future<var R> {
   Future<List<R>> get singleton async => <R>[await this];
}

The var R statically matches any type, and at runtime it gets bound to the actual runtime type argument.

With that, I don't think it's unreasonable to expect

class A<T> {
  nonvirtual void show() { print(T); };
}

to refer to the actual runtime type argument to A.

And even without type argument patterns, I'd still expect T to refer to the runtime type argument, because that's what the T variable denotes. There is no other type variable in scope, like there is for current extension declarations.
It would be very surprising if it meant anything else.

If we make it "inline extension methods", I'd probably propose a syntax like:

  void A<X>::show() { print(X); }

in order to introduce a name for the captured static type. You are not allowed to refer directly to the class's type parameters.

That also allows

  void A<X extends Comparable<X>>::sort() { ... }

which is an extension method which doesn't apply to all types of the containing class. You can declare that using extension, so it's reasonable to allow it for inline extensions too.

(I think that's C# syntax for extension methods, they just put them outside of the class, which means that ClassName:: isn't redundant.)

[lrhn]

The problem with having non-virtual fields and implicit class interfaces is that the non-virtual getter would be callable on the interface type (because there is no distinction between the type of the class and of the interface), but an implementation which doesn't extend the declaring class won't necessarily have that variable.

[eernstg]

The approach where a non-virtual method is an extension method that follows the type is pretty clear. We would bind the type variables of the enclosing class based on the static type of the receiver (which is a quite dangerous anomaly, by the way). Invocation would only occur based on static resolution.

For the "other" potential semantics, we could view it as follows: nonvirtual simply makes it a compile-time error to override a nonvirtual declaration with a concrete declaration (nonvirtual or not). This makes nonvirtual essentially the same thing as final on methods in Java.

We would then have access to the actual run-time value of type parameters of the enclosing class (no new features needed, and no funny discrepancies with other declarations in the same scope about the meaning of X). Invocations could occur using normal OO dispatch, but in the case where the nonvirtual declaration is statically known, it is possible to resolve a statically checked invocation as a static function call. Dynamic invocations would work as usual.

If a class B implements another class A that has a nonvirtual method m in its interface then the given (unique) implementation would be added to B using something like an implicit mixin application (so this would give us a feature which is somewhat similar to interface default methods). We need to enforce that the implementation is the same as in A, such that it is correct to resolve the invocation statically when the static type is A and the dynamic type is B or a subtype thereof.

Finally, it is a compile-time error if B extends S and implements S1, .. Sk, and the superinterfaces of B have multiple nonvirtual declarations with the same member name but different implementations. (We could have B implements B1, B2, and B1 implements A and B2 implements A, so both B1 and B2 could have an implementation of a non-virtual member which is added by that "implicit mixin" mechanism based on the implementation in A, and that's OK).

With respect to "static overriding": I'd very much prefer that we avoid allowing name clashes; if A.m and B.m are completely independent methods then they should have different names.

class A {
  nonvirtual void m() {}
}

class B extends/implements A {
  nonvirtual String m(int i) {} // Error. Different purpose, different name.
}

[eernstg]

With the extension method based approach, how would we handle this?:

class A1 { nonvirtual void m() { print('A1'); }}
class A2 { nonvirtual void m() { print('A2'); }}

class B implements A1, A2 {}

void main() => B().m();

I guess we could just make it a compile-time error, "ambiguous non-virtual method invocation".

[lrhn]

Making non-virtual be the same as Java final can work, but mainly if the class is also sealed to only allow extending subclasses.
The solution to allow other classes to implement the class's implicit interface, and implicitly mix in the non-virtual method, will prevent the non-virtual method from depending on implementation details. The behavior is closer to an interface default method then, just one which happen to also be final and non-replaceable.
An instance method can usually depend on this extending the declaring class, but a mixin method can only expect the class to implement that interface.

It's probably not a big difference. Depending on access to private members that the implementing class from another library doesn't know about is the big one, so don't do that. I expect some optimizations will be off the table too.

[leafpetersen]

With the extension method based approach, how would we handle this?:

class A1 { nonvirtual void m() { print('A1'); }}
class A2 { nonvirtual void m() { print('A2'); }}

class B implements A1, A2 {}

void main() => B().m();

I guess we could just make it a compile-time error, "ambiguous non-virtual method invocation".

I hint at this in the other issue I filed: I think I would propose that as you say, this is an error, but one which can be resolved by adding an "overriding" extension in B. This seems to match up well with comments from @rileyporter about how the JS interop use case works. They wish to combine "methods" from multiple super-views in sub-views, and sometimes they wish to hide something from a super-view in favor of an override in a sub-view.

I'm not convinced that the the "extension/nonvirtual" approach (using extension method semantics) is the right way to solve this problem from the standpoint of a user facing feature, but it does seem to fit the requirements well.

This does suggest to me that we might also want to take another look at the original idea that @lrhn was pushing to build this off of extension methods as "extension types". My main objections to that remain:

• The type of this in extension methods is wrong
• The preference for instance methods over extension methods is wrong.

The former though is possibly just a thing we could just change for extension types, and the latter may never arise here.

Lesser objections that I have to that approach:

• We would need a new way to talk about the "on type" value. We've proposed using "super", which is ok, but I continue to be reluctant about re-purposing existing syntax.
• We may need to invent new syntax for introducing a subtype hierarchy between extension types
• We would need to invent new syntax for "inheriting" extension type methods.
• We would need to invent constructor syntax.
• Probably other things

So in other words, it continues to feel to me that building these off of extension methods as extension types requires inventing a whole pile of new syntax that already exists on classes, so if we can find a way to hang these off of classes (or a slight variant thereof, e.g. structs) then we greatly reduce the surface area of the feature, both from an end user standpoint, and from a language design standpoint.

[lrhn]

• We would need a new way to talk about the "on type" value. We've proposed using "super", which is ok, but I continue to be reluctant about re-purposing existing syntax.

We should have an "on pattern" which allows naming the match. Example:

extension Fut on Future<var T> f {
  /// Handles error if it's an [E] and then evaluates to `null`, otherwise same as this future.
  Future<T?> handleError<E extends Object>(void Function(E error, StackTrace stack) handler) =>
    f.then<T?>((T v) => v, onError: (Object error, StackTrace stack) {
      if (error is E) {
        handler(error, stack);
        return null;
     } else {
       throw e; // rethrow
     }
   });
}

The binding pattern allows naming types (if we have type patterns) and naming the matched value itself,
so you get a name to refer to it. Don't need this or super then to refer to the source object then.

[lrhn]

I like the syntax

view ViewType(OnType id) { ... }

for views (OnType id is basically a pattern that allows static type matching against static types and run-time irrefutable binding), and I'd be fine with adopting it for extensions too, as:

extension ExtName(OnType id) { ... }

allowing access to the "on" object as id instead of this. We can then deprecate the old syntax and recommend the new one.

Generics work too, as extension Ext<T>(T x).

It looks like a "primary constructor" declaration, but unlike for a struct, the id variable does not become an instance variable, it's just a "local" variable available to the instance methods, the same way this is available to class/struct/view instance methods.

[eernstg]

@sigmundch, I created a rough proposal for a kind of non-virtual members in classes in #2510, known as 'class extension members'.

They are based on the semantics of regular extension members (as specified for now, they are just a thin layer of syntactic sugar on top of regular extension members).

One of the major motivations for exploring this kind of feature is that it could serve as a feature in its own right, and it could also serve as a precursor to view methods: "A view class is just a class where every member (except the final instance variable that holds the representation object) is required to be a class extension member; hence, the modifier extension is optional."

Would they serve the purposes you had in mind?

[sigmundch]

Thanks for sharing @eernstg - I like it :) - I see the parallel with the discussed in the other doc (what I had called "attached extensions"), and yes, I believe they give us a similar expressive power. I also especially like the idea of making views stand on simpler building blocks (a class that uses X and doesn't use Y)

[eernstg]

Sounds good, thanks!