Views on an object without a wrapper object

[eernstg]

Extension methods (#41) can extend the set of instance members of an interface type T with a set of members declared outside of T. We may describe this as a zero-cost abstraction, because the denoted instance of type T is used directly (without an additional wrapper object, or any other kind of run-time support), and because the extension members are resolved statically (such that extension member invocation can be even faster than instance method invocation). In short, extension methods allow us to add convenience functions operating on instances of T, and invoking them using the syntax of instance member invocations.

We may also wish to replace the set M of instance members of an interface type T by a set M_alternate of members declared outside of T. The two sets may overlap, but the point is that some of the instance members of T should not be used, or they should only be used in a specific way. A zero-cost abstraction mechanism that allows us to perform this replacement could help enforcing this particular kind of discipline without lowering the performance: If an instance member shouldn't be invoked at all then it would be omitted from M_alternate, and if certain instance members can be called, but only in specific ways, then there would be some members in M_alternate whose bodies will call those instance members in the appropriate manner. In short, this kind of abstraction would allow us to impose an added discipline on the use of an instance of type T and invoke them using the syntax of instance members.

For instance, if a given JSON value is modeled using instances of List<dynamic>, Map<String, dynamic>, bool, String, or numbers, and the JSON value is intended to conform to a given schema, then we could use this kind of abstraction to ensure that the object representing the JSON value is not modified (by not including any mutating members), and that it is only accessed in a way which is consistent with the schema (by providing member declarations that will access the parts of the value according to their schemas).

[jodinathan]

can view extend another view?
as it can be used as interfaces for JSON (#783), I am wondering if we could mix up multiple interfaces.

In TS it would be like interface User = BaseUser & Customer. Where BaseUser and Customer are static declared interfaces.

[eernstg]

Yes, a view (as currently proposed) may have an extends clause, which allows the view to reuse members of other views as long as the on-type of the current view is a subtype of the on-type of each view that it extends. I'm not sure what 'mix up' refers to, but a name clash is an error (so you'd need to hide any clashing names).

The TS interface mechanism (and TS type aliases) create names for types, and they don't have an associated implementation (they could be implemented in many different ways). That makes them very different from views: Views are inherently tied to a specific implementation, because it's one of the main reasons for having views that they allow for a zero-cost abstraction over an existing underlying "representation" type (the on-type), and this allows view methods to be resolved statically, and, in some cases, inlined.

But if we focus on the sets of members only, there are similarities: Multiple extends views on a view V would serve to reuse members from existing declarations (rather than writing them all inside V, perhaps duplicating existing code), just like type User = BaseUser & Customer in TS would allow User to have all members from BaseUser as well as all members from Customer.

[cedvdb]

I have an use case for this in the form of Partial. Typescript has the concept of Partial to interface its objects.

I hope this could be a thing with views as those have many use cases. Here are a few:

• It solves copyWith issues with default parameters. (I'm 100% for less rigid requirement on default parameters which also solve the issue, Partial is just another way to solve this problem).

• The U in crud. Updating an arbitrary number of fields is currently very (to say the least) problematic in dart. You can either 1. send a copy of the object with the updated properties and diff it with the cached version right before sending the updated fields to the backend, or 2. use a map which is untyped. There is just no way around that. While 1 seems acceptable it pales (to say the least again) in comparison of just having Partial. I'm quite shocked that no one complains about this actually..

Partial were a game changer for me, I can barely keep breathing now that I don't have those anymore :(

[jodinathan]

@cedvdb I guess Partial is closer to the Records proposal

[jodinathan]

could views be used to somehow make a core type stricter?
ie:

view PositiveInt on Int {
  PositiveInt() {
    if (this < 1) {
      throw '$this is not a positive integer';
    }
  }
}

void foo(PositiveInt someInt) {
  // ...
}

void main() {
  foo(-3); // throw ?
}

[Levi-Lesches]

You'd probably need an assert using only constants if you want that to throw at compile-time. Otherwise PositiveInt might as well be an if inside foo itself.

[jodinathan]

@eernstg could views statically implement a class and be returned from a method?

I've implemented js_bindings that expose the HTML stuff through package:js.

However, I am not able to correctly expose JS arrays because I can only use extension methods and Iterable needs a member Iterator, so NodeList is an Iterable in JS world but I can't make it loopable in Dart.

I've created a wrapper so you can do final (child in document.nodes.toList()), however, that is far from ideal.

I am talking with @sigmundch about it here, maybe this is a case that views could efficiently solve.

[eernstg]

@jodinathan wrote:

could views statically implement a class and be returned from a method?

Yes, if you box it.

Here's a more detailed story, saying the same thing:

A view can have an implements T1, .. Tk clause, and this does imply that it must have each of the members of T1, .. Tk (by having a concrete declaration of a member m, or via a extends V1, .. Vn where Vj provides that implementation of m for some j). In any case, the member signature of any given member m must be a correct override of the member signature of m in each of T1, .. Tk that has a signature with that name. This is working just like a class or a mixin.

However, view method invocation is resolved statically. This means that with a view V on S .., the compiler knows that it is running the m member of V in a situation where the receiver is actually an object of type S, and this means that a view method invocation can be optimized (for instance, it might be inlined) at the call site. It's a crucial property of views that there is no need to create a wrapper object, and view method invocation is just as cheap (and optimizable) as calling a static function where the receiver is passed as an argument.

This again means that even though we have view V implements T1, .. Tk, we cannot allow a variable of type Tj to refer to an object whose static type is V. So we can't return e; when e has static type V, and the return type is Tj.

However, we can box the viewed object: return e.box;. This creates a wrapper object whose type is a subtype of all implemented types T1, .. Tk. Method invocation on the wrapper object is standard OO method invocation (so the compiler does not have to know the exact type of the receiver, it can be treated just like any other object of type Tj for any j).

So that's probably very nearly the same kind of wrapper as the JsArrayWrapper that you wrote manually.

@sigmundch said the following in the comment you mentioned:

I wonder if for-in could be redesigned to consider working not just on Iterable
but also on types that adhere to certain API.

That would indeed be interesting, and perhaps quite useful! The question is how much optimization it would enable.

We did actually change for-in statements such that they require an Iterable; they used to work with "anything that has the required members", that is, an iterator getter on the iteration target itself, and moveNext() and current on the object returned by the iterator getter. The intention was that for-in loops should be more statically typed (rather than being a kind of macros that would ignore the types for as long as possible), and that's very much in line with the evolution of Dart over several years.

But it might still be worthwhile to consider a relaxation of this rule again, e.g., such that it would be ok to use an iterator getter from a view that implements Iterable, hence avoiding the creation of the wrapper object.

[sigmundch]

Thanks @eernstg for taking a look. Precisely, your thoughts here are very aligned with I'm suggesting. In this comment I raise a similar but more general question: are there other places where a language feature asks for implements X where it could instead be based on some static structural typing instead? Would that make the use of static-views more flexible in general?

Other than for-in loops, anything else comes to mind that is based on the underlying type? For example await an Future? Expansion of the ... spread operator on Maps and Sets literals?

If we go down this route, would it make sense to have additional declarations to express conformance to an API? for example:

abstract view ForInArgument {
  Iterator iterator;
}

class Iterable conforms ForInArgument { ... } // classes can conform a static API 
view JSArray conforms ForInArgument { ... } // static-views can also conform a static API

Unlike implementing an interface, this conformance cannot doesn't create an abstraction at the use site. The expansion of the use site is more like a macro, where the lookup of members is resolved entirely based on the static type of the udnerlying expression. That is, in for (var x in e) ... we would practically do an expansion that uses e.iterator, and that e.iterator is resolved depending on the static type of e. Conformance would only be useful to detect statically that the type of e has all the APIs expected to be available for that expansion that the compilers will do under the hood.

[jodinathan]

maybe change the for-in to also add room to patter matching

[Levi-Lesches]

are there other places where a language feature asks for implements X where it could instead be based on some static structural typing instead?

I believe that explicit inheritance or implementation was preferable to an implicit structure. Compare the Comparable interface to Python's approach to double-underscore methods. Having .length and .iterator might signal that the API is meant to be used similarly to an iterable, but the best way to signal this is by telling it to Dart explicitly: class MyClass with IterableMixin { }. If your class really does fit the iterable structure, it'll compile. And if not, you'll get clear and concise error messages telling you why. Compare that to implicit structure typing, where there is no visible indication of how the type is meant to be used (besides for comments), and failing to implement the interface exactly will result in nothing -- no errors, lints, or warnings.

That covers cases where you are writing the class. So long as there is a way to extend existing classes to implement an interface (like extensions or views), being explicit is almost always better.

[eernstg]

@sigmundch wrote:

could instead be based on some static structural typing

I agree with @Levi-Lesches that the connection to an explicitly declared type is good for correctness and for readability, and I think we might not need to rely very much on structural typing. In particular, a for-in statement could require that the iteratee has

• a class type that is a subtype of Iterable<T> for some T, or
• a view type that implements Iterable<T> for some T (directly or indirectly), or
• a view type that has an iterator getter that returns a view type that implements Iterator<T>.

It is known statically which one of those is applicable, so we can generate different code for the three cases.

With this approach, it is indicated explicitly (in the class type or in the view type) that it is intended to support usage as an "iterable" object by means of an "iterator" object, and it allows (1) the traditional semantics, purely OO, (2) an approach where we run a view method to obtain an Iterator<T> and then proceed as normally, plus (3) an approach where we obtain an object which plays the role as an iterator, but we're using two view methods to implement moveNext() and current. Note that we don't have to have an actual Iterable nor an actual Iterator In the third approach.

The following is probably buggy, but it illustrates the idea:

view JSArrayIterable<X> on JSArray<X> {
  JSArrayIterator<X> get iterator => _JSArrayIterator(this);
}

view JSArrayIterator<X> on _JSArrayIterator<X> implements Iterator<X> {
  X get current => ...; // JS magic that looks up jSArray[index].
  bool moveNext() => ++index < length;
}

class _JSArrayIterator<X> {
  final JSArray<X> jSArray;
  int index = 0;
  int length; // It might be useful to cache the length?
  _JSArrayIterator(this.jSArray): length = jSArray.length;
}

[sigmundch]

I think we are all in agreement :) - that's why I was asking for the notion of "conformance", so we can statically express that the for-in is well formed.

@eernstg - my suggestion for conforms seems very similar to your suggestion for implements. The reason I suggested "comformance" instead of "implements", is that I find it confusing to say that a view implements an interface if the value of that view is not boxed.

If we decide to overload the implements keyword here, I'd like to make sure it is not confusing for users. In particular, today implements X means two things: (a) that they have a static interface declared by X, but also that (b) the members of X are available at runtime and that X is a valid supertype. Because of (b), I think developers have the intuition that ([] as dynamic) is Iterable is true. With static views, we only get (a) (which is what I was referring to a conformance). Would using the implements keyword make it more confusing that (JSArray() as dynamic) is Iterable is false?)

[eernstg]

The current views proposal uses implements T1, .. Tk to indicate that the members are implemented by a view, and the signatures are correctly overriding the signatures in T1, .. Tk, but it would probably not be hard to parse a different word (it probably doesn't even have to be a built-in identifier).

One reason why implements might still be an acceptable keyword to use in a view declaration is that the word view is right there at the front, so we can read it as view implements vs. class implements/mixin implements, and then it might not be too confusing that the latter includes subtyping and the former does not. In any case, we'll have the subtyping relation after .box, so there is also a certain value in emphasizing the connection.

[eernstg]

@sigmundch, I made an attempt to develop the ideas about allowing for-in statements to work on an object which has an Iterable-ish view on it in #2150, with the two constraints that we have stated in various ways in this discussion:

• When a for-in statement is iterating over a collection whose static type is a view type, we want to work on the on-type instance of the view at run time.
• We do not want to go back in time and simply drop the static typing constraint that a for-in statement should iterate over an instance of Iterable<T> for some T. So we're considering various notions of "conforming to" Iterable<T> that are slightly more permissive than "is a subtype".

To explain the first constraint a bit more, we do not want to box the object. That would yield a regular object which is typable as Iterable<T> for some T, and we would then simply treat the for-in statement as any other for-in statement. But we'd pay a price in terms of run-time performance, because we're allocating a new object, and we're performing the iteration by calling instance methods on that new object.

In contrast, the purpose of this discussion is to improve the performance by "lowering" the for-in statement to work on the on-type instance, based on the view methods (which may well be inlined).

We could simply require that a collection which is iterated over in a for-in statement must be assignable to Iterable<T> for some T, or (and that's the new thing) it must be a view with an iterator getter that returns an Iterator<T>.

class C<X> ... {...}

view CAsIteratable<X> on C<X> {
  Iterator<X> get iterator => ...;
}

This means that we relax the static check on for-in statements, but we preserve the static type constraint that we must be able to obtain a proper Iterator<T> for the given T. That iterator will then be used in the same way as with a normal (non-view based) for-in statement.

However, we could also allow the Iterator<T> role to be held by a view type.

class C<X> ... {...}

view CAsIteratable<X> on C<X> implements Iterable<X> {
  CAsIterator<X> get iterator => this;
  .. // Other `Iterable` methods.
}

view CAsIterator<X> on C<X> implements Iterator<X> {
  bool moveNext() => ...;
  X current => ...;
}

This would allow us to lower the for-in statement even more, because we could both obtain and use the iterator based on view methods (and we'd never have to allocate an actual object of type Iterator<T>, we'd just generate code calling the view methods, and possibly inlining them).

However, this gets awfully close to the old ("untyped") approach where we did not require anything from a collection in order to be able to iterate over it in a for-in statement (other than it should have an iterator getter, and that getter should return something with members that allow moveNext() to be a bool, and current to be assignable to the element type T).

So I proposed some new rules for the implements clause of a view (in #2150), such that we could maintain (recursively) that the members of a view are "conforming to" an interface like Iterable.

The core of this relationship is the following kind of "lifting" relation:

abstract class A {
  A get next;
}

view V on ... implements A {
  V get next => ...; // `V get next` can implement `A get next` because `V implements A`.
}

We can do this (e.g., as proposed in #2150), but the recursion is highly incomplete in the following sense: We can maintain the relationship for return types, but it is hardly useful for formal parameter types, and it immediately gets out of hand for any higher-order cases.

For example, we can allow CAsIteratable<X> to have a getter iterator that returns CAsIterator<X> and still have implements Iterable<X>. But if Iterator<X> had had a getter with signature List<Iterator<X>> get foo then we wouldn't (realistically) be able to allow CAsIterable to implement that member with a getter with signature List<CAsIterator<X>> get foo.

The underlying issue is that the ability to "lift" a signature in this sense amounts to a limited amount of auto-boxing (so the boxed object would implement foo as A get foo => _wrappedOnTypeInstance.foo.box;), and we probably don't want to introduce auto-boxing in anything but a very small number of situations. In particular, we don't want to create a new List<Iterator<X>> and then populate it with a boxed version of each of the elements in the given List<CAsIterator<X>>.

In other words, the investigation of the situation that gave rise to #2150 has shown that it is at least difficult to generalize the approach that we've discussed here (which could actually be used with for-in statements) to a more wholesome language mechanism.

[mit-mit]

Filed feature issue #2727 for this. Please continue the discussion there.