Structs and extension structs as a solution for data classes and wrapper-less views

[leafpetersen]

This is a meta-issue to collect up discussion around a proposal I've landed here. This issue is intended to capture broad discussion about the approach. Specific points of discussion around the proposal should be filed as separate discussion issues: I will link the major discussion points in here as appropriate.

Summary

The core of this proposal is to explore an approach to supporting data classes and wrapper-less views while hewing as closely as possible to the syntax and semantics of existing Dart constructs. A secondary goal is to support object representations that are more amenable to compiler optimization, having predictable space layout, loosely specified identity, and limited polymorphism.

The approach here has the user-facing goal that for a consumer of APIs, the basic affordances of a declaration should be obvious to anyone familiar with classes (i.e. what assignments are allowed, and what methods are available). Quoting from the introduction of the proposal:

We aim to minimize the differences between structs and classes. As much as
possible, structs should behave as restrictions of classes, and extension
structs should behave as further restrictions of structs. We specifically aim
to avoid as much as possible having different behaviors for the same concept
(e.g. differences in scoping).

We also aim to minimize the amount of new syntax required, and to maximize the
amount of new functionality that we can provide relative to the syntactic real
estate consumed, and the new cognitive load imposed on users.

Note that while this proposal covers two separate feature sets, they are not necessarily tied together. We could choose to ship one without ever shipping the other, or we could choose to split them into unrelated features. Even if we ship both as designed here, we could (and likely would) choose to ship the pieces separately given existing timelines and plans.

Discussion issues

• Should structs and extension structs be tied together? #2361
• Should structs be immutable? #2362
• What declaration syntax should structs and extension structs use? #2363
• Primary constructor on classes #2364
• Should primary constructors for structs use named parameters, positional parameters, or allow either? #2365
• Should structs support sub-classing? #2367
• Should extension structs support sub-classing? #2368
• Should extension structs allow overrides in sub-classes? #2369
• Should extension structs be allowed to define methods also listed in their interfaces? #2370
• Should we allow extension structs with multiple fields? #2371
• What generated methods should structs provide? #2372
• Should non-virtual methods be generalized to classes? #2400

[leafpetersen]

cc cc @mit-mit @lrhn @eernstg @chloestefantsova @johnniwinther @munificent @stereotype441 @natebosch @jakemac53 @rakudrama @srujzs @sigmundch @rileyporter @mraleph @mkustermann

[Cat-sushi]

Note that while this proposal covers two separate feature sets, they are not necessarily tied together.

My simple question is why are those two separate feature sets discussed in the same issue/ proposal?
Is it mean that the views are limited for the structs dussed heare?

[leafpetersen]

My simple question is why are those two separate feature sets discussed in the same issue/ proposal?

I (and others) have concerns about the growing surface area of the language. Each additional feature we add is a source of cognitive load on the user. One of my goals in this proposal was to try to reduce this cognitive load, by making the new features largely be restrictions on existing features. So the idea would be that structs are restrictions on classes, and extension structs are restrictions on structs. This is essential, and I think I've somewhat convinced myself that it would be better to build both of them separately off of classes (e.g. view class + data class + general primary constructor), but the underlying goal remains. And even if we end up going that way, we will have gotten there via this exploration, so perhaps it was not entirely fruitless? :)

[munificent]

I hope this is the right place to post this. I've been thinking about the proposal and the discussion to generalize primary constructors to be allowed in classes too (#2364). I strongly want the latter. I think it would honestly provide more value to more users than anything else in the struct proposal. And I think it's entirely doable.

Here's a strawman then of a couple of other features we could compose that I think might cover the various use cases:

1. Add support for primary constructors to classes. This gives you a nice notation for defining a constructor and a set of fields initialized by it.

2. Allow ; instead of {} for empty class bodies. It's a tiny bit of sugar but is consistent with empty constructor bodies.

3. Allow using struct instead of class. This defines a nominal type that opts out of reference semantics (meaningful stable identity) and into value semantics (equality is field equality). Concretely it means:

   • The fields induced by the primary constructor are implicitly final.
   • Any fields explicitly declared in the class must be final.
   • You get synthesized implementations of hashCode, == defined in terms of the fields.
   • Calling identical() may return false if passed two references to the same value. The compiler is free to dematerialize values such that "same value" isn't tracked. identical() may return true if passed two structs that came from different constructor calls but have the same type and identical fields. The compiler is free to canonicalize equivalent structs.
   • Structs can't be used with Expando, WeakReference, etc.
   • Probably some kind of copyWith() method too.

   I really like the idea of having a single keyword that gives you a big bundle of commonly-shared functionality with maximum brevity.

4. Allow some sort of wrapper-less "view type" construct. This lets you create a new static type whose underlying representation is a single value of some other type. You can define methods on it which are dispatched statically.

5. Support capability modifiers on types. Let you define classes and structs that can't be extended, implemented, and/or mixed in.

Here's how those compose for the various use cases:

• Easier to define state classes. Stateful Flutter widgets need a corresponding state class. This class is basically just a nominal type that contains some data. It doesn't have value semantics—it's got normal reference object identity. For this, just use a primary constructor on a class.

• Value types. You want an aggregate type with value semantics—think vectors, points, colors, matrices, etc. You don't care about identity and want the compiler to have the freedom to optimize them since they are likely heavily used. For these, use struct. You may or may not choose to use a primary constructor depending on how they get created.

• "Data classes". That term means different things to different people, but I think the above two use cases capture all of this.

• Units of measure. You want user-defined numeric types for units of measure. You don't want them to be implicitly convertible to and from numbers because that can lead to invalid unit conversions. You don't want the overhead of boxing each number in an instance of a class. This is a view on a number type.

• JS interop. You want to hide the primitive types returned from JS behind new types that provide more useful Dart behavior. These are view types on the JS DOM classes.

• Inline allocation. You want an aggregate type that can be reliably allocated directly on the stack, in a contiguous array, or inline in a surrounding object. This means knowing the exact size of it statically and maybe that it doesn't need a vtable. This is a struct type marked base and closed so that it can't be polymorphic.

This is pretty close to the existing view and struct proposals, but I'm trying to tease out primary constructors, struct, and identity and then see how they compose back together. What do you think?

[ds84182]

• Inline allocation. You want an aggregate type that can be reliably allocated directly on the stack, in a contiguous array, or inline in a surrounding object. This means knowing the exact size of it statically and maybe that it doesn't need a vtable. This is a struct type marked base and closed so that it can't be polymorphic.

@munificent From some of the discussion in other issues, I think it is expected that structs do not allow for polymorphism. But here, it says that polymorphism is the default behavior and you have to (very explicitly) opt out. What I'd like to understand is why a polymorphic value type is needed, as it would always need to be boxed (therefore having reference semantics).

The only case of limited struct polymorphism I can think of is a Rust-style tagged union, but this would be accomplished by switch struct. e.g.

pub enum Shape {
  Rect { lt: Point, rb: Point },
  Circle { center: Point, radius: f64 },
  // ...
}

switch struct Shape;
struct Rect(Point lt, Point rb) extends Shape;
struct Circle(Point center, double radius) extends Shape;

My current understanding of type modifiers is that it matches Dart's "open by default" model by making restrictions opt-in. For structs I think the opposite should be true: restrictive by default, opt into openness. I would not expect extends, implements, etc. to be able to work on an arbitrary struct as it would directly conflict with it being a value type.

[munificent]

What I'd like to understand is why a polymorphic value type is needed, as it would always need to be boxed (therefore having reference semantics).

== and hashCode are instance methods, so to use a value type as a Map key, there will be some polymorphism involved. (In principle, if the Map's key type is that struct type, the compiler could theoretically do specialization and devirtualize those calls, but I don't think we have any plans to go that way yet.)

You might want a value type representing a vector to implement Comparable so that it can be put in a sorted collection.

I don't think boxing is a huge problem. Yes, you might need to box the value type when it ends up in a collection or in a variable whose static type is some interface. That's OK. I think the important part is that when you unbox it later, you're allowed to get a value back that does not guarantee that it will be identical() to the original object you put in.

But I'm also not an expert on the use cases some users have around extremely lightweight value types in Dart, so there may be a subtlety I'm missing.

[ds84182]

I don't think boxing is a huge problem. Yes, you might need to box the value type when it ends up in a collection or in a variable whose static type is some interface. That's OK. I think the important part is that when you unbox it later, you're allowed to get a value back that does not guarantee that it will be identical() to the original object you put in.

@munificent I think this is why I misinterpreted the "Inline allocation" portion. I don't think the ability to have a contiguous array of a struct is related to polymorphism and vtables as outlined in your first comment. It's instead related to variance. The difference between a List<T> and an Array<T> is that the list is covariant, the array is not. By preventing an Array<Vec3> from becoming an Array<Object> you can completely eliminate the need for boxing when used in a covariant situation (as there won't be one).

This reminds me of the various issues with getting value types into Java. Lots of really good info here: https://openjdk.org/projects/valhalla/ (current) and https://openjdk.org/projects/valhalla/legacy (for older prototypes and designs)

As for extending structs, extending would completely prevent unboxing EXCEPT in the case where you have a sealed struct.

But I'm also not an expert on the use cases some users have around extremely lightweight value types in Dart, so there may be a subtlety I'm missing.

For Flutter there are many struct-ish types like Offset, Color, Rect, etc. that are simple wrappers around primitive fields. My idea of a fast and efficient value type is one where these lightweight structs can be inlined into object allocations and placed within contiguous arrays. A class containing an Offset field should desugar to two double fields, which can then be unboxed in the VM.

For other lightweight value types, there are vector_math's vector types and dart:ffi's Pointer type.

FWIW, you can somewhat emulate unboxed value types on the VM today with some very careful code. While mostly performant it isn't fun to write.

[Levi-Lesches]

@munificent, I think it would be very beneficial for users to receive these as separate features as you describe. Specifically, primary constructors and structs cover various use-cases where classes are fine but boilerplate-y and can use optimization.

3. Allow using struct instead of class.

While a struct isn't the same as a class, it seems more like a subset of a regular class (at least intuitively). Would it make sense to make struct a modifier, like struct class or data class? #1659, #349, and #314 show that users tend to think of data instead of struct since the use-case is for data classes, while still generally expecting to see the class keyword to show it's still a class. (This point is also relevant for #2363, but I wanted to point it out specifically in the context of "rebranding" the struct proposal.)

• Easier to define state classes. Stateful Flutter widgets need a corresponding state class. This class is basically just a nominal type that contains some data. It doesn't have value semantics—it's got normal reference object identity. For this, just use a primary constructor on a class.

Small detail, but I think this will probably help Widgets a lot more than States. State objects usually care a lot more about methods than they do fields, while StatefulWidgets are the container for the actual state. There's even a lint about not passing values to a State's constructor: no_logic_in_create_state, and any fields they do declare are initialized in initState, not the constructor. The primary constructor would be very helpful for StatefulWidgets and StatelessWidgets though, as those are usually 50% fields/constructors, 50% build. IMO, that makes primary constructors even more attractive, as many users leave state management to packages like provider and use mainly StatelessWidgets.

[Cat-sushi]

I'm sorry, my thought is not organized, but I feel we need some issue analysis.

[AlexanderFarkas]

Any update on this? Is the proposal discarded?

[eernstg]

The feature specification about inline classes contains many elements from this proposal, covering the semantic space which was described as 'views' in this issue. Records support the notion of being a value, in the sense that records are immutable and their equality is based on pairwise getter equality ((a, b) == (c, d) iff a == c and b == d), and they do not have a guaranteed/stable object identity (which means that a compiler is allowed to allocate a record in a few registers, if possible, which could be a lot faster than storing it in the heap).

So there are many elements from here which are already covered in other language features.

One missing part is a concise syntax for data classes. The main issue for that is #314, and other issues can be found by looking for the label data-classes .

[Cat-sushi]

I'm looking forward to seeing this at the CHANGELOG.md.