Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enum variant types #2593

Closed
wants to merge 14 commits into from
Closed

Enum variant types #2593

wants to merge 14 commits into from

Conversation

@varkor
Copy link
Member

@varkor varkor commented Nov 10, 2018

Enum variants are to be considered types in their own rights. This allows them to be irrefutably matched upon. Where possible, type inference will infer variant types, but as variant types may always be treated as enum types this does not cause any issues with backwards-compatibility.

enum Either<A, B> { L(A), R(B) }

fn all_right<A, B>(b: B) -> Either<A, B>::R {
    Either::R(b)
}

let Either::R(b) = all_right::<(), _>(1729);
println!("b = {}", b);

Rendered

Thanks to @Centril for providing feedback on this RFC!


Current status of this RFC: #2593 (comment).

@varkor varkor mentioned this pull request Nov 10, 2018
@Centril Centril added the T-lang label Nov 10, 2018
@alexreg
Copy link

@alexreg alexreg commented Nov 10, 2018

Great work, @varkor. I've been looking forward to this for a long time. Just as a side-point, I'd love to follow this up with an RFC for the ideas in https://internals.rust-lang.org/t/pre-rfc-using-existing-structs-and-tuple-structs-as-enum-variants/7529 once this gets implemented in nightly (or perhaps even before). Since you've worked on this, would appreciate your thoughts at some point.

@bchallenor
Copy link

@bchallenor bchallenor commented Nov 10, 2018

Although sum types are becoming increasingly common in programming languages, most do not choose to allow the variants to be treated as types in their own right (that is, the author has not found any that permit this design pattern).

This is possible in Scala - as in your example, Left and Right are subtypes of Either, and can be referred to independently. Coming from Scala, I miss this feature in Rust, and I am fully in favour of this RFC.

@varkor
Copy link
Member Author

@varkor varkor commented Nov 10, 2018

This is possible in Scala - as in your example, Left and Right are subtypes of Either, and can be referred to independently.

Ah, great, I'll add that in, thanks!

@Ixrec
Copy link
Contributor

@Ixrec Ixrec commented Nov 10, 2018

I think I'm in favor of the proposed functionality and semantics here. Where I'm stumbling is the nomenclature/terminology/teachability(?); it's not clear to me that "introducing a new kind of type: variant types" is the best description of this. In particular, precisely because this proposal feels so lightweight compared to previous ones, it doesn't really "feel" like what we're doing is adding a whole new type kind the way structural records or anonymous enums would be doing. It sounds like it could be equally well described as doing the "duplicating a variant as a standalone struct" workaround automagically, so those extra structs are just always there (except they get a specific layout guarantee and different conversion syntax that regular structs wouldn't get). Is there some detail I overlooked that makes this clearly not a sugar?

I'm guessing this is at least partially ignorance on my part because

The loose distinction between the enum type and its variant types could be confusing to those unfamiliar with variant types.

makes it sound like "variant types" are an actual thing with their own special properties that no other kinds of types have, and I just have no idea what that would be (since being autogenerated, having a certain layout guarantee and different conversion syntax seem like "surface level" properties that aren't really part of the type system per se). Maybe I just need to see some more examples of how these types behave?

@leonardo-m
Copy link

@leonardo-m leonardo-m commented Nov 10, 2018

Nice RFC.

In all cases, the most specific type (i.e. the variant type if possible) is chosen by the type inference.

Is code like this still allowed, or is the compiler going to tell me that the Sum::B(b) branch of the match is impossible and needs to be removed?

enum Sum { A(u8), B(u8) }
let x = Sum::A(5); // x: Sum::A
match x {
    Sum::A(a) => {},
    Sum::B(b) => {},
}

Both options have advantages and disadvantages.

@varkor
Copy link
Member Author

@varkor varkor commented Nov 10, 2018

Is code like this still allowed, or is the compiler going to tell me that the Sum::B(b) branch of the match is impossible and needs to be removed?

This is a good question — I'll make note of it in the RFC. Although matching on variant types permits irrefutable matches, it must also accept the any other variants with the same type — otherwise it's not backwards compatible with existing code.

Where I'm stumbling is the nomenclature/terminology/teachability(?); it's not clear to me that "introducing a new kind of type: variant types" is the best description of this.

since being autogenerated, having a certain layout guarantee and different conversion syntax seem like "surface level" properties that aren't really part of the type system per se

It's quite possible there's a better way to explain this. They are essentially as you say, though they act slightly differently from structs (on top of the points you made) in the way they are pattern-matched (as above in this comment) and their discriminant value. I thought it would be clearer to describe them as an entirely new kind of type, but perhaps calling them special kinds of structs would be more intuitive as you say. I'll think about how to reword the relevant sections.

@varkor varkor force-pushed the enum-variant-types branch from 14a6e83 to 2b00420 Nov 10, 2018
@nrc
Copy link
Member

@nrc nrc commented Nov 10, 2018

This was previously proposed in #1450. That was postponed because we were unsure about the general story around type fallback (e.g, integer types, default generic types, etc). Enum variants would add another case of this and so we wanted to be certain that the current approach is good and there are no weird interactions. IIRC, there was also some very minor backwards incompatibility.

This RFC should address those and issues, and summarise how this RFC is different to #1450.

For the sake of completeness, an alternative might be some kind of general refinement type, though I don't think that is a good fit with Rust.

I'm still personally very strongly in favour of this feature! The general mood on #1405 was also positive.

@eddyb
Copy link
Member

@eddyb eddyb commented Nov 10, 2018

For the sake of completeness, an alternative might be some kind of general refinement type, though I don't think that is a good fit with Rust.

I'm not sure that would need to be at odds with variant types, if Rust ends up with refinement types I expect variant types to be refinements of their enum.

@Centril
Copy link
Contributor

@Centril Centril commented Nov 10, 2018

First, irrespective of what happens with the RFC;
I always appreciate the effort put into well thought out RFCs and this is one of those, so thank you!

I am of two minds and a bit torn about the proposal here.

  1. I think it would help immensely to make an API such as syn::Expr more ergonomic and avoid auxiliary structs such as syn::ExprBox. Here you don't need any implementations on the variant types you'd get because the variant types are for the most part just dumb data.

  2. At the same time, precisely because this RFC does not permit implementations on variant types, as I think is proper to avoid the pitfalls of Java-OOP-inheritance APIs, it will not allow you to refactor an API such as syn::Lit into one where syn::LitStr is a variant type (i.e. Lit::Str) because the implementations there would not be admitted by the type checker.

  3. All in all, I think the benefits of this proposal are well motivated and the costs in terms of understanding are not that great. I think this proposal is something that a subset of users would naturally expect; the user also doesn't have to do much, expressiveness is given for free to them.

  4. The RFC interacts well with #1806 as well as goals and plans for gradual struct initialization; in fact, it provides the "missing link" that makes gradual initialization more generally applicable in the type system. This is a wonderful thing.

  5. Thus on balance I think the RFC is a good idea.

(Feel free to integrate any points that you found relevant into the text of the RFC)


@nrc

This RFC should address those and issues, and summarise how this RFC is different to #1450.

👍

For the sake of completeness, an alternative might be some kind of general refinement type, though I don't think that is a good fit with Rust.

(Aside, but let's not go too deeply into this: I personally think that refinement / dependent typing is both a good idea, a good fit for Rust's general aim for correctness and type system power for library authors -- and RFC 2000 is sort of dependent types anyways so it's sort of sunk cost wrt. complexity -- the use cases for dependent/refinement types are sort of different than the goal here; With dependent types we wish to express things like { x: usize | x < 10 })


@eddyb

I'm not sure that would need to be at odds with variant types, if Rust ends up with refinement types I expect variant types to be refinements of their enum.

I agree; I think you can think of variant types in the general framework of refinement / dependent types;
With the notation due to @petrochenkov for pattern matching as a boolean operator, we can think of variant types as:

type FooVar = { x: Foo | x is Foo::Variant(...) };

@burdges
Copy link

@burdges burdges commented Nov 10, 2018

We do want formal verification of rust code eventually, and afaik doing that well requires refinement types. I'm not saying rust itself needs refinement types per se, but rust should eventually have a type system plugin/fork/preprocessor for formal verification features, like refinement types. I do like this feature of course, but ideally the syntax here should avoid conflicts with refinement types.

@Centril
Copy link
Contributor

@Centril Centril commented Nov 10, 2018

@burdges

I do like this feature of course, but ideally the syntax here should avoid conflicts with refinement types.

Are there any such conflicts in your view?

@Centril
Copy link
Contributor

@Centril Centril commented Nov 10, 2018

(Or to elaborate; if there are any conflicts with the RFC as proposed with refinement typing, then stable Rust as is has that conflict since the RFC does not introduce any new syntax...)

@ExpHP
Copy link

@ExpHP ExpHP commented Nov 10, 2018

Note that because a variant type, e.g. Sum::A, is not a subtype of the enum type (rather, it can simply be coerced to the enum type), a type like VecSum::A is not a subtype of Vec. (However, this should not pose a problem as it should generally be convenient to coerce Sum::A to Sum upon either formation or use.)

So, if I understand correctly, all existing code that uses enums now have coercions all over the place in order to ensure they continue functioning? I'm really not sure this works...

let mut x:

x = None;

// At this point the compiler knows the type
// of x is Option<?0>::None.

// But Option<_>::Some cannot be coerced to None
x = Some(1); // type error?

@leonardo-m
Copy link

@leonardo-m leonardo-m commented Nov 10, 2018

let mut x:
x = None;
// At this point the compiler knows the type
// of x is Option<?0>::None.
// But Option<_>::Some cannot be coerced to None
x = Some(1); // type error?

I think in theory the type system should infer x to be of type Option<i32> because it sees both assignments.

But I think we need a formalization of the involved type system rules, to assure soundness, before implementing this proposal...

@mark-i-m
Copy link
Member

@mark-i-m mark-i-m commented Nov 10, 2018

I would rather frame this as follows:

  • the type is Option
  • the compiler tracks the most specific variant the value is if it can be known statically. This could be done with a standard dataflow analysis.

@leonardo-m
Copy link

@leonardo-m leonardo-m commented Nov 10, 2018

We do want formal verification of rust code eventually, and afaik doing that well requires refinement types. I'm not saying rust itself needs refinement types per se, but rust should eventually have a type system plugin/fork/preprocessor for formal verification features, like refinement types.

While I don't dislike LiquidHaskell-like refinement typing, lately for the future of Rust I prefer a style of verification as in the Why3 language ( http://why3.lri.fr/ , that is also related to the Ada-SPARK verification style). We'll need a pre-RFC for this.

@leonardo-m
Copy link

@leonardo-m leonardo-m commented Nov 11, 2018

I hope this syntax is also supported (I suggest to add it to the RFC):

enum Sum { A(u32), B, C }
fn print_a1(Sum::A(x): Sum::A) {}

A question regarding the ABI: is the print_a1() function receiving the Sum discriminant too as argument?

And in future it could also be supported the more DRY syntax (I think suggested by Centril):

fn print_a2(Sum::A(x)) {}

You could also add a new (silly) example to this RFC that shows the purposes of this type system improvement:

enum List { Nil, Succ(u32, Box<List>) }

fn prepend(x: u32, lst: List) -> List {
    List::Succ(x, box lst)
}

With this improvement you can write instead:

fn prepend(x: u32, lst: List) -> List::Succ {
    List::Succ(x, box lst)
}

Then you can define a list_head_succ() function that returns the head of the result of prepend() without a unwraps or Option result:

fn list_head(lst: List) -> Option<u32> {
    match lst {
        List::Succ(x, _) => x,
        List::Nil => None,
    }
}

fn list_head_succ(List::Succ(x, _): List::Succ) -> u32 { x }

{ x: usize | x < 10 }

For the common case of integer intervals for Rust I sometimes prefer a shorter and simpler syntax like:

type Small = usize[.. 10];

Copy link
Member

@shepmaster shepmaster left a comment

"Overhead"

I'm mostly interested in this RFC from the point-of-view of "enums of lots of standalone other types". The biggest example I have is the AST expressed in fuzzy-pickles, a Rust parser which uses this pattern extensively:

pub enum Item {
    AttributeContaining(AttributeContaining),
    Const(Const),
    Enum(Enum),
    // ...

Unfortunately, I don't see this as being a large win for such a case due to the "forced overhead" of each enum variant still being the same size as all the other variants. It's an understandable decision, just not one that I see as helping as much as it could.

This is mentioned in the alternatives section, but I want to make sure the point is reiterated.

Multiple variants

I didn't see any mention of if multiple variants would be supported:

#[derive(Debug)]
enum Count {
    Zero,
    One,
    Many(usize),
}

fn example(c: Count) {
    use Count::*;
    match c {
        x @ Zero | x @ One => println!("{:?}", x), // what is the type of `x` here?
        x => println!("{:?}", x),
    }
} 

It may also be worth explicitly calling out what the type is for those catch-all patterns as well as in cases of match guards.

foo @

This may be swerving into refinement type territory, but I naturally wanted to not type the foo @ in the previous example:

match c {
    Zero | One => println!("{:?}", c),
    // ...

I feel this is a pretty hidden and uncommon aspect of patterns, and it'd be nice to just be able to intuit the type based on the pattern without adding the explicit binding. That might even mean we could do:

if let Count::Many(..) = c {
    println!("{}", c.0);
}

text/0000-enum-variant-types.md Show resolved Hide resolved
and `impl Trait for Enum::Variant` are forbidden. This dissuades inclinations to implement
abstraction using behaviour-switching on enums (for example, by simulating inheritance-based
subtyping, with the enum type as the parent and each variant as children), rather than using traits
as is natural in Rust.
Copy link
Member

@shepmaster shepmaster Nov 11, 2018

I'm a fan of the proposed style, but it might be worth stating why Rust the language wants to dissuade this pattern.

- Passing a known variant to a function, matching on it, and use `unreachable!()` arms for the other
variants.
- Passing individual fields from the variant to a function.
- Duplicating a variant as a standalone `struct`.
Copy link
Member

@shepmaster shepmaster Nov 11, 2018

I disagree that this goal is going to be as widely achieved by this RFC as I would like due to the following point:

the variant types proposed here have identical representations to their enums

That means that if I have an enum with large variants:

enum Thing {
    One([u8; 128]),
    Two(u8),
}

Even the "small" variants (e.g. Thing::Two) are still going to take "a lot" of space.

Copy link
Member

@eddyb eddyb Nov 11, 2018

If space is a concern then we could have it so variant types only convert to their enum by-value, so e.g. a &Thing::Two wouldn't be a valid &Thing.

That's weaker than something more akin to refinement typing, but maybe it's enough?

Copy link
Contributor

@Centril Centril Nov 11, 2018

@eddyb I think that's already the case; the RFC doesn't state anywhere, as far as I can tell, that &Thing::Two is a valid &Thing. Also note that the RFC explicitly states that Thing::Two and Thing having the same layout is not a guarantee so we could change the layout to be more space efficient.

@varkor
Copy link
Member Author

@varkor varkor commented Mar 12, 2021

As far as I know, no-one is currently working on implementing a prototype of this feature. I'm still happy to offer mentoring to anyone who'd like to try doing so.

@kogbok
Copy link

@kogbok kogbok commented Mar 12, 2021

Thank @varkor for your reply, if no one is implementing a prototype of this feature I'll be happy to start with your advice. But @Jezza hasn't started a prototype ?

@Jezza
Copy link

@Jezza Jezza commented Mar 12, 2021

@kogbok Yeah, I was building on the prototype that @jugglerchris started.
It provided a very good jumping off point, but unfortunately, I didn't get as much time as I would have liked, and as a result, didn't get very far.

I'll have to step down, so I do apologise, and hope I can revisit it in the future.

@kogbok
Copy link

@kogbok kogbok commented Mar 12, 2021

Thank you @Jezza. If you have made any improvements on @jugglerchris' work I will be very happy to read them. All additions are welcome.
I'm not sure I have time to build the whole prototype, but I can take a step or two. So, if you have time in the future, there will certainly be other steps to do ;-).

@jugglerchris and @Jezza if you agree I'll start reading your changes and try to get a first feel for the work to be done.
@varkor do you have any recommendations for planning this prototype?

@jugglerchris
Copy link

@jugglerchris jugglerchris commented Mar 13, 2021

@kogbok You are very welcome as far as I'm concerned!

@varkor
Copy link
Member Author

@varkor varkor commented Mar 13, 2021

@kogbok: I will need to refresh my memory as to how far the prototype is along, but once you get the source code building (or at least as far as the current version will compile), send me a message on Zulip, and we can discuss the next steps :) I think it's a little easier to discuss in real time than on here.

@kogbok
Copy link

@kogbok kogbok commented Mar 14, 2021

Thank @jugglerchris for your reply.
@kogbok: ok, When I'm ready, I'll contact you on Zulip.

@nikomatsakis
Copy link
Contributor

@nikomatsakis nikomatsakis commented Apr 20, 2021

@rfcbot fcp postpone

Hello everyone; we discussed this RFC in our backlog bonanza. The consensus was that we that we should postpone it, as we don't think we have the bandwidth to see to it right now. We would really like to see this change go through at some point, it's obvious that there's a need for it. We would like to encourage folks to raise enum variant types when the time comes for us to discuss our upcoming roadmap (one of the procedural changes we have in mind is to make it clearer when we'd be open to bigger proposals).

@rfcbot
Copy link

@rfcbot rfcbot commented Apr 20, 2021

Team member @nikomatsakis has proposed to postpone this. The next step is review by the rest of the tagged team members:

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

@nikomatsakis
Copy link
Contributor

@nikomatsakis nikomatsakis commented Apr 20, 2021

@rfcbot fcp reviewed

I checked the names of all folks who were present in the meeting.

@rfcbot
Copy link

@rfcbot rfcbot commented Apr 20, 2021

🔔 This is now entering its final comment period, as per the review above. 🔔

@wongjiahau
Copy link

@wongjiahau wongjiahau commented Apr 26, 2021

Just trying to add some useful idea to this feature, as mentioned at #2593 (comment), users that wanted to use this feature will probably also want to be able to refine an enum to a subset of more than one variants.

In TypeScript, with just mapped types and conditional types, we can achieve the effect of restricting a sum type arbitrarily.

For example (playground link):

type MyEnum = {
  type: "A",
  a: string
} | {
  type: "B"
  b: number
} | {
  type: "C"
  c: boolean
}

type A_or_B = Extract<MyEnum, {type: "A" } | {type: "B"}>

type no_A = Exclude<MyEnum, {type: "A"}>

The other way to achieve this effect is via anonymous sum types, which is called Polymorphic Variants in Ocaml, there is a well-defined semantics to get it work with type inference, albeit tedious.
For example :

type MyEnum = [ `A of string | `B of number | `C of boolean ]

type A_or_B = [ `A of string | `B of number ]

type no_A = [ `B of number | `C of boolean ]

That being said, if we can implement mechanism that is similar to Typescript conditional/mapped types or Ocaml's polymorphic variants, we can get this feature for free without having to bloat the existing primitive mechanisms.

@cakekindel
Copy link

@cakekindel cakekindel commented Apr 27, 2021

Just trying to add some useful idea to this feature, as mentioned at #2593 (comment), users that wanted to use this feature will probably also want to be able to refine an enum to a subset of more than one variants.

In TypeScript, with just mapped types and conditional types, we can achieve the effect of restricting a sum type arbitrarily.

For example (playground link):

type MyEnum = {
  type: "A",
  a: string
} | {
  type: "B"
  b: number
} | {
  type: "C"
  c: boolean
}

type A_or_B = Extract<MyEnum, {type: "A" } | {type: "B"}>

type no_A = Exclude<MyEnum, {type: "A"}>

The other way to achieve this effect is via anonymous sum types, which is called Polymorphic Variants in Ocaml, there is a well-defined semantics to get it work with type inference, albeit tedious.
For example :

type MyEnum = [ `A of string | `B of number | `C of boolean ]

type A_or_B = [ `A of string | `B of number ]

type no_A = [ `B of number | `C of boolean ]

That being said, if we can implement mechanism that is similar to Typescript conditional/mapped types or Ocaml's polymorphic variants, we can get this feature for free without having to bloat the existing primitive mechanisms.

Hey Wong, nice suggestion! Those would be cool to see in Rust but Mapped Types are a feature of a more loose, structural type system. I would struggle to think of a way they would provide value in rust that a map (HashMap) doesn't provide.

Conditional types are interesting but I don't think they necessarily solve the problem of "I want specific variants of this enum" or "I will only ever return specific variants of this enum," because variants are not separate types right now, they're constructors of the enum they're declared in. So as far as my understanding goes (happy to be wrong!), in order for conditional types to be useful with enum variants then they would have to first be types you can refer specifically to, which is the intent of this RFC.

@rfcbot
Copy link

@rfcbot rfcbot commented Apr 30, 2021

The final comment period, with a disposition to postpone, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

The RFC is now postponed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet