Allow fields in traits that map to lvalues in impl'ing type

[nikomatsakis]

UPDATE: The finishing touches and final conversation on this RFC are taking place in a dedicated repository, as described in this comment.

---

The primary change proposed here is to allow fields within traits and then permit access to those fields within generic functions and from trait objects:

• trait Trait { field: usize }
• Implementing type can "direct" this field to any lvalue within itself
• All fields within a trait or its supertraits must be mapped to
  disjoint locations

Fields serve as a better alternative to accessor functions in traits. They are more compatible with Rust's safety checks than accessors, but also more efficient when using trait objects.

Many of the ideas here were originally proposed in #250 in some form. As such, they represent an important "piece of the puzzle" towards solving #349.

cc @eddyb @aturon @rust-lang/lang

Rendered view.

Planned edits

• Adopt let syntax for declaring fields in traits and impls. (decided against this)
• Clarify that there are "private type in public API" rules that apply to the type of a field.
• Decide whether mut is required for mutable access. (At minimum, add as an unresolved question.)
• Typo

[oli-obk]

#1215 also addresses the borrowck problem with accessors, but still has the poor performance issue and there wasn't any talk about traits (yet). The suggestion was some kind of explicit disjoint partitions (e.g. struct field names) that can be mentioned in references to allow partial borrows.

[ticki]

Another possibility is to declare "fields" (accessors) using a syntax similar to variable declaration:

trait Trait {
    let field1: Type1;
    let field2: Type2;

    fn foo();
}

This also signifies that it isn't really a field, but more an associated value. It also looks nicer in impls.

[oli-obk]

this syntax is mentioned 10 lines below.

[ticki]

Oh, well. I missed that.

[nikomatsakis]

Another possibility is to declare "fields" (accessors) using a syntax similar to variable declaration:

trait Trait {
    let field1: Type1;
    let field2: Type2;

    fn foo();
}

This also signifies that it isn't really a field, but more an associated value.

Yeah, I don't hate this. It might be better, all things considered. I
like having all trait items start with a keyword, probably helps us
with future expansions of the syntax, and avoids the awkward , vs
; quesiton.

[ticki]

It also has a symmetry to type.

[ticki]

There is a typo here.

[nagisa]

<x as Trait<U, V>>.f?

EDIT: whoops.

[eddyb]

x is not supposed to be a type, but an expression, so that form wouldn't necessarily be unambiguous.

[nagisa]

Oh wow, I actually came to like the proposed syntax.

[DanielKeep]

Wouldn't the obvious equivalent be (x as Trait<U, V>).f?

[eddyb]

No, because casts are always rvalues and as at the type level is a completely unrelated construct.

[Manishearth]

Might this screw up the grammar? I'm thinking of ambiguities with 1. < foo::CONST >.

[arielb1]

@Manishearth

No. That works at the tokenizer level.

[nagisa]

s/wrote/write/

[Ericson2314]

To be clear, the paren restriction could be removed, as no trait can have a field and method with the same name, right?

[eddyb]

Same for calling direct fields vs methods, if we can properly disambiguate at type-checking time.

[Jexell]

You could also perhaps give either fields or methods a priority and introduce a lint.

[notriddle]

To be clear, the paren restriction could be removed, as no trait can have a field and method with the same name, right?

Why not?

[Nashenas88]

If we do leave it in, is this the right place to mention that the existing help span should be extended? There's one in place now that will recognize a field is a closure and rewrites your code with the parenthesis as a suggestion.

[petrochenkov]

Type::field/<Type>::field/<Type as Trait>::field is a very attractive syntax for field projection functions (aka pointers to data members):

struct Person { age: u8 }
let opt_age: Option<u8> = opt_person.map(Person::age);

It would be nice to preserve it while selecting syntax for this RFC.

[Ericson2314]

I like this a lot---it's broadly useful even outside the realm of "virtual structs", and very orthogonal to the listed future work, which are the good smells for this sort of thing.

[Kinda off topic] Another route of generalization is "first class lvalues". I don't know what his would look like, but it might be useful wrt things like map entry API. Teaching the borrow checker that entries for disjoint keys are disjoint would be neat. This is absolutely out of scope for this RFC, but if this is accepted, it opens the door to further exploration in that direction---great!

[futile]

This section contains references to types such as Container and Circle which are not mentioned in the RFC anywhere else. Probably a left-over from previous revisions?

[petrochenkov]

How this is different from

impl Trait for Type {
 fn private_field(&mut self) -> &mut FieldType { &self.private_field }
}

from the privacy point of view?
I wouldn't expect private-in-public checks to be applied to these fields at all.
(Type of the field should be checked though.)

[nikomatsakis]

@petrochenkov

How this is different from [a fn that references the field]

It feels different to me because I am projecting out the field itself. Thus I have effectively made the field public. In contrast, with a wrapper function, I am projecting out a method that mediates access to the underlying field.

It is roughly the same argument as with an associated type. Both field mappings and associated types give people another way to name something that already exists. Therefore, we have to respect the privacy rules on that underlying thing. In contrast, defining a method defines a new thing which is implicitly public.

If we had "function aliases", where you just map a function directly to another (rather than wrapping it), I would expect the privacy rules to be stricter there as well.

Type of the field should be checked though.

I guess I did not say that explicitly, but I agree.

[rkjnsn]

@nikomatsakis I disagree. That there is a field of a given type is part of the public interface exposed through the trait, but I would argue where in the struct it is stored, and what it is named there, could easily be considered an implementation detail. I want to have the flexibility to organize my struct (including grouping fields into member structs) without breaking the public interface, which I can't do if the fields are forced to be public.

[nikomatsakis]

@rkjnsn

I want to have the flexibility to organize my struct (including grouping fields into member structs) without breaking the public interface, which I can't do if the fields are forced to be public.

This is a strong argument. I've been somewhat reconsidering this point. One difference between the case of fields and the case of associated types is that associated types are true synonyms and normalized, so if we allowed private types to appear in associated type bindings, then we would effectively allow instances of those types to escape into generic functions, which they are not supposed to do.

With fields, somewhat like methods, so long as the type of the field is public, you are only allowing those parts of the value to escape that are designated in the trait. (This was @petrochenkov's original point as well, I believe).

[Manishearth]

Note that as an inheritance system this is not useful for Servo's DOM. This is just a datapoint, not a reason to block it 😄 Otherwise big 👍 , sounds useful.

[petrochenkov]

Any plans to incorporate mutability in this RFC?
E.g. if I want to emulate a getter function with a trait field, then I wouldn't want this field to be mutable.
Maybe these fields should even be immutable by default.

trait Tr {
    x: u8, // Immutable field access, "getter"
    mut y: u8, // Mutable field access, "setter/getter"
}

[eddyb]

@petrochenkov That would be great with fields in inherent impls!
It would mean you can have a private field with an immutable public view in an impl, so you can mutate it, but not anyone else.

[jFransham]

Big 👍 on this, I personally would prefer that trait fields are not immutable by default, but that mutability is controlled by whether or not the concrete value is mutable. I think if we can work out a general-case disjointness checking solution (the syntax that comes to mind is fn modstigate(self: &Self { field_a }) -> &TypeOfFieldA) then we could keep the semantics of trait fields and struct fields consistent, and use privacy to control field mutability on both (like we do already).

[kylone]

@jFransham Just to be clear, "use privacy to control field mutability" means using Cell & RefCell in std::cell, right?

[pnkfelix]

@Manishearth

Note that as an inheritance system this is not useful for Servo's DOM

Just to be clear: would something like the extension described in the Embedded notation and prefix layout section do more to address Servo's use case?

Or do you more broadly mean that we need all the things list in the Other changes section that followed that?

Or do you mean that there is something Servo needs that is not addressed by the items enumerated there?

[Manishearth]

@pnkfelix embedded notation works pretty well.

[golddranks]

@jFransham I don't quite understand what you're saying, but traits are basically interfaces. You code against an interface. If you mean by "concrete value" a field in the struct that implements a trait, how could you code generically against the interface, without the trait saying whether the value is mutable or immutable? You can't.

[eddyb]

@golddranks The mutability depends on whether the value implementing the trait is in a mutable slot or not. This is already the case, e.g. if you write accessors, you can call setters that take &mut self only if the value is in a mutable slot.

[golddranks]

@eddyb Ah, I see. Pardon my ignorance.

[nikomatsakis]

@petrochenkov

Any plans to incorporate mutability in this RFC?

I did not have any such plans. It's an interesting thought. I sort of wish we declared fields as mut as well, in which case this would be consistent. But we don't.

I think I would be more in favor if we also planned to add a mut keyword to ordinary fields and then to lint against assigning directly to a field (or a projection from a field) unless it is declared as mut. Put another way: you would be able to assign to x.f if (a) x is mutable, as today and (b) f is mutable. If (a) is violated, you get an error (as today). If (b) is violated, the code compiles, but you get a lint warning. Obviously the rules would still be mildly different then but if you followed "best practices" it would be consistent.

[nikomatsakis]

@Manishearth

Note that as an inheritance system this is not useful for Servo's DOM. This

I think you and @pnkfelix hashed this out, but to be clear: as the RFC states, this RFC alone is not intended to solve Servo's "DOM problem", but it is a major building block towards doing so. The section on future work lays out the additional steps that I believe would be needed to make a truly ergonomic DOM implementation. If you think anything else is required, it'd be good to speak up. =)

[Manishearth]

Yeah, understood. I personally don't really feel Servo needs a better DOM solution at this stage (If it exists, sure, we'd use it, but I don't want to push for it). I'm happy with our current set of hacks, and the only (minor) improvement I'd like would be some layout-guarantees so that the transmutes aren't technically UB.

I was just pointing out that if the motivation behind this was partially due to use cases like Servo's DOM, it doesn't apply cleanly there.

Basically, Servo would need cheap upcasting. I think with this proposal you need to use trait objects to get that effect, if it is even possible.

[nikomatsakis]

I don't have time to edit the draft now, but I've added a "planned edits" section to the main area.

[kbknapp]

Huge 👍 I've wanted this for a while but there's no way I could have articulated it that well!

[nikomatsakis]

OK, so, I've been thinking about this RFC for some time, but each time I start to write a comment in an effort to get it "restarted", I wind up getting side-tracked. Therefore, I am taking a different tack. I've created a custom repo dedicated to this RFC. I've created issues for all the major comments I saw in the thread, and tagged them somewhat with my estimate of their priority:

https://github.com/nikomatsakis/fields-in-traits-rfc

I would love to get feedback, particularly on the P-essential issues.

I think at this point I've basically decided to adopt the following:

• let syntax (nikomatsakis/fields-in-traits-rfc#3)
• non-interior fields (nikomatsakis/fields-in-traits-rfc#4)
• shared-only fields (nikomatsakis/fields-in-traits-rfc#5)
• permit impls to map to private fields (nikomatsakis/fields-in-traits-rfc#6)

There were some objections from @rust-lang/lang members to adopting let syntax, but I think that on this thread the idea has been universally popular. If you think it's a bad idea, though, I encourage you to comment in nikomatsakis/fields-in-traits-rfc#3.

[crumblingstatue]

#275 might be a good companion for this, as it would allow private trait fields that could be used by the underlying implementation of provided methods, but not accessible by users, allowing better control of invariants.

[alexreg]

No progress on this still...?

[nikomatsakis]

I am nominated this RFC to discuss in the @rust-lang/lang meeting. In particular, I'd like to decide how it fits in our overall priorities and what we ought to do with it.

I also owe some updates (well, perhaps on the dedicated repo) so discuss a few shortcomings of the RFC that I think still need to be addressed, but hopefully that'll come soon.

[nikomatsakis]

@rfcbot fcp postpone

OK, after some discussion in the @rust-lang/lang meeting, it seemed clear that while we are still interested in a change like this, we don't have the bandwidth to push this through right now, so we're going to postpone the change.

I'm going to keep the repo open and also try to catch up on the conversation there and highlight some of the most important open questions though.

If anyone is up for it, I'd be interested in working closely with someone to keep the design going, so we can perhaps revisit it once things are calming down.

[rfcbot]

Team member @nikomatsakis has proposed to postpone this. The next step is review by the rest of the tagged teams:

• @aturon
• @cramertj
• @eddyb
• @nikomatsakis
• @nrc
• @pnkfelix
• @withoutboats

No concerns currently listed.

Once these reviewers reach consensus, this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[kevincox]

If anyone is up for it, I'd be interested in working closely with someone to keep the design going, so we can perhaps revisit it once things are calming down.

I don't know exactly what this would entail but I have run into multiple use cases for this and would be glad to work on it as a new rust contributor.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[nikomatsakis]

@kevincox can we maybe schedule some time to chat? reach out over gitter or IRC or e-mail maybe?

[rfcbot]

The final comment period is now complete.

[rfcbot]

The final comment period is now complete.

[aturon]

Closing as postponed; activity should happen on the dedicated repo.

[kevincox]

I created https://internals.rust-lang.org/t/fields-in-traits/6933 to further discuss use cases. I started it off with some broad categories but would be interested in hearing more, and more specific use cases.