Specify #[repr(transparent)]

[nox]

Summary: Extend the existing #[repr] attribute on newtypes with a transparent option
specifying that the type representation is the representation of its only field.
This matters in FFI context where struct Foo(T) might not behave the same
as T.

Cc @eddyb @Manishearth

rendered

[edited to update rendered link]

[Amanieu]

One obvious alternative would be to do this automatically for all structs that have just a single field.

[nox]

It is done automatically for all newtypes, but then you can't pass that to FFI stuff without it becoming a struct on the C side too.

[nrc]

Is there a tooling solution here? Could a sufficiently smart bindgen work around this problem?

[nox]

No.

[nrc]

could you explain why please?

[nox]

Because there is no tooling that can be invented to do something that is not possible without this attribute. Either you do struct Foo(f64) and it can't be used in FFI, or you do #[repr(C)] struct Foo(f64) and it won't behave as a bare f64 in FFI.

[sfackler]

It seems like something like bindgen could pretty easily create wrappers that unbox transparent structs.

[nox]

This means an additional type, additional transmutes and/or additional boxing/unboxing functions. That's code bloat to me and I don't see why I would want that. It also doesn't help for structures with UnsafeCell<T> fields.

[nikomatsakis]

It seems like something like bindgen could pretty easily create wrappers that unbox transparent structs.

This would mean that the fields can't be private, however. Not sure how much that matters in practice.

[Amanieu]

I feel that the RFC could do with a note about transmute, for example specifying that transmute<TransparentNewtype, f64> should be a no-op since the two types have the same representation.

[Amanieu]

I can also see this attribute being useful for atomic types such as AtomicUsize, which RFC #1649 specifies should have the same representation as the underlying type. This attribute could also be added to Cell and UnsafeCell for the same reasons.

[nox]

This attribute could also be added to Cell and UnsafeCell for the same reasons.

Yep the whole reason I'm writing this RFC is for UnsafeCell<T> to be used in some structures that are passed to Gecko from Servo.

[nox]

@Amanieu Added a note about FFI and AtomicUsize use cases.

[nagisa]

I feel like this RFC is quite underspecified, especially around interactions with some more obscure features.

Overall the idea seems okay to me, though.

[nagisa]

I feel like this RFC is quite underspecified, especially around interactions with some more obscure features.

Overall the idea seems okay to me, though.

[nagisa]

I do not see why the (unspecified) Rust ABI {w,sh}ould share the same behaviour, as opposed to being “transparent” by default.

[petrochenkov]

+, transparent seems to be relevant only for repr(C) stuff, as a tweak to the default C ABI

[nox]

"+"?

[nikomatsakis]

I think @petrochenkov meant "and"

[nagisa]

Why not univariant unions and enums?

[nox]

I tried to be conservative for now given I don't have a use case for univariant unions and enums in FFI context.

[nagisa]

Oh, so you expect to use this when passing stuff into foreign functions… Then a multiple questions arise, for example:

#[repr(transparent)] struct Transp<T>(T);

extern { fn banana(x: Transp<SomeRustType>); } // does the non-c-types lint warn? probably yes, but not specified in the RFC
extern { fn banana(x: Transp<SomeCType>); } // does the non-c-types lint warn? probably no, but not specified in the RFC

#[repr(transparent)] struct TranspPack(PackedTy);  // is packed? Probably yes.
#[repr(packed,transparent)] struct TransPack(SomeTy); // is packed? No idea.
#[repr(transparent, align="128")] struct TransU32(u32); // is valid? If so, is alignment 128 or `align_of::<u32>()`? What happens if native alignment of u32 is greater than alignment specified for the transparent newtype?

[nox]

This RFC specifies that Transp<SomeRustType> should warn because it says the representation of Transp is whatever the representation of its single field is. So if the single field is not a proper C type, using it in FFI should warn.

But yeah, it should mention that no other repr should be used with it, so no align nor packed.

[nox]

Mentioned in the RFC that these are illegal.

[nox]

I feel like this RFC is quite underspecified, especially around interactions with some more obscure features.

I have no idea how to specify more, feel free to amend or give me directions.

[camlorn]

I agree with the people saying this should only apply to repr(C). If this is an optimization worth having, then Rust should do it automatically for repr(Rust). This doesn't sound like it would be too terribly hard to implement, though I'm not entirely sure of that offhand. Probably just adding a Newtype variant to Layout (alternatives end up in the territory of adding flags to Univariant, which feels like a hack to me).

I am working with @eddyb on optimizing struct layout, the first PR of which just merged into the compiler. While we could apply repr(transparent) to anything containing one field, it's only useful if that field also has repr(c) on its type. Otherwise, you eliminate the outer layer but may not have the fields in the order you want or expect. This would mostly match the C ABI now, but won't in future if I have my way.

This feels really hacky and temporary, and hacky/temporary-feeling things that we have a need for seem like a good time to revisit the general pattern. There is never any reason to leave repr(transparent) off a newtype that I can see, so perhaps someone should work out the semantics of newtype Foo = Bar instead of just leaving this is an ill-defined pattern that everyone eventually learns on their own.

[nox]

I agree with the people saying this should only apply to repr(C).

What? No, f64 is not #[repr(C)] explicitly but it is nonetheless usable in C functions.

If I have a newtype without an explicit representation, I can't pass it to C functions. If I have a newtype with #[repr(C)], I can't use it in place of its inner type in a C function because it may be handled differently on the ABI level.

[nox]

There is never any reason to leave repr(transparent) off a newtype that I can see, so perhaps someone should work out the semantics of newtype Foo = Bar instead of just leaving this is an ill-defined pattern that everyone eventually learns on their own.

That doesn't help to use UnsafeCell<T> in a field of a #[repr(C)] structure to signal that the C side of the code may change this field, which is what is needed to make Gecko and Servo work hand in hand in some cases.

[durka]

What about extra zero-sized fields? Out of scope?

#[repr(transparent)]
struct Wrapper<T: SomeKindaMarker> {
    wrapped: i32,
    _ghost: PhantomData<T>
}

[eddyb]

@camlorn This is exactly the newtype unpacking optimization I had in mind, which requires writing a not-entirely-recursive algorithm to handle cycles. It's pretty nasty stuff and I was waiting for old trans to die.
@durka At least for the Rust default, I would support that, too, since it affects ~all smart pointers.

For everyone else: there's some confusion here: #[repr(C)] is not the requirement of the FFI warning.
It's "you must only use types which are either some primitive with a cross-ABI representation or an user-defined type marked with #[repr(C)]". This RFC expands that rule to recurse into newtypes marked with #[repr(transparent)].

The reason that doing it automatically is not enough, and which @nox should most definitely make it the prime motivation of this RFC is that depending on it leaks implementation details by default.

If I have:

pub struct Opaque(T);

Then why should Opaque be usable with FFI if T is a primitive, or some #[repr(C)] struct?
Why should that detail be exposed and depended upon, if that field is private?

This is why you need #[repr(transparent)], to indicate that ABI properties of the wrapped private type, although not the type itself, are exposed by the public structure.

[camlorn]

So shouldn't we just do a proper newtype, then say that repr(transparent) is only allowed on newtypes?

If things like UnsafeCell need to be exposed through this method, then we change them to newtype in the standard library and put repr(transparent) on them. This shouldn't cause a backwards compatibility problem. As things stand, the case of UnsafeCell even being a newtype isn't documented by the std docs,

My problem here is that it feels like this takes a step in the direction of enshrining the single-field struct as a newtype forever. This is workable if we then do stuff like accept #1406 in some form, which may have some uses anyway. But it feels like this is one of those decision points: accepting this without a dedicated newtype makes it more likely that we never will.

Also, unless it's already named, I hereby define FFI-capable type to refer to any type which can be safely used with the C FFI. Run with it or not as you like.

[eddyb]

@camlorn But these types with private fields are not newtypes in the classical sense: it's not just a type disambiguator, but the field is opaque modulo the ABI implications of #[repr(transparent)] (if present).

"FFI-capable" sounds good, cc @steveklabnik.

[nagisa]

@nox

No, f64 is not #[repr(C)] explicitly but it is nonetheless usable in C functions.

f64 et al are POD and none of the repr stuff really applies to it.

---

@eddyb

leaks implementation details by default

Sure, nobody ought to rely on default behaviour if it involves repr(Rust) stuff, whereas repr(transparent) mandates a concrete behaviour, similarly to how repr(C) does…

@nox

That doesn't help to use UnsafeCell in a field of a #[repr(C)] structure to signal that the C side

…that being said, @camlorn’s critique seems very valid to me. There’s no reason why some alternative approach (e.g. newtype Peach<T> = Banana<T>?) couldn’t be functionally equivalent to #[repr(transparent)] struct Peach<T>(Banana<T>);.

At this point I would like to note that adding (in some cases) and removing (in all cases) repr(transparent) would become a breaking change. However, it being a plain attribute (OTOH, repr(C) shares the same problem) does not really signal that weight.

Interestingly, AFAIR rustdoc does not in any way make note of any representational particularities. These are part of the API and probably should be prominently presented to everyone interested. Perhaps fixing that would help to add that weight I was speaking of in the previous paragraph.

---

Bikeshed time. repr(newtype). Yay? Nay? Less to type, is clearer.

[arielb1]

@nagisa

You mean an ABI-breaking change? Isn't that the entire point of the repr attribute?

@eddyb

repr(Rust) is intentionally unspecified (but should be equivalent to repr(transparent) when that is valid). We want to hide that implementation detail.

OTOH, repr(transparent) only means that your type has the same ABI as the wrapped type - it's a specific kind of ABI declaration. IMO it should cause a compilation error if the wrapped type has more than 1 non-zero-sized fields.

@camlorn

The problem is that nobody agrees what a newtype is. Is an Rc a newtyped raw pointer? I can certainly imagine it being a #[repr(transparent)] wrapped pointer, for easier interoperability with C.

[eddyb]

@nagisa @arielb1 I agree fully. While guaranteeing pub struct Foo(pub T); as having this behavior is technically possible, it's neither sufficient nor desirable (for the various reasons you've mentioned).

The RFC does need to account for all of these requirements which lead to #[repr(transparent)].

[camlorn]

@arielb1
I can try to define newtype. Let the capability set of a type T be a set of all the logical actions a program can perform on T. A newtype U of some type T is a type whose capability set is a strict subset of T, plus any additional operations which can be written using only operations from the capability set of T. This may make sense only to me, is dense, and probably needs to be more rigorously defined and/or may have exceptions.

Under this definition, Rc is not a newtype because it also requires operations from the capability set of whatever type you use for the reference counted pointer in addition to the pointer itself.

Nevertheless, two things should be noted. A newtype meeting my definition should be adequate here because any way we do that is going to have to conceptually look like a struct with one field. And because newtypes are a subset, going from newtype Foo = Bar to some struct Foo or whatever isn't a breaking change, save that you have to lose repr(transparent) (which you would have to lose under this proposal anyway, should you suddenly need extra fields).

I don't have a problem with us keeping it as it is and never having newtype, but newtypes need machinery to forward some traits at least in order to be ergonomic. This machinery would have to apply to all structs and tuples without it. So either you gain the complexity of teaching people how to use newtype or the complexity of teaching people how to write good code that uses the trait forwarding machinery. I would personally prefer general-purpose trait forwarding machinery, but I can deal with a lot more complexity than most programmers. Consequently, I would begin by arguing against what I personally want in this case.

You could easily counter by saying "well, okay, trait forwarding will only work on single-field structs". Which is fine as far as it goes. But that's not a general feature, and I will argue for more general features over more specific ones. If we choose to leave newtypes as single-field structs then add newtype trait forwarding, someone has to answer the question of why we couldn't make it general.

All I'm trying to say is that I think it's necessary for someone with the power to make decisions to make a definitive decision as to what kind of newtypes we want longterm before we consider this RFC: single-field structs plus general-case machinery that affects more than newtypes or a dedicated mechanism for newtypes with newtype-specific functionality.

[nagisa]

You mean an ABI-breaking change? Isn't that the entire point of the repr attribute?

With #[repr(transparent)] and it inheriting the repr(C)-ness it would become potentially API-breaking too.

[nox]

@camlorn Replace every instance of "newtype" in that RFC by "structure with one field". This RFC doesn't need to be blocked by things that are completely unrelated like trait forwarding.

[camlorn]

@nox
Got into a discussion on IRC about this. The problem is that I wasn't asking what the harm is in allowing repr(transparent) on single-field structs; instead, I was asking something along the lines of why we should allow it on such if it's only useful on newtype. The former question puts this in a much better light. Then possibly needing to allow this with PhantomData came up.

So, anyway, carry on.

[Mark-Simulacrum]

Though the two are different in meaning as far as I can tell, it may be worth considering the similarity in naming to nrc's proposal for #[transparent] in #1744. (Just pointing it out, no suggestions).

[nrc]

I find this RFC confusing, I think because I am unclear what you mean by representation. Rust uses interior allocation for structs (which while not guaranteed because of no formal ABI, you can definitely rely on). So in that respect, the in-memory representation does not change at all. But you seem to be proposing changing the how newtypes interact with FFI? It might be better to write the RFC with that focus.

[nox]

#[repr(C)] can also influence how values are passed to and returned from FFI functions, notably struct Foo(f64) on ARM64.

[parched]

I think I agree with @nrc here. Unless I'm confused, this has nothing to do with 'repr' but is purely a calling convention issue.

However, if you change the calling convention like this then it is incompatible with FFI. Could you possibly give a more detailed example how you would use this in a FFI case?

[nox]

I'll write a FFI example, but calling convention is also affected by representation, cf. the example where f64 and Struct(f64) don't behave the same in C.

[nox]

Added the PhantomData unresolved question and a paragraph explaining why it matters, with a f64 wrapper example.

[parched]

but calling convention is also affected by representation

or rather, calling convention is affected by the type, f64 and Struct(f64) have the same layout but are different types so (can) have different calling conventions.

Why not just implement std::convert::{From,Into} for your struct type?

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

The final comment period is now complete.

[RalfJung]

An alternative that I expected to find discussed in the RFC but did not is to declare that in general, (i.e., no matter the repr), if a struct has a single non-ZST field, then it is guaranteed to have the same layout as that field. I would not be surprised if people are actually already relying on that.

In fact, #1789 does. That other RFC relies on Cell<T> having the same representation as T; it also sends a signal that we can actually rely on this kind of interconversion.

EDIT: I see this has been brought up in discussion, but not finally resolved yet. All right.

[Amanieu]

I agree with @RalfJung, I think this should be implicit as part of the Rust ABI. This would not apply to #[repr(C)] structs, which will have the layout of a C struct containing a single member. The main difference is that the latter will likely be passed as a function argument in a less efficient way.

[sfackler]

@Amanieu the main motivation for this feature is FFI use cases where it is normally very bad to pass a repr(Rust) struct into C. It seems strange for it to be bad except for this one use case. A repr(transparent) allows the programmer to explicitly identify that they are defining a C struct with one field (#[repr(C)]), or just a transparent wrapper for the inner type #[repr(transparent)].

[aturon]

Huzzah! This RFC has been merged. Tracking issue.

(Apologies for the delay!)

[briansmith]

Sorry, it's been forever since we had that conversation. Maybe I'm overlooking something, but I think #[repr(transparent, align=16)] would be very common. For example:

#[repr(transparent, align=16)]
struct Aes256Key([u8; 32]);

#[repr(transparent, align=16)]
struct AesNonce([u8; 16]);

// Keep this in sync with the C definition in aes.h
#[repr(C)]
struct Aes256Context {
    key: Aes256Key,
    ...
    nonce: AesNonce,
    ...
}

extern aes256Encrypt(context: &mut Aes256Context, ...);

I'm not sure how one would express this otherwise.

[briansmith]

(Sorry, the previous comment was intended for another issue.)

[arielb1]

@briansmith

What C declaration is this intended to match? Aka why can't you use

#[repr(C, align=16)]
struct AesNonce([u8; 16]);

[briansmith]

What C declaration is this intended to match? Aka why can't you use

#[repr(C, align=16)]
struct AesNonce([u8; 16]);

Because, for example, sizeof(AesNonce) isn't guaranteed to equal sizeof(AesNonce::0) because C allows the ABI to specify extra (even useless and arbitrary) padding at the end. More generally the're not guaranteed to have the same memory layout. See rust-lang/rust#43036 (comment)

[tbelaire]

The RFC says that this should be illegal.

#[repr(transparent, align = "128")]
struct BogusAlign(f64);

But this should be legal

#[repr(transparent, align = "128")]
struct OverAligned(f64); // Behaves as a bare f64 with 128 bits alignment.

I think you've made an error somewhere.

[willmo]

@tbelaire this RFC was amended to fix that in #2271.