repr(transparent): don't consider most length-0 arrays trivial

[Jules-Bertholet]

With this PR, an array type is considered trivial for the purpose of repr(transparent) only if its element type is—we emit the repr_transparent_non_zst_fields FCW (#78586) otherwise.

This has two benefits:

Forbid non-portable definitions

Some types have alignment 1 only on certain platforms. Prior to this PR, the following snippet would compile on AVR, and only on AVR:

#[repr(transparent)]
struct Foo(i32, [u16; 0]);

After this PR, the above now fails to compile on any target.

FFI and CFI compatibility

We want to add support for Control Flow Integrity to Rust at some point. There are some good reasons to want CFI to consider *const [u8; 0] and *const [u8; 1] compatible with one another. But that means we must consider *const [u8; 0] and *const () to be CFI-incompatible. Declaring [u8; 0] non-trivial for repr(transparent) makes that easier to achieve. See discussion on Zulip:
https://rust-lang.zulipchat.com/#narrow/channel/136281-t-opsem/topic/ABI-compatibility.20rules.20of.20ZST.20types/near/591412488

@rustbot label T-lang needs-fcp A-repr
Also needs a crater run.

[rustbot]

r? @dingxiangfei2009

rustbot has assigned @dingxiangfei2009.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: compiler
• compiler expanded to 73 candidates
• Random selection from 20 candidates

[rust-log-analyzer]

The job aarch64-gnu-llvm-21-1 failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

##[endgroup]
Executing "/scripts/stage_2_test_set1.sh"
+ /scripts/stage_2_test_set1.sh
+ '[' 1 == 1 ']'
+ echo 'PR_CI_JOB set; skipping tidy'
+ SKIP_TIDY='--skip tidy'
+ ../x.py --stage 2 test --skip tidy --skip compiler --skip src
PR_CI_JOB set; skipping tidy
##[group]Building bootstrap
    Finished `dev` profile [unoptimized] target(s) in 0.04s
##[endgroup]
---

---- [codegen] tests/codegen-llvm/repr/transparent.rs stdout ----
------rustc stdout------------------------------

------rustc stderr------------------------------
warning: type `f32x4` should have an upper camel case name
##[warning]   --> /checkout/tests/codegen-llvm/repr/transparent.rs:135:8
    |
135 | struct f32x4([f32; 4]);
    |        ^^^^^ help: convert the identifier to upper camel case (notice the capitalization): `F32x4`
    |
    = note: `#[warn(non_camel_case_types)]` (part of `#[warn(nonstandard_style)]`) on by default

error: zero-sized fields in `repr(transparent)` cannot contain array types with non-trivial element types
##[error]  --> /checkout/tests/codegen-llvm/repr/transparent.rs:50:43
   |
50 | pub struct WithZeroSizedArray(*const f32, [i8; 0]);
   |                                           ^^^^^^^
   |
   = note: this field contains `[i8; 0]`, which is an array with a non-trivial element type, so it is not guaranteed to have a trivial ABI in all situations.
   = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
   = note: for more information, see issue #78586 <https://github.com/rust-lang/rust/issues/78586>
   = note: `#[deny(repr_transparent_non_zst_fields)]` (part of `#[deny(future_incompatible)]`) on by default

warning: `extern` fn uses type `f32x4`, which is not FFI-safe
##[warning]   --> /checkout/tests/codegen-llvm/repr/transparent.rs:146:34
    |
146 | pub extern "C" fn test_Vector(_: Vector) -> Vector {
    |                                  ^^^^^^ not FFI-safe
    |
    = help: consider adding a `#[repr(C)]` or `#[repr(transparent)]` attribute to this struct
    = note: this struct has unspecified layout
note: the type is defined here
   --> /checkout/tests/codegen-llvm/repr/transparent.rs:135:1
    |
135 | struct f32x4([f32; 4]);
    | ^^^^^^^^^^^^
    = note: `#[warn(improper_ctypes_definitions)]` on by default

warning: `extern` fn uses type `f32x4`, which is not FFI-safe
##[warning]   --> /checkout/tests/codegen-llvm/repr/transparent.rs:146:45
    |
146 | pub extern "C" fn test_Vector(_: Vector) -> Vector {
    |                                             ^^^^^^ not FFI-safe
    |
    = help: consider adding a `#[repr(C)]` or `#[repr(transparent)]` attribute to this struct
    = note: this struct has unspecified layout
note: the type is defined here
   --> /checkout/tests/codegen-llvm/repr/transparent.rs:135:1
    |
135 | struct f32x4([f32; 4]);
    | ^^^^^^^^^^^^

error: aborting due to 1 previous error; 3 warnings emitted


------------------------------------------

error: compilation failed!
status: exit status: 1
command: env -u RUSTC_LOG_COLOR RUSTC_ICE="0" RUST_BACKTRACE="short" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/tests/codegen-llvm/repr/transparent.rs" "-Zthreads=1" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Z" "ignore-directory-in-diagnostics-source-blocks=/cargo" "-Z" "ignore-directory-in-diagnostics-source-blocks=/checkout/vendor" "--sysroot" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2" "--target=aarch64-unknown-linux-gnu" "--check-cfg" "cfg(test,FALSE)" "-O" "-Cdebug-assertions=no" "-Zcodegen-source-order" "--emit" "llvm-ir" "-C" "prefer-dynamic" "-o" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen-llvm/repr/transparent/transparent.ll" "-A" "internal_features" "-A" "incomplete_features" "-A" "unused_parens" "-A" "unused_braces" "-Crpath" "-Cdebuginfo=0" "-Copt-level=3" "-C" "no-prepopulate-passes"
stdout: none
--- stderr -------------------------------
warning: type `f32x4` should have an upper camel case name
##[warning]   --> /checkout/tests/codegen-llvm/repr/transparent.rs:135:8
    |
135 | struct f32x4([f32; 4]);
    |        ^^^^^ help: convert the identifier to upper camel case (notice the capitalization): `F32x4`
    |
    = note: `#[warn(non_camel_case_types)]` (part of `#[warn(nonstandard_style)]`) on by default

error: zero-sized fields in `repr(transparent)` cannot contain array types with non-trivial element types
##[error]  --> /checkout/tests/codegen-llvm/repr/transparent.rs:50:43
   |
50 | pub struct WithZeroSizedArray(*const f32, [i8; 0]);
   |                                           ^^^^^^^
   |
   = note: this field contains `[i8; 0]`, which is an array with a non-trivial element type, so it is not guaranteed to have a trivial ABI in all situations.
   = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
   = note: for more information, see issue #78586 <https://github.com/rust-lang/rust/issues/78586>
   = note: `#[deny(repr_transparent_non_zst_fields)]` (part of `#[deny(future_incompatible)]`) on by default

warning: `extern` fn uses type `f32x4`, which is not FFI-safe
##[warning]   --> /checkout/tests/codegen-llvm/repr/transparent.rs:146:34
    |
146 | pub extern "C" fn test_Vector(_: Vector) -> Vector {
    |                                  ^^^^^^ not FFI-safe
    |
    = help: consider adding a `#[repr(C)]` or `#[repr(transparent)]` attribute to this struct
    = note: this struct has unspecified layout
note: the type is defined here
   --> /checkout/tests/codegen-llvm/repr/transparent.rs:135:1
    |
135 | struct f32x4([f32; 4]);
    | ^^^^^^^^^^^^
    = note: `#[warn(improper_ctypes_definitions)]` on by default

warning: `extern` fn uses type `f32x4`, which is not FFI-safe
##[warning]   --> /checkout/tests/codegen-llvm/repr/transparent.rs:146:45
    |
146 | pub extern "C" fn test_Vector(_: Vector) -> Vector {
    |                                             ^^^^^^ not FFI-safe
    |
    = help: consider adding a `#[repr(C)]` or `#[repr(transparent)]` attribute to this struct
    = note: this struct has unspecified layout
note: the type is defined here
   --> /checkout/tests/codegen-llvm/repr/transparent.rs:135:1
    |
135 | struct f32x4([f32; 4]);
    | ^^^^^^^^^^^^

error: aborting due to 1 previous error; 3 warnings emitted
------------------------------------------

[RalfJung]

Why do you allow any arrays here? Seems easier to just reject them all?
I'd be surprised if there is much/any use of arrays as "trivial" types in repr(transparent).

View changes since the review

[RalfJung]

Also needs a crater run.

For that we need a version of this that emits a hard error.