Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fn() -> Out is a valid type for unsized types Out, and it implements FnOnce<(), Output = Out> #82633

Closed
steffahn opened this issue Feb 28, 2021 · 30 comments · Fixed by #102710
Closed
Assignees
Labels
A-associated-items Area: Associated items such as associated types and consts. A-closures Area: closures (`|args| { .. }`) A-traits Area: Trait system A-typesystem Area: The type system C-bug Category: This is a bug. E-easy Call for participation: Easy difficulty. Experience needed to fix: Not much. Good first issue. E-needs-test Call for participation: An issue has been fixed and does not reproduce, but no test has been added. F-unboxed_closures `#![feature(unboxed_closures)]` I-unsound Issue: A soundness hole (worst kind of bug), see: https://en.wikipedia.org/wiki/Soundness P-high High priority T-lang Relevant to the language team, which will review and decide on the PR/issue. T-types Relevant to the types team, which will review and decide on the PR/issue.

Comments

@steffahn
Copy link
Member

steffahn commented Feb 28, 2021

fn() -> Out is a valid type, even for unsized types Out, and it implements FnOnce<(), Output = Out>, even though FnOnce::Output is not ?Sized. So far, I haven’t found a way to turn this into unsoundness without using unboxed_closures.

#![feature(unboxed_closures)]

trait A { fn a() where Self: Sized; }

fn foo<F: FnOnce<()>>() where F::Output: A { F::Output::a() }

fn main() { foo::<fn() -> dyn A>() }

(Playground)

code after rustfmt
#![feature(unboxed_closures)]

trait A {
    fn a()
    where
        Self: Sized;
}

fn foo<F: FnOnce<()>>()
where
    F::Output: A,
{
    F::Output::a()
}

fn main() {
    foo::<fn() -> dyn A>()
}

Errors:

   Compiling playground v0.0.1 (/playground)
error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-Wl,--eh-frame-hdr" "-L" "/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/playground/target/debug/deps/playground-51f3bbb2c25d281d.playground.4dy3dxin-cgu.0.rcgu.o" "-o" "/playground/target/debug/deps/playground-51f3bbb2c25d281d" "/playground/target/debug/deps/playground-51f3bbb2c25d281d.1r48kp97232mukin.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-nodefaultlibs" "-L" "/playground/target/debug/deps" "-L" "/playground/target/debug/build/libsqlite3-sys-f766054917a587df/out" "-L" "/playground/target/debug/build/ring-f55e8d4980a6255d/out" "-L" "/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--start-group" "-Wl,-Bstatic" "/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-e3026a7ea720d3a3.rlib" "/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-dea899c54966188d.rlib" "/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-dda4c0b69607e93b.rlib" "/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-4b7dae8949ac132c.rlib" "/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-978e97832b309706.rlib" "/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-073b1b693304b876.rlib" "/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-c07f996a53ee6558.rlib" "/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-0ae8ed6a282247d0.rlib" "/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-72f6aee6e444f535.rlib" "/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-14b94bdd9a47d665.rlib" "/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-97d562419076c156.rlib" "/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-022f1a0e7cd794ec.rlib" "/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-6c8051b8141a3b3d.rlib" "/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-3aeb407930ebd519.rlib" "/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-6ab1ee6dbc17ad08.rlib" "/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-166dae07beec0398.rlib" "-Wl,--end-group" "/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-761b290f47712921.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc"
  = note: /usr/bin/ld: /playground/target/debug/deps/playground-51f3bbb2c25d281d.playground.4dy3dxin-cgu.0.rcgu.o: in function `playground::foo':
          /playground/src/main.rs:5: undefined reference to `playground::A::a'
          /usr/bin/ld: /playground/target/debug/deps/playground-51f3bbb2c25d281d: hidden symbol `_ZN10playground1A1a17h9a4c65829fc0d7d1E' isn't defined
          /usr/bin/ld: final link failed: bad value
          collect2: error: ld returned 1 exit status
          

error: aborting due to previous error

error: could not compile `playground`

To learn more, run the command again with --verbose.

@steffahn steffahn changed the title fn() -> Out is a valid type unsized types Out, and it implements FnOnce<(), Output = Out> fn() -> Out is a valid type for unsized types Out, and it implements FnOnce<(), Output = Out> Feb 28, 2021
@jonas-schievink jonas-schievink added A-closures Area: closures (`|args| { .. }`) A-traits Area: Trait system C-bug Category: This is a bug. F-unboxed_closures `#![feature(unboxed_closures)]` labels Feb 28, 2021
@steffahn
Copy link
Member Author

steffahn commented Feb 28, 2021

The labeling is a bit tricky on this one. This is most definitely I-unsound 💥-worthy when using unboxed_closures. (And you can, of course, also generate ICE’s in a multitude of ways.)

On the other hand, the fact that there is a type implementing FnOnce with unsized Output type is definitely observable on stable, so this issue is not really only about the #![feature(unboxed_closures)] case. I mean, not even trait objects allow you to skip a Sized bound like that even though trait object types don’t seem to care too much about behaving soundly in other cases (#57893, #80800).

@steffahn
Copy link
Member Author

steffahn commented Feb 28, 2021

@rustbot modify labels: A-associated-items, A-typesystem, T-compiler

@rustbot rustbot added A-associated-items Area: Associated items such as associated types and consts. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. A-typesystem Area: The type system labels Feb 28, 2021
@matthewjasper matthewjasper added the I-unsound Issue: A soundness hole (worst kind of bug), see: https://en.wikipedia.org/wiki/Soundness label Mar 2, 2021
@rustbot rustbot added the I-prioritize Issue: Indicates that prioritization has been requested for this issue. label Mar 2, 2021
@jyn514
Copy link
Member

jyn514 commented Mar 2, 2021

I had trouble seeing the unsoundness, so here's another example:

#![feature(trivial_bounds)]
#![feature(unboxed_closures)]

trait A { fn a() where Self: Sized; }
impl A for str {
    fn a() where Self: Sized {
        unsafe { std::hint::unreachable_unchecked() }
    }
}

fn foo<F: FnOnce<()>>() where F::Output: A { F::Output::a() }

fn main() { foo::<fn() -> str>() }

str: Sized is false, so str::a() should never be executed:

error[E0277]: the size for values of type `str` cannot be known at compilation time
  --> src/main.rs:15:5
   |
4  | trait A { fn a() where Self: Sized; }
   |           ------------------------- required by `A::a`
...
15 |     str::a();
   |     ^^^^^^ doesn't have a size known at compile-time

yet it is anyway:

/playground/tools/entrypoint.sh: line 11:     7 Illegal instruction     timeout --signal=KILL ${timeout} "$@"

@camelid
Copy link
Member

camelid commented Mar 2, 2021

And note that you can observe the unsoundness on stable (@steffahn's code, I just wanted to highlight it):

fn weird_situation<F: FnOnce() -> str>() {
    println!("if this is reachable, there’s a function type with unsized Output type `str`");
}

fn main() {
    weird_situation::<fn() -> str>()
}

This code compiles, but it shouldn't.

@LeSeulArtichaut LeSeulArtichaut added P-high High priority and removed I-prioritize Issue: Indicates that prioritization has been requested for this issue. labels Mar 2, 2021
@LeSeulArtichaut
Copy link
Contributor

@camelid
Copy link
Member

camelid commented Mar 2, 2021

Another way to observe unsoundness:

fn foo<F: FnOnce() -> R, R: ?Sized>() {}

(Playground)

This code compiles, but it shouldn't because Fn::Output is not ?Sized.

@camelid
Copy link
Member

camelid commented Mar 2, 2021

That code used to give a compiler error:

error[E0277]: the size for values of type `R` cannot be known at compilation time
   --> <source>:1:11
    |
1   | fn foo<F: FnOnce() -> R, R: ?Sized>() {}
    |           ^^^^^^^^^^^^^  - this type parameter needs to be `Sized`
    |           |
    |           doesn't have a size known at compile-time

but regressed in 1.49 such that it is no longer erroring.

@camelid camelid added E-needs-bisection Call for participation: This issue needs bisection: https://github.com/rust-lang/cargo-bisect-rustc regression-from-stable-to-stable Performance or correctness regression from one stable version to another. labels Mar 2, 2021
@camelid
Copy link
Member

camelid commented Mar 2, 2021

I suspect this might be more fallout from #73905, but we'll need a bisection to be sure.

@camelid
Copy link
Member

camelid commented Mar 2, 2021

Yep, it's #73905 alright:

searched nightlies: from nightly-2020-10-01 to nightly-2020-12-31
regressed nightly: nightly-2020-10-07
searched commits: from a1dfd24 to 98edd1f
regressed commit: 08e2d46

bisected with cargo-bisect-rustc v0.6.0

Host triple: x86_64-apple-darwin
Reproduce with:

cargo bisect-rustc --regress=success --start=2020-10-01 --end=2020-12-31 -- check 

@camelid camelid removed the E-needs-bisection Call for participation: This issue needs bisection: https://github.com/rust-lang/cargo-bisect-rustc label Mar 2, 2021
@camelid
Copy link
Member

camelid commented Mar 2, 2021

I wonder if the issue is that, quoting from #73905:

Associated type bindings on trait objects are verified to meet the bounds declared on the trait when checking that they implement the trait. (#27675)

But the compiler is the one that implements FnOnce for fn types? Maybe that's a different thing though.

@steffahn
Copy link
Member Author

steffahn commented Mar 2, 2021

I’m not 100% sure what you mean by “observing the unsoundness”.

Something like

fn foo<F: FnOnce() -> R, R: ?Sized>() {}

doesn’t feel necessarily unsound in and by itself. As you quoted,

Associated type bindings on trait objects are verified to meet the bounds declared on the trait when checking that they implement the trait.

which sounds pretty reasonable to me.

Similarly my reproduction on stable only observes that fn() -> str has an illegal FnOnce implementation, but I wasn’t able to achieve much “actually unsound behavior” with this implementation without the ability to name the Output type, hence using unboxed_closures. But just the inability to name Output (probably) preventing bad things for now doesn’t feel right.


As to ways this should be fixed: Either disallow all types fn(...) -> R with R: !Sized in general, or at least change the compiler so that they don’t implement FnOnce.

@steffahn
Copy link
Member Author

steffahn commented Mar 2, 2021

In particular, I don’t think that regression-from-stable-to-stable is appropriate. I can get ICE’s all the way since mid-2017 (bisect-rustc doesn’t seem to like older dates) with code like this

#![feature(unboxed_closures)]

fn foo<F: FnOnce<()>>()
{
    let _: Vec<F::Output> = vec![];
}

fn main() {
    foo::<fn() -> str>()
}

Similarly, reading static memory and segfaulting can be archieved all the way back to e.g. nightly-2017-03-01 (go much earlier and rustup complains about missing cargo) with code such as

CLICK TO EXPAND
#![feature(unboxed_closures)]

trait Foo<T> {
    fn foo(&mut self) -> T;
}
impl<T> Foo<T> for Option<T> {
    fn foo(&mut self) -> T {
        self.take().unwrap()
    }
}

trait Bar<T> {
    fn bar(&mut self) -> T;
}
impl<S, T> Bar<T> for S {
    fn bar(&mut self) -> T {
        unimplemented!()
    }
}

fn transmute<S, T>(x: S) -> T {
    let x: &mut (Foo<S>) = &mut Some(x);
    step_one::<fn() -> _, T>(x)
}

fn step_one<F: FnOnce<()>, T>(x: &mut F::Output) -> T {
    let x: &mut Bar<T> = x; // casts &mut dyn Foo<S> to &mut dyn Bar<T>
                                // compiler is happy since `F::Output: Sized`
                                // on the other hand, the compiler later implements the actual
                                // cast as a no-op since the reference `x` already contains a vtable pointer

    x.bar() // effectively calls x.foo() from the vtable
}

static X: usize = 12345;

fn main() {
    let x = transmute::<&'static usize, &'static [usize; 1000000 as _]>(&X);
    assert_eq!(x[0], 12345); // works, nice!

    // let’s keep reading the rest of static memory and whatever follows:
    println!("{:?}", &x[1..]);

    // eventually segfaults
}

I would believe that the bug is not a regression but just always has been there.

@rustbot modify labels: -regression-from-stable-to-stable

@rustbot rustbot removed the regression-from-stable-to-stable Performance or correctness regression from one stable version to another. label Mar 2, 2021
@apiraino
Copy link
Contributor

apiraino commented Mar 4, 2021

@rustbot label +I-nominated

@joshtriplett
Copy link
Member

Starting a poll to check lang-team consensus:

@rfcbot poll T-lang Should we prohibit types like fn(...) -> Out if Out is not Sized?

@rfcbot
Copy link

rfcbot commented Mar 30, 2021

Team member @joshtriplett has asked teams: T-lang, for consensus on:

Should we prohibit types like fn(...) -> Out if Out is not Sized?

@joshtriplett
Copy link
Member

Clarification: this does not prevent us from potentially improving support for unsized values in the future, it just prevents such function types from existing until we do.

@joshtriplett
Copy link
Member

ping @estebank

Do you still have the bandwidth to implement this? Looks like we have consensus.

@pnkfelix
Copy link
Member

pnkfelix commented Mar 30, 2021

(we may want to warning-cycle the prohibition, to be clear)

@estebank
Copy link
Contributor

@joshtriplett I do. I'll cook up a PR.

@estebank
Copy link
Contributor

estebank commented Apr 6, 2021

@joshtriplett published #83915. Sorry it took me so long, I had multiple false starts. It might need some changes to be good enough for merging, but that PR enforces Out: Sized for all Outs except impl Trait and projections.

As follow up work, I would like to change the spans to be tighter/easier to understand, but for now this is good enough:

error[E0277]: the size for values of type `(dyn A + 'static)` cannot be known at compilation time
  --> $DIR/closure-return-type-must-be-sized.rs:54:5
   |
LL |     a::foo::<fn() -> dyn A>();
   |     ^^^^^^^^^^^^^^^^^^^^^^^ doesn't have a size known at compile-time
   |
   = help: the trait `Sized` is not implemented for `(dyn A + 'static)`

@joshtriplett
Copy link
Member

Is there any way the exceptions for impl Trait or projections could be used to generate unsoundness in the same style as the PoCs in this thread?

@estebank
Copy link
Contributor

estebank commented Apr 6, 2021

@joshtriplett I could envision that being the case, but haven't come up with a way to trigger it.

estebank added a commit to estebank/rust that referenced this issue Apr 6, 2021
In a `fn() -> Out` bound, enforce `Out: Sized` to avoid unsoundness.

Fix rust-lang#82633.
@estebank estebank self-assigned this Apr 6, 2021
@estebank
Copy link
Contributor

Niko presented an alternative approach.

@jackh726 jackh726 added the WG-traits Working group: Traits, https://internals.rust-lang.org/t/announcing-traits-working-group/6804 label Feb 3, 2022
@oli-obk oli-obk added T-types Relevant to the types team, which will review and decide on the PR/issue. and removed WG-traits Working group: Traits, https://internals.rust-lang.org/t/announcing-traits-working-group/6804 labels Jul 15, 2022
@lcnr
Copy link
Contributor

lcnr commented Oct 4, 2022

fixed by #100096 which removes the builtin Fn* impl for function pointers with unsized return values.

marking as needs test

@lcnr lcnr added the E-needs-test Call for participation: An issue has been fixed and does not reproduce, but no test has been added. label Oct 4, 2022
@estebank
Copy link
Contributor

estebank commented Oct 5, 2022

@estebank estebank added the E-easy Call for participation: Easy difficulty. Experience needed to fix: Not much. Good first issue. label Oct 5, 2022
@Rageking8
Copy link
Contributor

@rustbot claim

For adding the test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-associated-items Area: Associated items such as associated types and consts. A-closures Area: closures (`|args| { .. }`) A-traits Area: Trait system A-typesystem Area: The type system C-bug Category: This is a bug. E-easy Call for participation: Easy difficulty. Experience needed to fix: Not much. Good first issue. E-needs-test Call for participation: An issue has been fixed and does not reproduce, but no test has been added. F-unboxed_closures `#![feature(unboxed_closures)]` I-unsound Issue: A soundness hole (worst kind of bug), see: https://en.wikipedia.org/wiki/Soundness P-high High priority T-lang Relevant to the language team, which will review and decide on the PR/issue. T-types Relevant to the types team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging a pull request may close this issue.