Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking issue for Fn traits (unboxed_closures & fn_traits feature) #29625

Open
1 of 2 tasks
aturon opened this issue Nov 5, 2015 · 50 comments
Open
1 of 2 tasks

Tracking issue for Fn traits (unboxed_closures & fn_traits feature) #29625

aturon opened this issue Nov 5, 2015 · 50 comments
Labels
B-RFC-implemented Approved by a merged RFC and implemented. B-unstable Implemented in the nightly compiler and unstable. C-tracking-issue Category: A tracking issue for an RFC or an unstable feature. F-unboxed_closures `#![feature(unboxed_closures)]` S-tracking-design-concerns Blocking design concerns T-lang Relevant to the language team, which will review and decide on the PR/issue. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.

Comments

@aturon
Copy link
Member

aturon commented Nov 5, 2015

Tracks stabilization for the Fn* traits.

Random bugs:

@aturon aturon added T-lang Relevant to the language team, which will review and decide on the PR/issue. B-unstable Implemented in the nightly compiler and unstable. labels Nov 5, 2015
@alexcrichton alexcrichton added the T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. label Mar 7, 2016
@nagisa
Copy link
Member

nagisa commented May 13, 2016

Inability to delegate calls to other FnOnce implementors like this:

struct A<T>(T);

impl<T, Args> FnOnce<Args> for A<T>
where T: FnOnce<Args> {
    type Output = <T as FnOnce<Args>>::Output;
    fn call_once(self, args: Args) -> Self::Output { FnOnce::call_once(self.0, args) }
}

is the reason I have a safety issue in libloading.

@petrochenkov
Copy link
Contributor

petrochenkov commented Jul 12, 2016

FnOnce::Output is stabilized in #34365

@nrc
Copy link
Member

nrc commented Aug 17, 2016

Could someone summarise what is blocking stabilisation here please?

@nagisa
Copy link
Member

nagisa commented Aug 18, 2016

@nrc uncertainty about Args being the right thing given the possibility of variadic generics coming along around 2030 is the most major reason this is unstable.

@aturon
Copy link
Member Author

aturon commented Aug 27, 2016

@nrc also possibly some questions around the inheritance relationship, see #19032

@brunoczim
Copy link

brunoczim commented Jan 18, 2018

There should be a way of casting fn(A) -> B to Fn(A) -> B

@SimonSapin
Copy link
Contributor

SimonSapin commented Jan 18, 2018

@brunoczim That coercion already happens implicitly, but Fn() as a type is a trait object so it needs to be behind some kind of pointer:

use std::rc::Rc;
fn main() {
    let _: &Fn() = &main;
    let _: Box<Fn()> = Box::new(main);
    let _: Rc<Fn()> = Rc::new(main);
}

@Michael-F-Bryan
Copy link

Michael-F-Bryan commented Apr 2, 2018

One issue we've been discussing on TheDan64/inkwell#5 is the ability to mark something as unsafe to use. What about having an UnsafeFn marker trait which could be used to tell the compiler a callable is unsafe to call?

For context, inkwell is a wrapper around LLVM and we're trying to figure out how to return a function pointer to a JIT compiled function, when calling the function is fundamentally unsafe for the same reasons FFI code is unsafe to call.

@CodeSandwich
Copy link

CodeSandwich commented May 4, 2018

There is another reason to add UnsafeFn: currently unsafe fns don't implement Fn. There is no way to pass unsafe fn as a function argument, they are second class citizen: example.

UnsafeFn could be implemented for things implementing Fn, so Fns can be used wherever UnsafeFn is required. There probably also should be UnsafeFnMut and UnsafeFnOnce.

@alexreg
Copy link
Contributor

alexreg commented Aug 20, 2018

@Michael-F-Bryan @CodeSandwich This sounds like something for which an RFC would really be appreciated. It probably wouldn't be an overly long or intricate one to write, even. I would support it, for sure, and judging by an issue I saw about this not long ago (a long-standing one), others would too.

@CodeSandwich
Copy link

CodeSandwich commented Aug 21, 2018

@alexreg Ok, I'll prepare it in spare time. Unfortunately I lack knowledge and experience to actually implement it or even fully understand the idea.

@alexreg
Copy link
Contributor

alexreg commented Aug 21, 2018

@CodeSandwich Don't worry, so do I! Maybe get yourself on Rust's Discord (#design channel) and we can discuss it with some people who really know the nitty gritty? You can ping me there, same username.

@RalfJung
Copy link
Member

RalfJung commented Aug 22, 2018

With every unsafe function comes a manually written "contract" saying when that function may or may not be called.

With UnsafeFn, who is setting that contract?

@Michael-F-Bryan
Copy link

Michael-F-Bryan commented Aug 23, 2018

With UnsafeFn, who is setting that contract?

I'd say this is done on a case-by-case basis. It's really hard to specify the various invariants and assumptions you make in unsafe code in something as rigid as a type system, so anything that accepts an UnsafeFn would probably also need to document its assumptions.

Before writing up a RFC I thought I'd make a post on the internal forum. It'd be nice to hear what opinions other people have on this topic and the different solutions they come up with.

@Vurich
Copy link

Vurich commented Nov 5, 2018

I don't know how one would realistically implement this, but probably the ideal semantics is to have any function that takes an unsafe function as an argument to also be unsafe.

@SimonSapin
Copy link
Contributor

SimonSapin commented Nov 5, 2018

A different interface/API likely requires a separate set of traits. Though multiplying the number of traits doesn’t sound great, especially if we later want to also support const fn closures (and unsafe const fn?).

@audrey-jensen
Copy link

audrey-jensen commented Nov 6, 2018

Is there a way to implement these as wrapper traits? Unsafe<T> where T : FnOnce, Const<T> where T : FnOnce, Async<T> where T : FnOnce and so on?

@scottmcm scottmcm changed the title Tracking issue for Fn traits (unboxed_closures feature) Tracking issue for Fn traits (unboxed_closures & fn_traits feature) Dec 8, 2021
@joshtriplett joshtriplett added the S-tracking-design-concerns Blocking design concerns label Dec 8, 2021
@joshtriplett
Copy link
Member

joshtriplett commented Dec 8, 2021

We definitely want to stabilize the Fn family of traits at some point, allowing people to impl them.

Marking this as "design-concerns" because we need to determine if we should wait for variadic generics or stabilize the tuple-based rust-call ABI.

@bjorn3
Copy link
Contributor

bjorn3 commented Dec 9, 2021

Should Args be turned into an associated type to prevent people from using it to implement operator overloading based on argument type?

@Lokathor
Copy link
Contributor

Lokathor commented Dec 9, 2021

no

@danielhenrymantilla
Copy link
Contributor

danielhenrymantilla commented Dec 10, 2021

@bjorn3 wouldn't that break higher-order signatures (which are implemented as overloads over the input lifetimes)? FWIW, this question is related to that of the Resume parameter for Generators.

a silly thing 🙈
trait FnOnce<'lifetimes..> { // variadic lifetime-generics?
    type Args;
    type Output;

    extern "rust-call"
    fn call_once (self, _: Self::Args)
      -> Self::Output
    where
        Self : Sized,
    ;
}

@bjorn3
Copy link
Contributor

bjorn3 commented Dec 10, 2021

wouldn't that break higher-order signatures (which are implemented as overloads over the input lifetimes)?

Right, didn't think about that.

@clarfonthey
Copy link
Contributor

clarfonthey commented Feb 17, 2022

Silly question, but is there any particular reason why the Fn* traits actually need to stabilise the rust-call ABI in order to be implementable?

I mean, we already have the custom Fn(A, B, ...) -> C syntax as sugar for Fn<(A, B,), Output = C>, so, I don't think it'd be unreasonable to adopt a special syntax just for implementing them too.

Maybe something like:

struct MyFn(u32);
impl MyFn {
    fn(self, x: u32, y: u32) -> u32 {
        x + y + self.0
    }
}

Could get desugared to:

struct MyFn(u32);
impl FnOnce<(u32, u32)> for MyFn {
    type Output = u32;
    extern "rust-call" fn call_once(self, (x, y): (u32, u32)) -> u32 {
        /* body */
    }
}

@yasuo-ozu
Copy link

yasuo-ozu commented Aug 5, 2022

@clarfonthey I think there's some needs to implement Fn* traits for arbitrary types.

  1. Giving multiple function signatures to an object.
  2. In following URL, there's some difficulty defining trait Handler, so we want to use Fn* trait directly.
    https://users.rust-lang.org/t/type-inference-in-closures/78399

@bjorn3
Copy link
Contributor

bjorn3 commented Aug 5, 2022

Giving multiple function signatures to an object.

This is something I think we shouldn't support in the first place, even with Fn*.

In following URL, there's some difficulty defining trait Handler, so we want to use Fn* trait directly. https://users.rust-lang.org/t/type-inference-in-closures/78399

If I understand it correctly, preventing multiple impls of Fn* would fix this issue.

@yasuo-ozu
Copy link

yasuo-ozu commented Aug 6, 2022

@bjorn3
Thanks for replying.

we shouldn't support in the first place

preventing multiple impls of Fn*

Do you have any standings for preventing higher-order signaitures as mentioned below?
#29625 (comment)

@bjorn3
Copy link
Contributor

bjorn3 commented Aug 6, 2022

I don't know how to support higher-order signatures while at the same time preventing multiple call signatures for a type other than keeping Fn* perma-unstable.

@peterjoel
Copy link
Contributor

peterjoel commented Aug 6, 2022

@zseri
Copy link
Contributor

zseri commented Aug 6, 2022

@peterjoel but that wouldn't scale to function-like objects with generic arguments, for which an associated type wouldn't work. What we really want to prevent here are functions with multiple simultaneous argument counts (because varying argument types is already possible, although a bit complicated, using sealed traits), which could be solved with 2 additional traits (one for tuples, one for functions, which binds tuples to the count of directly contained objects, and functions to their argument counts, either via type-level integers, or using const generics + associated constants)

@bjorn3
Copy link
Contributor

bjorn3 commented Aug 6, 2022

but that wouldn't scale to function-like objects with generic arguments, for which an associated type wouldn't work.

Closures can't be generic either.

@yasuo-ozu
Copy link

yasuo-ozu commented Aug 6, 2022

@zseri

What we really want to prevent here are functions with multiple simultaneous argument counts

Is this really harmful? it looks fine in current unstabilized fn-traits:
https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=157729b98808b439ecc992e4ba59273e

@bjorn3
Copy link
Contributor

bjorn3 commented Aug 6, 2022

It may be fine from a technical perspective, but from a language perspective we have chosen to not allow function overloading. See for example https://internals.rust-lang.org/t/justification-for-rust-not-supporting-function-overloading-directly/7012, https://users.rust-lang.org/t/is-it-possible-to-specialize-hashmap-index-for-copy-types/7750/5, https://users.rust-lang.org/t/what-is-the-reason-for-not-having-overloaded-versions-of-fn/55208 and https://users.rust-lang.org/t/is-there-a-simple-way-to-overload-functions/30937.

@nyxtom
Copy link

nyxtom commented Aug 6, 2022

I recently did some experiments with function traits that requires the use of trait specialization to get it working.

https://github.com/nyxtom/composition/blob/main/src/lib.rs

pub trait Func<Args, T> {
    type Output;
    fn call(&self, args: Args) -> Self::Output;
}

// Default implementation of a func for T as output
impl<A, B, Args, T> Func<Args, ()> for (A, B)
where
    A: Fn<Args, Output = T>,
    B: Fn<T>,
{
    type Output = B::Output;

    #[inline]
    fn call(&self, args: Args) -> Self::Output {
        let args = self.0.call(args);
        self.1.call(args)
    }
}

// Subset of (A, B) T is (T,)
impl<A, B, Args, T> Func<Args, (T,)> for (A, B)
where
    A: Fn<Args, Output = T>,
    B: Fn<(T,)>,
{
    type Output = B::Output;
    #[inline]
    fn call(&self, args: Args) -> Self::Output {
        let args = self.0.call(args);
        self.1.call((args,))
    }
}

Specifically it allowed me to compose between two functions that is already a tuple being returned and apply them as arguments, or in the natural case where the return value is a single type.

fn foo() {}

fn test() -> i32 {
    3
}
fn plus(a: i32) -> i32 {
    a + 1
}
fn multiply(a: i32, b: i32) -> i32 {
    a * b
}
fn output() -> (i32, i32) {
    (4, 2)
}

fn assert_func<Args, T>(_: impl Func<Args, T>) {}

#[test]
fn test_assert_funcs() {
    assert_func((foo, foo));
    assert_func((foo, test));
    assert_func((test, plus));
    assert_func((plus, plus));
    assert_func((multiply, plus));
    assert_func((output, multiply));
}

Does this constitute function overloading or just a use of trait specialization? I did later expand on this to support more composition (than 1 argument) by using the recursive tuple structure like so:

// Subset of (A, B) where A is already a tuple that implements Func
impl<A, B, Args, T, F> Func<Args, ((), (), F)> for (A, B)
where
    A: Fn<Args, Output = T>,
    B: Func<T, F>,
{
    type Output = B::Output;
    #[inline]
    fn call(&self, args: Args) -> Self::Output {
        let args = self.0.call(args);
        self.1.call(args)
    }
}

// Subset of (A, B) where is A is Func and B takes (T,)
impl<A, B, Args, T, F> Func<Args, ((), (T,), F)> for (A, B)
where
    A: Fn<Args, Output = T>,
    B: Func<(T,), F>,
{
    type Output = B::Output;
    #[inline]
    fn call(&self, args: Args) -> Self::Output {
        let args = self.0.call(args);
        self.1.call((args,))
    }
}

This allowed me to perform expressions like so:

#[test]
fn test_assert_nested_func() {
    assert_func((multiply, (plus, plus)));
    assert_func((plus, (plus, plus)));
    assert_func((output, (multiply, plus)));
}

This does require the use of the Fn<T> rather than the parenthetical notation but it's quite useful here as it allows some nice composition to happen. As well, you can make use of variadics with just a macro that turns it into a recursive structure.

pub struct Function<F, T>(F, PhantomData<T>);

impl<F, Args, T> Fn<Args> for Function<F, T>
where
    F: Func<Args, T>,
{
    extern "rust-call" fn call(&self, args: Args) -> Self::Output {
        self.0.call(args)
    }
}

impl<F, Args, T> FnMut<Args> for Function<F, T>
where
    F: Func<Args, T>,
{
    extern "rust-call" fn call_mut(&mut self, args: Args) -> Self::Output {
        self.0.call(args)
    }
}

impl<F, Args, T> FnOnce<Args> for Function<F, T>
where
    F: Func<Args, T>,
{
    type Output = F::Output;
    extern "rust-call" fn call_once(self, args: Args) -> Self::Output {
        self.0.call(args)
    }
}

macro_rules! compose {
    ( $last:expr ) => { $last };
    ( $head:expr, $($tail:expr), +) => {
        ($head, compose!($($tail),+))
    };
}

macro_rules! func {
    ( $head:expr, $($tail:expr), +) => {
        Function(($head, compose!($($tail),+)), PhantomData::default())
    };
}

fn assert_fn<Args>(_: impl Fn<Args>) {}

#[test]
fn test_assert_fn() {
    assert_fn(func!(output, multiply, plus));
}

With function composition I can guarantee that the composition of func!(A, B, C) is the type safe equivalent of the input to A and output of C. The main thing here that makes it easier is having Fn<T> rather than Fn(A). As without the generics argument, I end up having to create an entirely different macro that implements these cases for Fn(A), Fn(A, B), Fn(A, B, C), ...etc.

I should note that the above example doesn't necessarily require the use of impl<Args, T> Fn<Args> for Function<Args, T> it just makes it easier to do:

compose!(plus, plus, plus).call((4,));

vs

func!(plus, plus, plus)(4)

@yasuo-ozu
Copy link

yasuo-ozu commented Aug 6, 2022

@bjorn3 Thanks very much, but I cannot understand the difference of higher-order function arguments and arbitrary function signatures(function overloading) clearly.

Thinking of this example, what is the main factor to distinguish the two concepts?

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=c13f39de7df4c9fbc26b19fd7da4a197

Is this the correct way forcing not to use arbitrary function signatures by language design?

@bjorn3
Copy link
Contributor

bjorn3 commented Aug 6, 2022

Higher-order function arguments is when the function arguments only differ by lifetimes.

@yasuo-ozu
Copy link

yasuo-ozu commented Aug 6, 2022

@bjorn3 If so, the following is different signatures?

fn f<T: AsRef<[u8]>>(t: T) { unimplemented!() }
f("hello");
f("hello".to_owned());
f(vec![1,2,3]);

@parasyte
Copy link

parasyte commented Aug 6, 2022

"hello".to_owned() and vec![1,2,3] have the same lifetime (they are stack-allocated). &'static T is special because it is valid for every lifetime.

@Lokathor
Copy link
Contributor

Lokathor commented Aug 6, 2022

if by stack you mean heap, yes

@parasyte
Copy link

parasyte commented Aug 6, 2022

They both happen to point to the heap, but the structs themselves are stack-allocated when used in the argument position like that. Or when assigned to a variable with a let binding, for instance.

@Lokathor
Copy link
Contributor

Lokathor commented Aug 6, 2022

Anyway, the fact that people can already make up their own trait based operator overloading, and even paper over the arity issue with a do_call! macro or whatever, means that there's not much reason to make the Fn traits themselves specifically and magically reject the possibility of overloading. We're just giving people a hard time.

I found tour first reference link as to why not function overloading most interesting because the second reply is from an actual T-lang member that said:

the desire to not have monomorphization-time errors in generics.

and if overloading is happening strictly through the trait system it should end up preventing the post-monomorph errors.

@nyxtom
Copy link

nyxtom commented Aug 6, 2022

Anyway, the fact that people can already make up their own trait based operator overloading, and even paper over the arity issue with a do_call! macro or whatever, means that there's not much reason to make the Fn traits themselves specifically and magically reject the possibility of overloading. We're just giving people a hard time.

I found tour first reference link as to why not function overloading most interesting because the second reply is from an actual T-lang member that said:

the desire to not have monomorphization-time errors in generics.

and if overloading is happening strictly through the trait system it should end up preventing the post-monomorph errors.

This has been my experience (as I've seen implemented in other places). One macro to implement arity to fix the Fn(A), Fn(A, B)..etc and another to be able to call with do_call!(A, B, C). Having the generics on the actual trait here Fn<Args> makes this less difficult to work with and avoids the extra macro expansion. That being said, it's not strictly required to have an impl Fn<Args> for F since you can do this with the trait based approach but the ability to use the extern "rust-call" hack to turn a struct into a function call is a bit different.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
B-RFC-implemented Approved by a merged RFC and implemented. B-unstable Implemented in the nightly compiler and unstable. C-tracking-issue Category: A tracking issue for an RFC or an unstable feature. F-unboxed_closures `#![feature(unboxed_closures)]` S-tracking-design-concerns Blocking design concerns T-lang Relevant to the language team, which will review and decide on the PR/issue. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests