Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't require objects to always be wrapped in Arc<> #1672

Merged
merged 1 commit into from Dec 6, 2023

Conversation

bendk
Copy link
Contributor

@bendk bendk commented Jul 26, 2023

When reviewing #1662 I felt like I would be very annoyed to always have to wrap error objects with Arc::new(), this is an attempt to see if can avoid that. It contains a general FfiConverter implementation for the struct itself, that simply wraps the value in Arc::new() then forwards the call to the normal Arc<> impl.

For constructors and other methods that return an object, UniFFI can wrap the object in an Arc<> just fine, the library doesn't need to do this. Allowing the library to skip the wrapping can improve ergonomics, especially if we start to allow interfaces as errors.

@bendk bendk requested a review from a team as a code owner July 26, 2023 00:50
@bendk bendk requested review from jhugman and removed request for a team July 26, 2023 00:50
}

fn try_lift(v: Self::FfiType) -> ::uniffi::Result<Self> {
panic!("Can't lift")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the main issue with this change. If we implement FfiConverter on a type, then UniFFI-exported functions can input that type as an argument. However, it's not possible to input the object type directly if it's already wrapped in an arc (into_inner doesn't seem right).

Right now this is just implemented as a panic, which is unfortunate because if a library does try to input an object type it would be better to catch the issue at compile time. I don't think there's a static assertion to check that a function is never called though.

One solution would be to split up FfiConverter into 2 traits: one for lifting and one for lowering. That seems like a big change though, is there anything easier?

@jplatte
Copy link
Collaborator

jplatte commented Jul 26, 2023

I agree that this should be supported, but I don't think anything should change / needs to change about the traits. This can be 100% a function of the scaffolding code generation: If a #[uniffi::constructor] returns exactly Self or the same Ident that's found in the impl header, we can add Arc::new to the generated scaffolding, otherwise omit it like we do now. Wdyt?

@bendk
Copy link
Contributor Author

bendk commented Jul 26, 2023

I agree that this should be supported, but I don't think anything should change / needs to change about the traits. This can be 100% a function of the scaffolding code generation: If a #[uniffi::constructor] returns exactly Self or the same Ident that's found in the impl header, we can add Arc::new to the generated scaffolding, otherwise omit it like we do now. Wdyt?

That's a neat idea, but I'm also concerned about non-constructurs that return interfaces. In particular, if we want to use interfaces as errors, it's going to be annoying to have to add map_err(Arc::new) to every returned Result.

@jplatte
Copy link
Collaborator

jplatte commented Jul 26, 2023

I see what you mean. Well in that case, I'm fully in support of splitting the trait in two, which I wanted anyways. We could allow users to opt into this for regular types as well, to reduce the amount of generated code when a certain type is either only ever used as (part of) a parameter, or only ever used as (part of) a return value.

@bendk
Copy link
Contributor Author

bendk commented Jul 26, 2023

I wonder what the right split is. In addition to FfiLift and FfiLower traits, I think it would also be nice to have an FfiCallbackError trait since errors used in callback interfaces have extra requirements. Maybe we should just go really fine grained and split out traits like FfiRead, FfiWrite, FfiReturn, etc.

@jplatte
Copy link
Collaborator

jplatte commented Jul 26, 2023

FfiReturn is something I had previously, maybe you remember. If we have that separately, things like Option<()> that are not representable in many languages will be disallowed which they currently aren't (because () will only implement FfiReturn, none of the other traits).

@bendk bendk changed the title WIP: Don't require objects to always be wrapped in Arc<> Don't require objects to always be wrapped in Arc<> Dec 1, 2023
@bendk
Copy link
Contributor Author

bendk commented Dec 1, 2023

We have split up the FfiConverter trait so now this one works great. By implementing LowerReturn, users can return an object from a constructor/function/method and UniFFI will wrap it in an Arc<> for you.

A step further would be to implement Lower, which would allow you to pass an object into a callback interface method without wrapping it. That feels a bit too magical to me though, so I didn't do it.

@bendk
Copy link
Contributor Author

bendk commented Dec 1, 2023

Another option would be to restrict this even more and only support it for constructors. That feels like the least magical of all, but it would require adding a new FFI trait, maybe we could call it LowerNew.

@mhammond
Copy link
Member

mhammond commented Dec 1, 2023

Another option would be to restrict this even more and only support it for constructors. That feels like the least magical of all,

I'm torn here. Allowing all methods to return an object and have the arc wrapping done does seem convenient and might even mean consumers need to make fewer changes to their existing APIs to allow UniFFI to expose it. I don't think it will mis-lead too many people - our docs make it quite clear everything is an Arc.

OTOH, I am sympathetic to "explicit is better than implicit", so expect people will disagree and respect that.

However, I am struggling a little with the concept that is makes sense for constructors but not elsewhere - both arguments above apply equally in both cases IMO.

(See also #1063 where there was strong objection to doing this for constructors. My memory is faulty, because for some reason I thought we supported that everywhere except constructors, but I don't think that's true)

@bendk
Copy link
Contributor Author

bendk commented Dec 2, 2023

However, I am struggling a little with the concept that is makes sense for constructors but not elsewhere - both arguments above apply equally in both cases IMO.

I don't disagree, but one detail is that if we want to support returning T objects, then what about Vec<T>, Option<T>, etc. If we only implement LiftReturn then those won't be supported but users might expect them to be.

That's an argument for the two extreme cases: either limit the auto-arc-wrapping to constructors or support it anytime we lift a value.

Copy link
Member

@mhammond mhammond left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking out loud before - I should try to stop that :)

I don't think it's worth extra effort to not do something in the name of "purity" when it's actually quite useful.

@@ -75,7 +75,7 @@ impl MyObject {
// ...
}

// `Arc<Self>` is also supported
// Returning objects is also supported, either as `Self` or `Arc<Self>`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nano-nit, double-backtick

@bendk
Copy link
Contributor Author

bendk commented Dec 4, 2023

I was thinking out loud before - I should try to stop that :)

I don't think it's worth extra effort to not do something in the name of "purity" when it's actually quite useful.

I don't know, I found it useful. In fact, it made me think I was doing the same thing. Why only implement LowerReturn when we could also implement Lower and make users lives easier? (see the question in Matrix today where a user wanted to return Option<Self>)

The only drawback I can see is that it allows users to pass in objects into callback interface methods without wrapping them in an arc, but why stop that? I can't see any real footguns here. Maybe some users won't understand what UniFFI is doing behind the scenes, but that seems fine to me.

@mhammond
Copy link
Member

mhammond commented Dec 4, 2023

So whether we can is different to whether we should :) A big part of that for me is how magic it seems and whether it helps or hurts the mental model users have with uniffi.

IMO, the direct question here (returning objects from constructors without the Arc<>) has clear utility and IMO isn't going to confuse anyone's mental model of the world. The fact functions also get this "for free" unless we take additional action to prevent it can be classed a "happy accident" - it might come in useful and I don't really see footguns there, so I'm not sure it's worth additional effort to not support it. In the same way, I doubt I'd be that keen on spending additional effort to support it if it didn't come for free. It could certainly be argued that these limitations are going to be confusing for users, and I suspect a good argument could persuade me if anyone feels strongly about it - but at face value I currently thing it's fine.

However, moving beyond this starts to move into uncertain territory for me. Vec<>, Option<> etc seem much more of a gray zone. What about hashmaps? What does an example hashmap look like in this scenario? Is the user also going to expect a degree of magic when interacting with objects in dictionaries etc? How would we explain some embedded Arcs can omit the Arc (eg, in options/hashmaps) but not others (eg, when returning a struct literal which itself has object members)

So without thinking too much about it, I'm kinda +1 on adding this capability to constructors, +1 on leaving a happy accident that other functions get the simple case for free too, and -0.5 on expanding this too far at this stage, because it's much easier to add this capability in the future than it would be to take it away if it turns out to be confusing for people.

For constructors and other methods that return an object, UniFFI can
wrap the object in an `Arc<>` just fine, the library doesn't need to do
this.  Allowing the library to skip the wrapping can improve ergonomics,
especially if we start to allow interfaces as errors.
@bendk
Copy link
Contributor Author

bendk commented Dec 6, 2023

After talking with Mark, I'm not sure if implementing Lower is right or not so I reverted it back to the version that only implemented LowerReturn.

Going to merge this one since that change was already approved.

@bendk bendk merged commit 59c1e3e into mozilla:main Dec 6, 2023
5 checks passed
kegsay added a commit to matrix-org/matrix-rust-sdk that referenced this pull request Mar 21, 2024
Up until uniffi 0.26 it was not possible to send objects
across the boundary unless they were wrapped in an `Arc<>`,
see mozilla/uniffi-rs#1672

The bindings generator used in complement-crypto only supports
up to uniffi 0.25, meaning having a function which returns objects
ends up erroring with:
```
error[E0277]: the trait bound `TaskHandle: LowerReturn<UniFfiTag>` is not satisfied
   --> bindings/matrix-sdk-ffi/src/room_directory_search.rs:109:10
    |
109 |     ) -> TaskHandle {
    |          ^^^^^^^^^^ the trait `LowerReturn<UniFfiTag>` is not implemented for `TaskHandle`
    |
    = help: the following other types implement trait `LowerReturn<UT>`:
              <bool as LowerReturn<UT>>
              <i8 as LowerReturn<UT>>
              <i16 as LowerReturn<UT>>
              <i32 as LowerReturn<UT>>
              <i64 as LowerReturn<UT>>
              <u8 as LowerReturn<UT>>
              <u16 as LowerReturn<UT>>
              <u32 as LowerReturn<UT>>
            and 133 others

error[E0277]: the trait bound `TaskHandle: LowerReturn<_>` is not satisfied
   --> bindings/matrix-sdk-ffi/src/room_directory_search.rs:82:1
    |
82  | #[uniffi::export(async_runtime = "tokio")]
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `LowerReturn<_>` is not implemented for `TaskHandle`
    |
    = help: the following other types implement trait `LowerReturn<UT>`:
              <bool as LowerReturn<UT>>
              <i8 as LowerReturn<UT>>
              <i16 as LowerReturn<UT>>
              <i32 as LowerReturn<UT>>
              <i64 as LowerReturn<UT>>
              <u8 as LowerReturn<UT>>
              <u16 as LowerReturn<UT>>
              <u32 as LowerReturn<UT>>
            and 133 others
```

This PR wraps the offending function in an `Arc<>` to make it uniffi 0.25 compatible,
which unbreaks complement crypto.
andybalaam pushed a commit to matrix-org/matrix-rust-sdk that referenced this pull request Mar 21, 2024
Up until uniffi 0.26 it was not possible to send objects
across the boundary unless they were wrapped in an `Arc<>`,
see mozilla/uniffi-rs#1672

The bindings generator used in complement-crypto only supports
up to uniffi 0.25, meaning having a function which returns objects
ends up erroring with:
```
error[E0277]: the trait bound `TaskHandle: LowerReturn<UniFfiTag>` is not satisfied
   --> bindings/matrix-sdk-ffi/src/room_directory_search.rs:109:10
    |
109 |     ) -> TaskHandle {
    |          ^^^^^^^^^^ the trait `LowerReturn<UniFfiTag>` is not implemented for `TaskHandle`
    |
    = help: the following other types implement trait `LowerReturn<UT>`:
              <bool as LowerReturn<UT>>
              <i8 as LowerReturn<UT>>
              <i16 as LowerReturn<UT>>
              <i32 as LowerReturn<UT>>
              <i64 as LowerReturn<UT>>
              <u8 as LowerReturn<UT>>
              <u16 as LowerReturn<UT>>
              <u32 as LowerReturn<UT>>
            and 133 others

error[E0277]: the trait bound `TaskHandle: LowerReturn<_>` is not satisfied
   --> bindings/matrix-sdk-ffi/src/room_directory_search.rs:82:1
    |
82  | #[uniffi::export(async_runtime = "tokio")]
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `LowerReturn<_>` is not implemented for `TaskHandle`
    |
    = help: the following other types implement trait `LowerReturn<UT>`:
              <bool as LowerReturn<UT>>
              <i8 as LowerReturn<UT>>
              <i16 as LowerReturn<UT>>
              <i32 as LowerReturn<UT>>
              <i64 as LowerReturn<UT>>
              <u8 as LowerReturn<UT>>
              <u16 as LowerReturn<UT>>
              <u32 as LowerReturn<UT>>
            and 133 others
```

This PR wraps the offending function in an `Arc<>` to make it uniffi 0.25 compatible,
which unbreaks complement crypto.
bendk added a commit to bendk/uniffi-rs that referenced this pull request Mar 28, 2024
Since mozilla#1672 was merged, we support both `Arc<Self>` and `Self` return
types.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants