-
Notifications
You must be signed in to change notification settings - Fork 155
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
get_array_elements and friends unnecessarily restrict lifetimes #302
Comments
Sounds good. Why don't you submit a PR with your suggested changes? |
As we're coming up to releasing 0.21 I wanted to give a heads up that there are changes that overlap with this change that was made for 0.20 (sorry for the churn @jrose-signal):
As part of #398 there is now more consistent naming and usage of a As part of #400 Hopefully these changes for 0.21 are for the better overall but it's possible they will cause some churn. Hopefully these migration notes will be of some help too: https://github.com/jni-rs/jni-rs/blob/master/docs/0.21-MIGRATION.md |
Thanks, but there are still more lifetimes than necessary. It should be possible to have a single output lifetime that is constrained by all the input lifetimes. |
Heya, just to clarify - I wasn't so much sending the heads up due to the change in lifetime parameters specifically - it was more that I was skimming over some of the 0.20 changes and happened to notice that there was a distinct overlap here with Tbh I didn't remember any details of this specific change. Although I rebased and landed PR #318 for 0.20, I was probably relying on the previous review that had been done and would have generally seen that the change wasn't controversial. It looks like the original issue here was related to inferred lifetime parameters specified via With the latest iteration of As it is now,
I think those three lifetimes are currently all necessary, but maybe you can clarify what you're thinking here? In general though I'm sure there's more that can be done to try and improve the |
The original issue is no longer there, yes. However, the type pub struct AutoElements<'a, T> {
array: &'a JPrimitiveArray<'a, T>,
len: usize,
ptr: NonNull<T>,
mode: ReleaseMode,
is_copy: bool,
env: JNIEnv<'a>,
}
impl<'a, T> AutoElements<'a, T> {
/// # Safety
///
/// `len` must be the correct length (number of elements) of the given `array`
pub(crate) unsafe fn new_with_len<'local, 'array>(
env: &mut JNIEnv<'local>,
array: &'array JPrimitiveArray<'_, T>,
len: usize,
mode: ReleaseMode,
) -> Result<Self, ()>
where
'local: 'a,
'array: 'a,
{
//...
}
} (You can see the compiler accepting this in the Rust playground.) I don't know how much you want to do this kind of overhaul, but as a starting principle you almost never need multiple lifetimes in the same object. There's one smallest lifetime that's relevant, and everything else has to be alive at least that long. The main place that doesn't happen is when |
…etime parameter. Fixes jni-rs#302.
@jrose-signal, here's a commit with this change applied, but what you describe doesn't seem to work for
But if I add the |
It goes to this part:
For the non-Critical version, the lifetime of the reference to JNIEnv wasn't important for the resulting AutoElements, because the function did an func bad<'ref, 'local>(env: &'ref mut JNIEnv<'local>) {
let completely_new_jvm = spin_up_new_jvm();
let new_env: JNIEnv<'short> = completely_new_jvm.env(); // you can't actually name 'short in Rust, but pretend you can
let short_env: &'short mut JNIEnv<'local> = env; // this is okay, 'short is smaller than 'ref
let short_env_only: &'short mut JNIEnv<'short> = short_env; // this is not okay, because of the next line
*short_env_only = new_env; // oops!
} So if you're wrapping something that's |
One thing that probably also needs to be considered here is that (so I think that can give us separate
Originally I had thought that would actually be undefined JNI behaviour to try and do that but it looks like that's probably actually OK to do as far as JNI is concerned, but it does results in a duplicate reference where you're not able to explicitly drop the original reference. See discussion here, for more context: #392 (comment) We could re-consider whether that's really a problem and potentially change this, but wanted to highlight that here since I think it may break some of the assumptions above. |
It does not. What makes the |
At the same time, you have a number of pub fn alloc_object<'other_local, T>(
&mut self,
class: T
) -> Result<JObject<'local>>
where
T: Desc<'local, JClass<'other_local>> Why isn't this just |
At least for me I've been thinking that So in this case I thought the seperation was necessary to ensure it would be possible to allocate an object based on a class that exists outside the current fn foo(env: &mut JNIEnv) -> Result<()> {
let outer_klass = env.find_class(FOO_CLASS)?;
let ref = env.with_local_frame_returning_local(|env| {
let obj = env.alloc_object(outer_klass);
Some(obj)
})?;
Ok(())
} |
I guess you can't have it both ways. Either the local frame prevents references from outside the local frame, or it doesn't; as noted elsewhere, having a second JNIEnv on the same thread breaks many of these assumptions. |
Alternately, you can force all these lifetimes to be invariant, by using EDIT: here's a slightly more polished version from |
not quite sure which "assumptions" you're referring to from having multiple env's per thread but something else to perhaps note is that any API that can create new references now has to mutably borrow the environment, and since |
I think the only reason (at least in my mind) for wanting them to be invariant was to account for the weird case where code tries to move a local reference from an outer scope into a Personally I would be happy to remove Otherwise I wonder if it's actually reasonable enough to just document and warn against this corner case with |
You mean add that constraint to pub fn with_local_frame<F, T, E>(&mut self, capacity: i32, f: F) -> std::result::Result<T, E>
where
F: for<'new_local where 'local: 'new_local> FnOnce(&mut JNIEnv<'new_local>) -> std::result::Result<T, E>,
E: From<Error>,
{} As far as I know, an HRTB lifetime can be declared to outlive some other lifetime, but not the reverse, and we need the reverse here. If we can do this somehow, it should be sound to do so, since this should allow local references' lifetimes to be narrowed but not widened. (Correct me if I'm wrong!)
It does need to describe let mut outer_env: JNIEnv<'outer_local> = _;
let some_class: JClass<'outer_local> = _;
outer_env.with_local_frame(|inner_env: JNIEnv<'inner_local>| {
inner_env.call_static_method(&some_class, "someMethod", "()V", &[])?.v()
})?; The
Why? As far as I know, that's perfectly valid. It works, |
Reopening this since we seem to still be discussing it. |
Yep, you confirmed it works before, it just results in an extra reference and you can no longer explicitly drop the original reference. (ref your comment here too: #392 (comment)) (so even though we determined that JNI is fine with this we briefly noted at the time that it could still lead to an awkward leak) |
Ah, I thought I had looked into this before. Thanks for the reminder. @jrose-signal, how do you feel about this? If we connect the outer and inner lifetimes, then we can eliminate some potentially-confusing and arguably-unnecessary lifetime parameters from And, again, if we are to connect these two lifetimes, then I also need to know how to do that with HRTB syntax. |
@argv-minus-one considering your example about accessing a class from an outer frame is almost the same as the one I gave (#302 (comment)), I guess we may both have had a similar misunderstanding here. Actually if I make this change: index 12c5c71..6b0c34f 100644
--- a/src/wrapper/jnienv.rs
+++ b/src/wrapper/jnienv.rs
@@ -895,9 +895,9 @@ impl<'local> JNIEnv<'local> {
/// Allocates a new object from a class descriptor without running a
/// constructor.
- pub fn alloc_object<'other_local, T>(&mut self, class: T) -> Result<JObject<'local>>
+ pub fn alloc_object<T>(&mut self, class: T) -> Result<JObject<'local>>
where
- T: Desc<'local, JClass<'other_local>>,
+ T: Desc<'local, JClass<'local>>,
{
let class = class.lookup(self)?;
let obj = jni_non_null_call!(self.internal, AllocObject, class.as_ref().as_raw()); this code actually builds and runs fine: #[test]
fn with_local_frame_access_outer_ref() {
let mut env = attach_current_thread();
let int_class = env.find_class(INTEGER_CLASS).unwrap();
let _obj = env.with_local_frame_returning_local(16, |env| {
env.alloc_object(int_class)
}).unwrap();
} (though I would have previously not expected that to work, because I thought that the two environment lifetimes were more disconnected than they are) |
You're correct. I thought you could put it after the function result, like a top-level (I do also think calling this a "memory leak" is incorrect; the memory is held on to longer than necessary, but it will still be released when the outer frame goes away.) |
NB: If your JNI code is running in a thread that's not part of a native method implementation then you may have a base local frame that you don't unwind. Still though it's quite a contrived set of circumstances. It's very hard to imagine this corner case being hit in a way that's really problematic. Arguably it's just a programming error. With the way that references are non-Copy now there can be plenty of ways in which you could lose access to a reference by moving / dropping a If you have code that is going to run within a thread where the frame won't unwind regularly then it's probably fair to say that you need to be conscious of cleaning up any transient local references. |
This, however, doesn't work: fn with_local_frame_access_outer_ref<'local>(
env: &mut JNIEnv<'local>,
int_class: &JClass,
) -> Result<JObject<'local>> {
env.alloc_object(int_class)
}
The compiler isn't happy unless fn with_local_frame_access_outer_ref<'local>(
env: &mut JNIEnv<'local>,
int_class: &JClass<'local>,
) -> Result<JObject<'local>> {
env.alloc_object(int_class)
} Seems like an unnecessary and unergonomic restriction. |
Are you sure that |
I'm not sure what you mean by If you're referring to local reference frames, this function doesn't actually create a new local reference frame (sorry, I forgot to rename it). It's just a function that accepts a |
Sorry, I'm mixing code snippets to respond to. I was wondering more why the previous example worked than why this one didn't, and my conclusion is that there are bunch of complicated Desc implementations such that I can't manually trace where the lifetime is changing. >< Stepping back, I see that adding lifetimes to everything makes sense: you don't want locals to outlive a JNIEnv. You're also trying to have nested local frames use a different lifetime, but those nested frames can still use values attached to the lifetime of the outer frame. You're not trying to protect against multiple independent JNIEnvs being in play at the same time using lifetimes, which is good because lifetimes aren't great at that (being covariant by default). I did not understand all that when I started my discussion. Nevertheless, I'm still pretty sure AutoElements only needs one lifetime parameter (the minimum lifetime of all its local references), and AutoElementsCritical only needs two (the minimum of all its local references, plus the exact lifetime attached to the JNIEnv it holds). Same for JList and JMap and JValue (the referency one). |
Another thing to note here that seems relevant (considering The reason for this was to help ensure you can't easily create multiple This got reverted at the last minute though due to a concern about the ergonomics we'd have with In terms of consolidating these lifetimes, I'm honestly a little bit confused currently. As noted - my mental model so far has been that we needed to consider invariant I don't currently have a clear understanding of how the HRTB I might have misunderstood this comment: #302 (comment) that I initially thought was asserting that SummaryI guess (but am not confident) that we actually do need to consider invariant We'd probably be happy to have covariance via a If we're stuck with needing to consider invariant Although the So maybe for now we do need to keep these three lifetimes? |
yeah this is a tricky one. I've been thinking it feels a bit lucky that we do get away with not needing to explicitly connect the lifetimes of native method arguments like this but at the same time it's convenient that it's not currently required. we can see this as a simpler reason (than talking about conceptually it's a bit like all of these arguments belong to a series of separate |
From the label you've added, I take it we're going to leave this alone for now and think about it later? |
Yeah, I think for now at least it seems like it's reasonable to not block the 0.21 release based on this. In a number of places (not just for Maybe we can change this by investigating what constraints we can convey via HRTBs / We can maybe assume that the In general though I'd quite like to make a 0.21 release soon and not get too hung up on this. I feel like this API isn't exactly perfect in a number of ways, so I also wouldn't be surprised if we end up making this issue redundant anyway when thinking about how to tackle other safety issues with the current API. I don't want to cause unnecessary churn for people but at the same time, some corners of the Does that sound reasonable @argv-minus-one + @jrose-signal ? |
Keeping the issue open seems like a reasonable compromise for now. And at least 0.21 shouldn't have the original issue of tying the lifetime to the JNIEnv reference instead of the array object, if I'm reading it correctly. |
Okey, thanks for the ACK. I think that means I can probably make the 0.21 release either tonight or tomorrow 🤞 |
Now that
JNIEnv<'a>
isCopy
, it's clear that the lifetime of the current context is the lifetime parameter'a
, not the lifetime of any individual (Rust) reference to it. However,get_array_elements
et al have an inferred result type that matches the lifetime of&self
, not'a
:This could be fixed to include
'a
in the result type, but now thatJNIEnv<'a>
isCopy
, there's no reason for AutoArray (and AutoPrimitiveArray, and JList/JMap/JavaStr) to store a reference to one at all.This would allow the AutoArray to outlast the current JNIEnv value, which makes sense when passing them around by Copy instead of by reference.
The text was updated successfully, but these errors were encountered: