-
Notifications
You must be signed in to change notification settings - Fork 12.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bogus higher-ranked lifetime error
in an async block
#102211
Comments
We discussed this in the wg-async triage today. We think the best course of action is to try to get a minimal test case for this. Afterwards, we'll probably want to get T-Types input on this. @vincenzopalazzo is going to try to reproduce this and minimize the test case. |
@rustbot label AsyncAwait-Triaged |
@rustbot claim |
I wonder if the following snippet wouldn't be a reduction: struct Type<'a, 'b>(&'a &'b (), ::core::marker::PhantomData<*mut ()>);
// Note: if this had an *implicit* `'b : 'a` things would work.
unsafe impl<'b : 'a, 'a> Send for Type<'a, 'b> {}
fn foo() -> impl Send { async {
let local = ();
let r = &&local;
async {}.await;
let it = Type(r, <_>::default());
async {}.await;
}} |
@sfackler - Would it be possible to share more of the surrounding code for this example so we can see more of what's going on? |
I'm working on minimizing it. |
Here is a semi-minimized repro. It still depends on Hyper, but it could probably be replaced with some PhantomData inserts with some more work: https://gist.github.com/sfackler/a7f3aaf2ce3f5ff4c99e1a3d67f828ed
|
For whatever reason, the presence of the |
Thank @sfackler, I think the core concept missed in your previous report are: Using a wrapper around pub struct AdaptorFuture<F> {
inner: F,
}
impl<F, T> Future for AdaptorFuture<F>
where
F: Future<Output = T>,
{
type Output = Result<T, io::Error>;
fn poll(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll<Self::Output> {
panic!()
}
} and your handle_service implementation impl<S, R> hyper::service::Service<R> for AdaptorService<S>
where
S: Service<R>,
{
type Response = S::Response;
type Error = io::Error;
type Future = AdaptorFuture<S::Future>;
fn poll_ready(&mut self, _: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
Poll::Ready(Ok(()))
}
fn call(&mut self, req: R) -> Self::Future {
AdaptorFuture {
inner: self.inner.call(req),
}
}
} |
@sfackler I'm trying to reproduce your error without hyper, and till now I had no luck. Maybe you can help with some tips? I had the code there vincenzopalazzo/rio#13 |
Not sure I have a better idea of how to minimize than you, unfortunately. |
I think I was able to recreate it? It took me like 35 minutes to figure out where it was because the compiler only gives this message with nothing else and no extra details:
|
This looks promising @brandonros! I will give it a shot and also try to reproduce inside vincenzopalazzo/rio#13 Looks like that when the runtiime try to span the following task there is an error like the one reported from you async fn robinhood_task() {
let robinhood = Robinhood::new();
let token = String::from("fake");
let instrument_ids = vec![String::from("fake")];
robinhood.get_options_market_data(token, instrument_ids).await;
} In particulat I play a little bit with it and I found the following example to reproduce the error use futures::StreamExt;
use log::info;
use serde::{Deserialize, Serialize};
pub struct Robinhood;
impl Robinhood {
pub fn new() -> Robinhood {
return Robinhood {};
}
async fn foo(&self, t: String, b: Vec<String>) -> Vec<String> {
// Looks like that the problem is generated from the following code
let b: Vec<&[String]> = b.chunks(2).collect();
let futures = b.into_iter().map(|it| {
return async { vec![] };
});
let results: Vec<String> = futures::stream::iter(futures)
.buffer_unordered(4)
.collect::<Vec<_>>()
.await
.into_iter()
.flatten()
.collect();
results
}
}
async fn foo_ok(t: String, b: Vec<String>) -> Vec<String> {
let b: Vec<&[String]> = b.chunks(2).collect();
let futures = b.into_iter().map(|it| {
return async { vec![] };
});
let results: Vec<String> = futures::stream::iter(futures)
.buffer_unordered(4)
.collect::<Vec<_>>()
.await
.into_iter()
.flatten()
.collect();
results
}
async fn robinhood_task() {
let robinhood = Robinhood::new();
foo_ok(String::new(), vec![]).await;
robinhood.foo(String::new(), vec![]).await;
}
fn main() {
async {
let handle1 = tokio::task::spawn(robinhood_task());
let _ = tokio::join!(handle1);
};
} |
I'm sure this is obvious to you, I just want to make sure it's documented somewhere: switching |
Unsure if this is the same exact issue, but I ran into a "higher-ranked lifetime error" when running https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=e8d2668ceca75a8ebf4f0c16cee04958. I tried to work around the failure by boxing it like suggested, but it still doesn't seem to work.
EDIT: if I pin & box the |
Had the same problem, but @awesomelemonade's example helped. It worked by pinning the // let objects = futures::future::try_join_all(futs)
// .await?
// .into_iter()
// .collect::<HashMap<Self::Address, C>>();
let f: Pin<
Box<
dyn Future<Output = anyhow::Result<Vec<(<Self as StorageProvider>::Address, C)>>>
+ Send,
>,
> = Box::pin(futures::future::try_join_all(futs));
f.await; |
@Tails @awesomelemonade you can skip the heap allocation with a helper function: fn assert_send<'u, R>(fut: impl 'u + Send + Future<Output = R>)
-> impl 'u + Send + Future<Output = R>
{
fut
}
let f = assert_send(futures::future::try_join_all(futs));
f.await All this, and more is present in that Discord discussion I mentioned: |
I was running into this problem in a project tonight (which prompted me to go through a bunch of its dependencies and enable I did update my nightly to latest, but I'm not sure what version I was on before. Not something more than a few days old, though, I'm pretty sure. |
To add another example of code that causes the error: trait Trait {
type Item;
}
impl<B, I, F> Trait for (I, F)
where
F: FnMut(I) -> B,
{
type Item = I;
}
struct Struct1<I: Trait>(I::Item);
impl<I: Trait> Struct1<I> {
fn new(i: I) -> Self {
todo!()
}
}
async fn af3() {}
async fn af2(_: &mut (), i: &()) {
let d = Struct1::new((i, |c| c));
af3().await
}
async fn af1() {
let a = Box::leak(Box::new(()));
let b = Box::leak(Box::new(()));
spawn(af2(a, b));
}
pub fn spawn<T: Send>(future: T) {
todo!()
}
I've hit this in a codebase of mine that uses an Somewhat unrelated: https://github.com/marxin/cvise helped a lot in getting this example down to what it is now (though you need to fix the syntax after using it, since it's designed for C). |
I face this issue with use futures::stream::StreamExt;
fn higher_ranked_lifetime_error(
) -> std::pin::Pin<std::boxed::Box<dyn std::future::Future<Output = ()> + Send>> {
Box::pin(async {
let vec_with_readers: Vec<Box<dyn std::io::Read + Send>> = Vec::new();
let _ = futures::stream::iter(
vec_with_readers
.into_iter()
.map(|_| futures::future::ready::<()>(())),
)
.buffered(1)
.count()
.await;
()
})
} while removing Could anyone point me to the workaround for the issue please? Workaround from #102211 (comment) will require changes for |
Or better yet, using @frxstrem's suggestion of replacing the closure for a function pointer and skipping copying things around unnecessarily: |
I'm facing the same issue after introducing Anyone know another workaround? |
In my case the issue was when registering a route handler with axum for To fix this, I had to change Another fix would be to not use async fn foo() -> Result<()> {
let xs = vec![...];
do_something_with_xs(&xs).await
}
async fn do_something_with_xs(xs: &[X]) -> Result<()> {
// Broken version:
let ys = futures::stream::iter(xs.iter().map(|x| do_something_with_x(x)));
// Working version when using xs: Vec<X>
// let ys = futures::stream::iter(xs.into_iter().map(|x| async move { do_something_with_x(&x).await }));
// Working version when using xs: &[X] without buffered
// let ys = futures::stream::iter(xs.iter()).then(|x| do_something_with_x(x));
tokio::pin!(stream);
while let Some(y) = ys.next().await {
// Do something with y
}
}
// You can keep the reference here
async fn do_something_with_x(x: &X) -> Result<Y> {
// ...
} |
i have a same problem.
the error:
|
Sorry, I do not understand. Is the problem being solved? |
Looks like @danielhenrymantilla's comment is the most up-to-date look into what's wrong here. It also looks like @vincenzopalazzo was previously assigned this issue, but has removed themself. It's now unassigned, but is part of a tracking issue for lifetime errors in async code. Based on Daniel's explanation, it looks like this is due to a bug. As he explained, auto traits (like
I'm not part of the team handling this, so I have no idea where this sits in their priority, but I do think this describes the current state. I highly recommend looking at Daniel's post (which I linked to at the start of this comment), which details a workaround. |
One potential workaround I've discovered for the This does introduce some copying and is only really practical when the source of the futures is synchronous (though the futures themselves can be asynchronous). However, this workaround has worked for me in a real world example and I figured I would share it in case it helps someone else. |
Instead of collecting the futures into a |
Running stable 1.64.0.
Unfortunately this happened in a complex bit of code that I can't easily reduce :(
The future returned by
handle_service.call(connection)
is definitelySend
, and I can work around the failure by boxing it:The text was updated successfully, but these errors were encountered: