Using axum and reqwest to upload a file to cloud bucket, streaming it

[frederikhors]

I'm also opening this question here with the lively hope of being able to resolve it with your invaluable help.

Stackoverflow question is here: https://stackoverflow.com/questions/74815430/using-axum-and-reqwest-to-upload-a-file-to-cloud-bucket-streaming-it.

I've created a small Github repo that mimics exactly what I'm trying to achieve: streaming file uploads to both file systems and the cloud (S3) using axum (and reqwest).

I can change all the code you see. There is no restriction. All signatures were invented by me who are in my early days with the awesome Rust.

If you want I can also eliminate reqwest, as long as I understand what to use instead! 😄

Can you help me understand how to fix the only error left?

I think that the ``static` bound multer applies is causing this. Is that bound strictly needed? Can we avoid it?

error[E0521]: borrowed data escapes outside of associated function
  --> src\adapter_s3.rs:56:19
   |
41 |     async fn put_file<'a>(
   |                       -- lifetime `'a` defined here
42 |         &'a self,
   |         -------- `self` is a reference that is only valid in the associated function body
...
56 |             .body(reqwest::Body::wrap_stream(ReaderStream::new(reader)))
   |                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |                   |
   |                   `self` escapes the associated function body here
   |                   argument requires that `'a` must outlive `'static`

This is the relevant code I think:

async fn put_file<'a>(
    &'a self,
    filename: &'a str,
    reader: Pin<Box<dyn AsyncRead + Send + Sync + 'a>>,
) -> Result<()> {
    let presigned_url_duration = Duration::from_secs(60 * 60);

    let action = self.bucket.put_object(Some(&self.credentials), filename);

    let client = reqwest::Client::new();

    client
        .post(action.sign(presigned_url_duration))
        // .body(reader)
        // .body(reqwest::Body::wrap_stream(reader))
        .body(reqwest::Body::wrap_stream(ReaderStream::new(reader)))
        .send()
        .await?
        .error_for_status()?;

    Ok(())
}

UPDATE:

I tried with hyper too and it's the same error obviously:

hyper::Client::new()
    .request(
        hyper::Request::builder()
            .uri(action.sign(presigned_url_duration).to_string())
            .body(hyper::Body::wrap_stream(ReaderStream::new(reader)))
            .unwrap(),
    )
    .await
    .unwrap();

error[E0521]: borrowed data escapes outside of associated function
    |
66  |     async fn put_file<'a>(
    |                       -- lifetime `'a` defined here
67  |         &'a self,
    |         -------- `self` is a reference that is only valid in the associated function body
...
124 |                     .body(hyper::Body::wrap_stream(ReaderStream::new(reader)))
    |                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |                           |
    |                           `self` escapes the associated function body here
    |                           argument requires that `'a` must outlive `'static`

[SergioBenitez]

Apologies, but I'm struggling to see why this was posted here. Where is multer involved?

[frederikhors]

In axum.