big file download speed is 3 times slower than the python script

[mu0641]

The chunk length is always 16384, how can I receive more data at one time?
Now its download speed is 3 times slower than the python script, and the cpu consumption is also very high. Is there any good way?

#[tokio::main]
async fn main() -> std::result::Result<(), Box<dyn std::error::Error>> {

    let client = reqwest::Client::builder().build()?;
    let mut res = client
    .get(&url)
    .header("referer", &config.referer)
    .send()
    .await
    .unwrap();

    loop {
        match res.chunk().await {
            Ok(Some(chunk)) => match writer.write(&chunk).await {
                Ok(_result) => {

                  // This is always showing a chunk length of 16384, how can I receive more data at one time.
                    println!("block len：{}",chunk.len());
                    let new = std::cmp::min(
                        downloaded + (chunk.len() as u64),
                        total_size,
                    );
                }
                Err(err) => panic!("~~~{}-- {} --{} ", err, path, url),
            },
            Ok(None) => {
                println!("Finished！");
                break;
            }
            Err(e) => {
                // _ss.send(row.clone()).await.unwrap();
                break;
            }
        }
    }
}

[horacimacias]

did you try bytes() instead of looping over chunk() ?

[mu0641]

@horacimacias The logic behind bytes() should also use chunk() to loop and merge into a large byte array

[seanmonstar]

It's very likely that is how the server is sending chunks. The internal decoder is happy to read faster if the data is available, and would do so automatically.

[max-block]

@mu0641 Have you found a way how to change a chunk size?