When downloading to 99.9x%, it may become very slow

[wszgrcy]

When downloading files, most of them can be downloaded quickly, but sometimes they get stuck at 99.9x%
Then I looked at the log and found that it took about a minute to move forward again, as if it was stuck somewhere

If the connection is disconnected at this time and the download is restarted, there is a high probability that the download will succeed and not get stuck, so it is unclear whether it is due to network issues or timeout settings?

Another issue that has been discovered is that when the download becomes slow and reaches 100%, it will continue to download?

Because it was accidentally triggered, there is no detailed log. If it is triggered next time, I will post a more detailed log and reply

   let obj: Partial<DownloadFileOptions> = {
      url: url,
      cliProgress: false,
      skipExisting: true,
      parallelStreams: 8,
      retry: {
        retries: 50,
        factor: 1.5,
        minTimeout: 1_000,
        maxTimeout: 5_000,
      },
    };

2025-01-14 16:36:53.851 [info] 99.91%
2025-01-14 16:36:53.852 [info] 99.92%
2025-01-14 16:36:53.853 [info] 99.92%
2025-01-14 16:36:53.853 [info] 99.92%
2025-01-14 16:36:53.854 [info] 99.92%
2025-01-14 16:36:53.858 [info] 99.92%
2025-01-14 16:36:53.858 [info] 99.92%
2025-01-14 16:36:53.872 [info] 99.92%
2025-01-14 16:36:54.053 [info] 99.92%
2025-01-14 16:37:01.467 [info] 99.92%
2025-01-14 16:37:01.468 [info] 99.92%
2025-01-14 16:37:01.739 [info] 99.92%
2025-01-14 16:37:15.292 [info] 99.92%
2025-01-14 16:37:15.567 [info] 99.92%
2025-01-14 16:37:41.554 [info] 99.93%
2025-01-14 16:37:41.735 [info] 99.93%
2025-01-14 16:37:41.916 [info] 99.93%
2025-01-14 16:37:42.188 [info] 99.93%
2025-01-14 16:37:42.189 [info] 99.93%
2025-01-14 16:37:42.189 [info] 99.93%
2025-01-14 16:37:42.190 [info] 99.93%
2025-01-14 16:38:37.213 [info] 99.93%
2025-01-14 16:38:37.485 [info] 99.93%
2025-01-14 16:38:37.485 [info] 99.93%
2025-01-14 16:38:37.485 [info] 99.93%
2025-01-14 16:38:37.486 [info] 99.93%
2025-01-14 16:38:37.486 [info] 99.94%
2025-01-14 16:38:37.487 [info] 99.94%
2025-01-14 16:38:37.487 [info] 99.94%
2025-01-14 16:38:37.487 [info] 99.94%
2025-01-14 16:40:23.710 [info] 99.94%
2025-01-14 16:40:23.711 [info] 99.94%
2025-01-14 16:40:23.711 [info] 99.94%
2025-01-14 16:40:23.711 [info] 99.94%
2025-01-14 16:40:23.711 [info] 99.94%
2025-01-14 16:40:23.980 [info] 99.94%
2025-01-14 16:40:23.980 [info] 99.94%
2025-01-14 16:40:23.981 [info] 99.94%
2025-01-14 16:40:23.981 [info] 99.94%
2025-01-14 16:40:23.982 [info] 99.94%
2025-01-14 16:40:23.982 [info] 99.95%
2025-01-14 16:40:23.982 [info] 99.95%
2025-01-14 16:42:26.230 [info] 99.95%
2025-01-14 16:42:26.591 [info] 99.95%
2025-01-14 16:42:26.592 [info] 99.95%
2025-01-14 16:42:26.592 [info] 99.95%
2025-01-14 16:42:26.592 [info] 99.95%
2025-01-14 16:42:26.861 [info] 99.95%
2025-01-14 16:42:26.861 [info] 99.95%
2025-01-14 16:42:26.862 [info] 99.95%
2025-01-14 16:42:26.862 [info] 99.95%
2025-01-14 16:42:26.863 [info] 99.95%
2025-01-14 16:42:26.863 [info] 99.95%
2025-01-14 16:42:26.863 [info] 99.96%
2025-01-14 16:44:28.928 [info] 99.96%
2025-01-14 16:44:29.290 [info] 99.96%
2025-01-14 16:44:29.471 [info] 99.96%
2025-01-14 16:44:29.471 [info] 99.96%
2025-01-14 16:44:29.739 [info] 99.96%
2025-01-14 16:44:29.739 [info] 99.96%
2025-01-14 16:44:29.740 [info] 99.96%
2025-01-14 16:46:32.170 [info] 99.96%
2025-01-14 16:46:32.351 [info] 99.96%
2025-01-14 16:46:32.351 [info] 99.96%
2025-01-14 16:46:32.352 [info] 99.96%
2025-01-14 16:46:32.352 [info] 99.96%
2025-01-14 16:46:32.352 [info] 99.96%
2025-01-14 16:46:32.619 [info] 99.97%
2025-01-14 16:46:32.619 [info] 99.97%
2025-01-14 16:48:35.250 [info] 99.97%
2025-01-14 16:48:35.251 [info] 99.97%
2025-01-14 16:48:35.251 [info] 99.97%
2025-01-14 16:48:35.252 [info] 99.97%
2025-01-14 16:48:35.252 [info] 99.97%
2025-01-14 16:48:35.252 [info] 99.97%
2025-01-14 16:48:35.253 [info] 99.97%
2025-01-14 16:48:35.538 [info] 99.97%
2025-01-14 16:48:35.539 [info] 99.97%
2025-01-14 16:48:35.539 [info] 99.97%
2025-01-14 16:48:35.540 [info] 99.97%
2025-01-14 16:50:37.751 [info] 99.98%
2025-01-14 16:50:38.112 [info] 99.98%
2025-01-14 16:50:38.112 [info] 99.98%
2025-01-14 16:50:38.378 [info] 99.98%
2025-01-14 16:50:38.379 [info] 99.98%
2025-01-14 16:50:38.379 [info] 99.98%
2025-01-14 16:52:40.631 [info] 99.98%
2025-01-14 16:52:40.991 [info] 99.98%
2025-01-14 16:52:40.992 [info] 99.98%
2025-01-14 16:52:40.992 [info] 99.98%
2025-01-14 16:52:40.992 [info] 99.98%
2025-01-14 16:52:41.257 [info] 99.98%
2025-01-14 16:52:41.258 [info] 99.98%
2025-01-14 16:52:41.258 [info] 99.99%
2025-01-14 16:52:41.258 [info] 99.99%
2025-01-14 16:52:41.259 [info] 99.99%
2025-01-14 16:52:41.259 [info] 99.99%
2025-01-14 16:52:41.259 [info] 99.99%
2025-01-14 16:54:43.331 [info] 99.99%
2025-01-14 16:54:43.874 [info] 99.99%
2025-01-14 16:54:43.874 [info] 99.99%
2025-01-14 16:54:43.875 [info] 99.99%
2025-01-14 16:54:43.875 [info] 99.99%
2025-01-14 16:54:44.141 [info] 99.99%
2025-01-14 16:54:44.141 [info] 99.99%
2025-01-14 16:54:44.142 [info] 99.99%
2025-01-14 16:54:44.142 [info] 99.99%
2025-01-14 16:54:44.143 [info] 99.99%
2025-01-14 16:54:44.143 [info] 99.99%
2025-01-14 16:56:46.391 [info] 99.99%
2025-01-14 16:56:47.017 [info] 100.00%
2025-01-14 16:58:49.092 [info] 100.00%
2025-01-14 16:58:49.634 [info] 100.00%
2025-01-14 16:58:49.635 [info] 100.00%
2025-01-14 16:58:49.635 [info] 100.00%
2025-01-14 16:58:49.636 [info] 100.00%
2025-01-14 16:58:49.636 [info] 100.00%
2025-01-14 16:58:49.899 [info] 100.00%
2025-01-14 16:58:49.899 [info] 100.00%
2025-01-14 16:58:49.900 [info] 100.00%
2025-01-14 16:58:49.900 [info] 100.00%
2025-01-14 16:58:49.900 [info] 100.00%
2025-01-14 16:58:49.901 [info] 100.00%
2025-01-14 16:58:49.901 [info] 100.00%
2025-01-14 16:58:49.902 [info] 100.00%
2025-01-14 16:58:49.902 [info] 100.00%
2025-01-14 16:58:49.903 [info] 100.00%
2025-01-14 16:58:49.903 [info] 100.00%
2025-01-14 17:00:52.333 [info] 100.00%
2025-01-14 17:00:52.779 [info] 100.00%
2025-01-14 17:00:52.780 [info] 100.00%
2025-01-14 17:00:52.780 [info] 100.00%
2025-01-14 17:00:52.780 [info] 100.00%
2025-01-14 17:00:52.781 [info] 100.00%
2025-01-14 17:02:55.033 [info] 100.00%
2025-01-14 17:02:55.394 [info] 100.00%
2025-01-14 17:02:55.395 [info] 100.00%
2025-01-14 17:02:55.395 [info] 100.00%
2025-01-14 17:02:55.395 [info] 100.00%
2025-01-14 17:02:55.396 [info] 100.00%
2025-01-14 17:02:55.660 [info] 100.00%
2025-01-14 17:02:55.660 [info] 100.00%
2025-01-14 17:02:55.660 [info] 100.00%
2025-01-14 17:02:55.661 [info] 100.00%
2025-01-14 17:02:55.661 [info] 100.00%
2025-01-14 17:02:55.661 [info] 100.00%
2025-01-14 17:02:55.662 [info] 100.00%
2025-01-14 17:02:55.662 [info] 100.00%
2025-01-14 17:04:57.733 [info] 100.00%
2025-01-14 17:04:58.278 [info] 100.00%
2025-01-14 17:04:58.544 [info] 100.00%
2025-01-14 17:07:01.156 [info] 100.00%
2025-01-14 17:07:01.421 [info] 100.00%

[ido-pluto]

What is happening is that IPull open multiple connection but once most of the file is downloaded, and only one chunk(default to 5MB) remains, it downloads the remaining chunk using a single stream (without parallelism).

You can adjust the default chunk size to something smaller, like 512KB–2MB, so it will try to parallel even at small size.

const downloader = await downloadFile({
    url,
    directory: ".",
    cliProgress: true,
    chunkSize: 1024 * 512 // 512 KB
});

Additionally, try enabling the CLI progress to observe the download speed when the file reaches 99.9x%

[wszgrcy]

Thank you for your reply. May I ask if it is possible to control downloads more finely, such as closing and reconnecting a download thread when its average speed falls below a certain threshold

I suddenly thought, if I manually pause and resume downloading, will I resend the request and have a chance to restore normal speed
like this

   downloader.pause();
    setTimeout(() => {
        downloader.resume();
    }, 5_000);

[ido-pluto]

You can but this is not going to help for 99.9x%.

It will not create a new connection; it will just pause it. I am planning in the next major release to do smart parallelization that will try to auto maximize your download speed...

I suggest you will try to adjust the chunkSize and see id that helps...
BTW: If you change the "chunkSize" of an already exits download, it will do nothing (the download structure ready created), try to delete the download and than change the "chunkSize"