Optimize file download by 512kb chunk #193

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

sazid merged 1 commit into dev from api-cert

Jun 22, 2022

Member

sazid commented Jun 22, 2022

PR Type

Refactor

PR Checklist

Tests for the changes have been added / updated.
Documentation comments have been added / updated.
(Team) Label with affected action categories and semver status.

Overview

This PR increases the speed of attachment downloads.
Benchmark for downloading large files: bigfile.zip (575.2 MB)

chunk size	time (seconds)
1 byte	56.872249603271484
1 KB (1024)	7.789804697036743
512 KB (512*1024)	0.2882239818572998
1 MB (1024*1024)	0.51566481590271
8 MB (810241024)	0.46999025344848633
1 GB (102410241024)	0.6988034248352051
shutil.copyfileobj	0.6343142986297607

We can see that 512KB is the ideal chunk size for large file downloads.


          Optimize file download by 1mb chunk

07d083f

sazid added the node-internals label

sazid requested review from Muntasib-creator, Oytizzo and Sakib75

June 22, 2022 08:21

sazid self-assigned this

sazid merged commit b5220a3 into dev

sazid deleted the api-cert branch

June 22, 2022 08:22

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels