Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Best chunk-size dynamics/management method? #23416

Closed
IEWbgfnYDwHRoRRSKtkdyMDUzgdwuBYgDKtDJWd opened this issue Dec 15, 2019 · 1 comment
Closed

Comments

@IEWbgfnYDwHRoRRSKtkdyMDUzgdwuBYgDKtDJWd
Copy link

@IEWbgfnYDwHRoRRSKtkdyMDUzgdwuBYgDKtDJWd IEWbgfnYDwHRoRRSKtkdyMDUzgdwuBYgDKtDJWd commented Dec 15, 2019

Checklist

  • I'm asking a question
  • I've looked through the README and FAQ for similar questions
  • I've searched the bugtracker for similar questions including closed ones

Question

As of right now, I have a bash script with http-chunk-size as a variable, so I can just do

script.sh download.url 300k

and if that is too slow, or if i can go faster with a larger chunk size, i will just terminate the script and retry it changing the chunksize variable. after a few tries, i settle on a sweet spot.

there is no science to this, and the best chunk size is always dependent on multiple factors like which server i am downloading from, time of day, etc etc.

i was thinking of going full blown rube goldberg machine with a script that downloads with a chunk size for a few seconds and quits, logs the time/speed of the small bit downloaded, pipes that into a variable, then does it again with a different size.. and after i have two different test runs with different chunk sizes, pipe that into an equation to find the average and come up with a best guess for the best chunk size off of that..

but wanted to check here before i do all that crap. thanks.

@IEWbgfnYDwHRoRRSKtkdyMDUzgdwuBYgDKtDJWd

really not seeing the duplicate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.