Repository navigation
Replies: 1 comment 2 replies
|
Thanks for testing this -- you're right about URLSession. Speeds this low on a single connection aren't typical for HF, though. For most people one connection already uses the full bandwidth, so parallel ranges would add a lot of complexity for little gain. A limit like this usually comes from the network (ISP, proxy, etc). Are you using a proxy or mirror? And how fast is Not planned for now, but if more people run into this, we'll look at it again. |
2 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Downloads in Llama use one HTTP connection per file. On my connection, HF's CDN limits every connection, so most of my bandwidth is wasted.
I did a real test on this, and the result is clear: downloading the same file as several byte ranges in parallel can improve the speed a lot, while a single stream cannot use up the bandwidth.
Run 1 stream 8 ranges
1 0.52 MB/s 2.67 MB/s
2 1.54 MB/s 9.37 MB/s
M5 Max, macOS 27.2, same 8.7 GB GGUF, 45 s per run, runs alternated.
But just sending many Range requests through one URLSession is not enough. URLSession sends them all on a single HTTP/2 connection, so they still share this connection's limit. Every range needs its own separate connection.
So please add this support: large files can be split into several ranges and downloaded at the same time, each range with its own connection, and then joined together at the end.
Say one more thing: original draft is based on Chinese writing, used AI translation, and the translation back to Chinese is checked by me manually.
All reactions