Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory used all up when downloading model #58

Open
whami opened this issue Mar 16, 2023 · 1 comment
Open

Memory used all up when downloading model #58

whami opened this issue Mar 16, 2023 · 1 comment

Comments

@whami
Copy link

whami commented Mar 16, 2023

When downloading the model it seems like it wants to load the whole file into RAM before writing it to the disk. Could the download method be changed, so that it gradually downloads and saves the file, without putting it completely to memory.

This is how it looks like on my machine:

grafik

while downloading the model:
grafik

The memory consumption grew gradually during the download till it stops working because the memory is full.

@mathiasfoster
Copy link

Files appear to be downloaded in chunks of 15.2gb at a time.
Memory usage increases while the chunk downloads, then drops down afterwards.
You'll need enough memory spare to download 15gb at once (or else source the weights via torrent, but I'm not sure which torrent or how to add this to llama.cpp).

Here's a screenshot showing my memory usage dropping after one chunk finished downloading:
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants