Support loading GGUF model from memory buffer (not just file) #17309
calebnwokocha
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Problem: Currently, llama_model_load_from_file(...) requires a file path. Llama.cpp uses mmap or reads from disk.
Proposal: Add a function like llama_model_load_from_buffer(const void* buf, size_t size, llama_context_params params) to allow loading GGUF from an in-memory buffer.
Use Cases:
Beta Was this translation helpful? Give feedback.
All reactions