Intended functionality of gguf_init_from_buffer

[gartia]

I'm in the process of implementing gguf_init_from_buffer and need some guidance on the expected behavior.

Here are the options I'm considering:

• Duplicate the buffer: Manage a separate copy of the buffer internally. My initial implementation takes this approach, but it may not be optimal due to the high memory demands of duplicating large models.
• "Steal" ownership of the buffer: Assume ownership and responsibility for freeing the buffer. This could potentially reduce memory usage.
• Map directly to the buffer: Use the provided buffer directly without creating a copy. This approach would minimize memory usage but could complicate buffer management.
• Alternative approach: Any other suggestions or best practices that could be more suitable?

Given the potential for large model sizes and the significant memory overhead involved, I am particularly concerned about the efficiency of the current approach where buffers are duplicated. This can result in double the RAM usage temporarily, which is not ideal.

I'm looking for input on the most appropriate approach that aligns with the overall design principles of ggml.

[slaren]

I don't think we can add an option to take ownership of the buffer because we don't know what function needs to be used to free it. I think that a good option could be to use the no_alloc parameter in the same way it behaves already. If it is set to true, memory for the tensors would not be allocated, and it is the responsibility of the application to initialize the tensors. To do so, you would only need to set the data pointer of the tensors to their location in the buffer, either directly if using the old CPU-only API, or with ggml_backend_cpu_buffer_from_ptr and ggml_backend_tensor_alloc if using ggml-backend.