Why GGUF supports mmap specially?

[AsakusaRinne]

According to the doc of GGUF, GGUF format has an advantage that it supports mmap, while ggml not. I'm so curious about it so I opened a discussion here.

In my thought, mmap maps an area of file to an area of memory. As long as the file content we want to map is contiguous, it will work. According to the code converting pth to ggml, it writes the numpy array data to the file, which I think indicates that the data area in the file is contiguous.

Maybe I mistaken something and I'll appreciate it if someone could correct me.😊

@philpax Please allow me to apologize for the interruption.😶‍🌫️ I think you are the most knowledgeable man about GGUF.

[Green-Sky]

It is not a difference, but a desired feature.

btw @philpax , here the GGML should be GGUF (i think):

Extensible: new features can be added to GGML without breaking compatibility with existing models.

sidenote: the GGJT version of GGML referenced was specifically introduced to make it mmap-able.

[philpax]

Fixed. And yeah, this is right - the GGJT format supported mmap, we wanted to carry that forward into GGUF and make sure it was a primary design consideration.

[AsakusaRinne]

Thank you for all your answers!