local cache file-system #48
Description
We could create a file-system type, which makes local copies of any file which is opened for reading on a given other file-system: download-on-first-access. That would allow for local, and therefore fast, access to some parts of a much bigger data-set. This is probably not much work, but we would need to decide how and where to to store the files.
The MMap file-cache also demonstrates only downloading those blocks of a file which are accessed into a sparse file. That could also be implemented for absolute minimal storage.
@rabernat , this is the kind of thing you had in mind, and it is an interesting idea, and probably not hard to implement.