Sparse file local cache

[kuenishi]

With adding local_cache=True keyword to opening a zip FS e.g. using from_url() , a local cache file is created in a path (by default, ~/.cache/pfio/ ). The cache file is a named temporary file, but potentially, it can easily be persisted locally. It costs the local disk space, but the amount is just as same as the total amount of data transferred over the network to the client.

This will be a major step toward working around #271, as well as improving latency on reading a file from remote file system such as S3.

You can see the difference of space consumed like this:

$ ls -lh  ~/.cache/pfio/tmpjwvpycrg
-rw------- 1 kota kota 155G Jun  6 12:29 /home/kota/.cache/pfio/tmpjwvpycrg
$ du -h  ~/.cache/pfio/tmpjwvpycrg
184M    /home/kota/.cache/pfio/tmpjwvpycrg

Concurrency control

• For threads: while pread() and pwrite() are thread-safe, other internal data structure such as known ranges is not thread safe. This is future work.
• For processes: POSIX doesn't guarantee consistent read and write against single file by multiple processes; and little is also supported in Linux. Same technique as we do in pfio.cache.MultiprocessFileCache can be applied to the class CachedWrapper(). This is also future work.