Experimental: FUSE-based Algorithmia FileSystem
A user-mode virtual filesystem backed by the Algorithmia API. Basically, it handles filesystem requests by turning them into API calls and lazily building a local cache of remote resources. The end result is that you can mount Algorithmia data to a local directory, and use standard file operations to work with Algorithmia data.
Screenshots demonstrate basic traversal and read operations from CLI and file explorer:
Currently, this is only an experimental filesystem. You should NOT rely on it for critical work.
In it's current state, it works as a basic read-write filesystem with several caveats:
- Connector support is too limited to be useful, and better support is blocked by the API - see Issue #1)
- Writes persist when closing the last open handle to a file. If the close fails, it's likely the data isn't persisted.
- Directory listing is permanently cached, so if you change a directory's contents outside of AlgoFS, you have to unmount and remount AlgoFS before those changes appear. Issue #4
- The entire inode and file cache lives in RAM, so if you download a 4GB file, it will occupy 4GB of RAM until it is closed.
- Testing so far is very limited.
- General network filesystem caveats apply, e.g. some file operations may appear slow
Build, Test, Run, Debug
To build and test (tests are pretty barebones):
$ cargo build $ cargo test
To mount the filesystem:
$ mkdir ~/algofs $ target/debug/algofs ~/algofs
algofs executable will print all the current debug output,
so currently it works best to browse the
~/algofs from another terminal.
Note: some shell enhancements can cause a lot of extra listing operations. And file explorers may trigger a lot of extra reads to preload or preview files.
To stop algofs, unmount it as root. (Note: killing
algofs will stop request handling, but leaves
~/algofs as a volume with no transport connected).
fusermount -u ~/algofs # or `sudo umount ~/algofs`