fileserver: Support virtual file systems #4909
Conversation
This change replaces the hard-coded use of os.Open() and os.Stat() with the use of the new (Go 1.16) io/fs APIs, enabling virtual file systems. It introduces a new module namespace, caddy.fs, for such file systems. Also improve documentation for the file server. I realized it was one of the first modules written for Caddy 2, and the docs hadn't really been updated since!
(Since we do not use a root directory, it is dynamic.)
|
Caddyfile support added here: ebd6abc |
|
@mholt New to caddy and don't see this mentioned in the docs - do you have an example on how to serve content directly from an archive file (e.g. .zip or .tar.gz)? |
|
@robbymilo Someone will need to write a Caddy plugin to do it. I'd recommend using https://github.com/mholt/archiver. I think @WeidiDeng has been looking into this, or maybe even already done it! |
|
It's very easy to write one, basically replace embed.FS in caddy-embed with archive.FS, and create filesystem from archive file path. ArchiveFS is too experimental to be used in production systems imo, afero is an alternative that I use in my custom file server. |
|
@WeidiDeng ArchiveFS is new, but I don't want to discourage people from using it, otherwise it'll never be "production ready" 🙃 Thanks for helping me improve it -- I need to get back to those PRs soon! |
This change replaces the hard-coded use of
os.Open()andos.Stat()withthe use of the new (Go 1.16)
io/fsAPIs, enabling virtual file systems.It introduces a new module namespace,
caddy.fs, for such file systems.Also improve documentation for the file server. I realized it was one of
the first modules written for Caddy 2, and the docs hadn't really been
updated since!
Anyway, this is super cool, since it enables Caddy's file server to serve not-so-static files, or files on remote resources like cloud storage or network devices. (S3? Database? No problem now.) It can serve files from any virtual file system, i.e. any Go type that has
Open()andStat()methods. So even if the content isn't strictly static, if the virtual file system can assemble the contents with a call toOpen(), Caddy doesn't care whether it's static or not. Dynamic file server!The first
caddy.fsmodule I might implement (other than the unexported, implicitly-enabled, 2-linerosFSin this PR) isarchiver.FileSystem()so that Caddy can serve files from zip/tar archives including transparently decompressing its contents!I might still need to tweak a few things about this before finalizing.
Closes #3720