-
Notifications
You must be signed in to change notification settings - Fork 18k
io/fs: add func Sub(fsys FS, dir string) FS #42322
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
The main issue with this approach is the whole optional method thing; the moment you use this helper all additional FS methods are lost. To do it "correctly", you end up having to do what httpsnoop had to do to deal with the optional interface problem with For I'm still not certain if this is a "problem" with how static serving works in |
In the comments on the io/fs proposal, as I understood it, the consensus was that the "right" way to deal with the optional interfaces was to implement them all and then return |
Forgive me, I meant |
@zikaeroh The general recommendation for optional methods is to add them all and fall-back to the appropriate helper in If there is an optional interface for which this approach doesn't work, details would be very useful (probably in a separate bug) because it would point to a serious design issue which should be discussed before go 1.16 is released. I don't know of any such instance yet, but that doesn't mean it doesn't exist. In particular, I don't see any problems in interactions with optional interfaces I know about and a hypothetical I think this proposal is a good idea and I would very much like to see it happen. To me, this is a fundamental primitive of composability for file systems and I would like to see it in |
/cc @rsc |
My initial instinct was that this would be useful because it could add more safety when trying to "sandbox" parts of the filesystem. But then I remembered that in the (apologies for poorly formatted example; written directly in browser.) type WPFS struct {
inner FS
prefix string
}
func (fs WPFS) Open(p string) (File, error) {
return fs.inner.Open(inner.prefix + "/" + p)
}
// same thing but for ReadFile, Stat, ReadDir, Glob, and maybe Rename, OpenFile?
func WithPrefix(f FS, p string) WPFS {
return WPFS{f, p}
} |
No, not a subtle reason. The non-subtle reason is what I mentioned above: It's a fundamental primitive of composition, so making it available is akin to other top-level functions in the |
An alternative name could be Chroot |
When we talked about adding this during the proposal discussion on Reddit, I was thinking it could be called just fs.Sub, as in sub-tree. Chroot is a bit too obscure (but accurate!), and WithPrefix is maybe too much about the mechanics (and maybe inaccurate! The argument to Open is without the prefix). I was thinking we could put this off until the next release and get more experience with io/fs, but I agree that it is a critical piece to have to use with embedding. Does anyone object to adding |
What should the SubFS interface look like? |
I agree that the interface should return an error, because if, for example, |
It's a lot more awkward to call Sub compared to http.StripPrefix if it returns an error. |
@carlmjohnson, no there is no "not implemented". If an implementation wants to provide a Sub but doesn't know how, it can call fs.Sub. |
Based on the discussion, this seems like a likely accept, and for Go 1.16 so that it is part of the initial FS API. |
This comment has been minimized.
This comment has been minimized.
@rsc I'm thinking about an |
@icholy No, you pass the wrapped file-system to |
I think the proposal is useful, but I think it should be documented that this is not a security features, as even just reading a symlinked file allows to escape the filesystem root (in fact, at some point we might want to add a |
If fs.Jail is a thing, maybe it matters less that fs.Sub has no error checking. fs.Jail could complain that you don't have permission to use a directory in the first place, etc. Having them separate would also mean that fs.Sub wouldn't have to clean all paths of |
But if fs.Sub is less secure than fs.Jail, would we want to encourage its use in http.FS? |
http.Dir has the same caveat. I think that's fine. |
No change in consensus, so accepted. |
I went to implement this, and it does seem like we need the error result if only for diagnosing invalid arguments properly. |
Change https://golang.org/cl/274856 mentions this issue: |
Originally posted by @zikaeroh in #41191 (comment)
Trying this out now; one wart with the
go:embed
directive is that if I embedbuild/*
, the filenames still have the prefixbuild/
. If I want to then serve that directory viahttp.FS
, there's no easy way to add the prefix that's required to access them if needed (without writing a wrapper, which then hits the problem of needing to list out every potential method that the FS may have...).e.g.:
I know the intent is that one could write
go:embed foo/* bar/* baz.ext
and get all of those files, but I think it's going to be very common to simply embed a directory and serve it as static assets via the http package. I expect this to be a gotcha as people switch from things likehttp.Dir("static")
orpkger.Dir("/internal/web/static")
where the prefix is already handled, to the newembed.FS
.I'm not really sure how to file this, as it's sort of an interplay with
embed
,io/fs
, andnet/http
.My comment on @zikaeroh's comment:
I think the best way to resolve this is by adding a general purpose fs.WithPrefix helper that creates a new FS that is restricted to the given subdirectory prefix. The example above would become:
I think this should be in FS so that it can implement optional interfaces as they're invented. I think it will have general applicability for things like creating a zipfile FS and restricting it to a subdirectory and whatnot.
The text was updated successfully, but these errors were encountered: