net/http, security: http.FileServer may inadvertently expose sensitive directories

[kevinburke]

The default http.FileServer(http.Dir(".")) implementation exposes all files and folders in the working directory. This includes directories like .git or .svn, which an attacker can use to recover the source code comprising the server, if for example people deploy by running a "git clone" and a "go install." This is how Heroku handles Go deployments and I am sure others do as well; it's common if you run Macs locally but deploy on Linux.

This is not a hypothetical attack, one estimate found 9700 of the Alexa top 1 million sites expose the .git directory: https://en.internetwache.org/dont-publicly-expose-git-or-how-we-downloaded-your-websites-sourcecode-an-analysis-of-alexas-1m-28-07-2015/

I am wondering whether the Go standard library should do anything to help prevent these inadvertent directory exposures. Our hands are slightly tied by the current API, which exposes everything and can't change.

Some options:

• Do nothing

• Document the potential vulnerability and leave it at that.

• Add http.SafeDir or similar, that 404's on any file beginning with a .. This should cover most private files, including .htpasswd.

• Add SafeDir to httputil.

• Modify http.Dir to exclude files that begin with a .. This would break backwards compatibility, and seems like a bad idea.

[bradfitz]

Documenting seems fine.

[kevinburke]

Turns out Heroku deletes that dir but you can see a few others on this sample app, including the raw binary for the server https://go-simple-http-server.herokuapp.com/

[gopherbot]

CL https://golang.org/cl/46468 mentions this issue.

[crazcalm]

I recently wrote a blog post on how to create a file server that both hides and denies access to dot files (post).

I am willing to write this documentation (and an example_test.go for the net/http package). Can someone point me in the right direction as to where the documentation for this issue should be written? Also, I would like to know how the example_test.go files work and if there are any standards when writing one.

Any help would be gladly appreciated! Thanks!

[bradfitz]

See https://blog.golang.org/examples

[crazcalm]

@bradfitz Thanks!

[heschik]

Hey @crazcalm, we met at the NYC meetup. Good to see you again.

Was there something specific you wanted to add to the documentation? The comment on Dir seems pretty clear to me.

I've never written an example, but the final code snippet in the blog post looks like a pretty good start to me. It might be nice if it included some sample output, but that would require a fair bit of setup. Feel free to send me the CL to me for a first review if you'd like.

[crazcalm]

Hey @heschik! Nice to see you again!

I agree that the comment on http.Dir seems pretty clear.

In terms of the example, after reading the godoc post, I have a fairly good idea of how to go about writing the example. However, I am having trouble thinking of a reasonable way to add sample output. Given that the sample out will be checked against the actual output, we would need a directory to have dot files and to be static in nature.

Unless the example is not of hiding dot files and is of a different type of filter. For example, if I filter go/src to only allow items that begin with "c" to be shown, then the output would be relatively static. However, finding the path to every user's go/src directory would be a new problem that the example would have to solve.

Is the sample output worth the added complications? Thoughts?

@heschik How should I go about sending you the CL? And what does CL stand for (I am currently interpreting it as Pull Request, but I not sure that I am right)?

[gopherbot]

Change https://golang.org/cl/127576 mentions this issue: net/http: Added an example of creating a custom FileSystem

[crazcalm]

@heschik @bradfitz The above pull request is the example of creating a custom FileSystem to hide dot files. There are no output strings and I have set the example method name so that it shows "Example (CustomFileSystem)"

Let me know what changes need to be made.

Thanks for all the help!