net/http: document security considerations for serving internet traffic

[mmcloughlin]

The net/http/pprof package implicitly registers HTTP handlers through its init() function. I argue this implicit behavior is too subtle and may contribute to people inadvertently leaving such endpoints open. Some IPv4 scans reveal a non-trivial number of pprof endpoints exposed (http://mmcloughlin.com/posts/your-pprof-is-showing). Since Go 1.9 an exposed pprof endpoint leaks source code.

I would like to spark a discussion on the possibility of moving to an explicit handler registration model. It is not clear to me that this would qualify for the Security exemption of the Go 1 compatibility guarantee, but I thought it was worthy of mention. Perhaps a warning in documentation would be enough.

[dlsniper]

It only registers those handlers implicitly if you use the default http server instance. If you use a custom one, that doesn't happen. The problem is that people are using http.ListenAndServe without understanding its implications, not net/http/pprof imho. Furthermore, the default server usually needs tweaking when exposed to the Internet.

[mmcloughlin]

This is ultimately user error as you say. People are indeed using http.ListenAndServe and net/http/pprof without understanding its implications. I'm suggesting that this is an easy and understandable mistake, and we should strive to make the consequences as benign as possible.

If we were to address this by making backwards incompatible changes, I would suggest that replacing the init() function of net/http/pprof with a RegisterHandlers() (or similar) would be the most effective.

Arguably this is not serious enough to break compatibility. I think at least a warning in documentation is warranted.

[docmerlin]

I concur about a warning in documentation!

[mappu]

I understand the function names and file names are embedded into all binaries for panics and for reflection. I don't even particularly mind the pprof behaviour of attaching to the default mux.

But leaking source code is beyond the pale.

How does pprof get this information? Is it compiled into 1.9 binaries or loaded externally? If the former, is it only compiled into 1.9 binaries when pprof is used?

[dlsniper]

@mmcloughlin shouldn't you update your blog post if you think that the better solution is to not use the default ListenAndServe / http server and instead initialize your own?

As for Go documentations, PRs are welcomed by the team, so improving the usage example of net/http/pprof would be nice as well.

[4ad]

The source code is not embedded in the binary! go tool pprof has to have access to the source code in order to print the source code. The binaries only contain source code coordinates for symbols.

You ran all your tests on localhost, where pprof had access to the source of the server.

Of course, this does not mean the current functionality is not a footgun. I wish net/http/pprof would not work like this and I wish it didn't have a default mux, but it's too late to change any of that.

[mmcloughlin]

@4ad You're completely right, apologies for my mistake. Classic case of getting excited at a result and not doing the due diligence. Thanks for pointing this out. I'll update the post accordingly.

As you say there are still risks here people should be aware of.

@dlsniper I believe the Prevention section of the post covers this.

[rsc]

There are a number of packages that automatically register handlers under /debug/.... We chose that prefix specifically so that servers that serve direct internet traffic can put a handler in front of the http.DefaultServeMux to filter those out for untrusted clients.

@FiloSottile also has some tips about what to do as far as configuring other HTTP settings to avoid problems when serving direct internet traffic.

Probably a doc fix is correct, and probably http.ListenAndServe would be a good place, since (probably?) more people read the ListenAndServe docs than the pprof docs. Or maybe a link to a wiki page? Unclear.

[rsc]

It seems like perhaps the right next step is for someone to write a wiki page, and then the ListenAndServe docs can have a short link to the page. Putting lots of detail into the doc comment doesn't make as much sense as a wiki page we can update out-of-band.