Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
GitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
proposal: net/http: handle Seek error in range request by sending entire file #42173
In the io/fs proposal, I made the comment that if an fs.File did not support Seek then net/http couldn't serve it when presented by a range request. Many people pointed out that strictly speaking that statement is incorrect: it is always allowed for an HTTP server to respond to a request with a Range header by sending the entire content (the response headers clearly indicate whether ranges or the entire content are being sent).
That said, the net/http server behavior on Seek failure has been here for a long time, long before io/fs. While we could handle a Seek failure by responding with full content, it is less clear to me whether we should. Hence this issue, to make sure the question is properly evaluated.
There are two cases where a Seek failure can happen:
Clearly in both these cases (with the complication in the second), we could detect a Seek failure before sending any ranges and respond by sending the whole file.
On the other hand, I have vague memories of a browser-based PDF reader being exceedingly slow against Go servers because it was making range requests and getting the full content back and discarding all but the ranges it had requested. (Maybe this was before we supported Range at all, or maybe our Range implementation was sending full content; I don't remember.) So the browser was downloading the same, entire large file over and over and over. Yes, the bug was on the client side, but even a correct implementation in that case (notice the full response and save it) would still just appear to be very slow (but not exceedingly slow) when it downloads an entire (say) 100 MB PDF to display just the first page. It may be that any reasonable server should support range on files being served, and that sending the entire file as a pedantically correct response, instead of giving a clear error that would cause the server author to add Seek support, causes more problems than it avoids.
That is, if a client asks for 1 kB of a 1 GB file, it may technically comply with the spec to send the whole GB, but at the same time it may be a worse response than an error.
(The spec is written this way because clients have to deal with the possibility of an old server that has no idea what a range request is. But that doesn't mean Go pretending to be an old server is a good idea.)
I honestly don't know what's right here. What we can do is different from what we should do.
I vote error. If the client thinks the resource is large enough to request a Range, we should assume it is, and as you say, returning 4GB for a 4k request is bad. In-practice, IMO error is more often the better solution. And it is RFC-compliant to do so.
If this were common, I'd suggest a config option. But all modern filesystems can seek, so this should be rare. Configs for every rare error are untenable.
Agreed. If the server has already started sending data, there's no correct answer except to terminate the connection. In theory, Go could do a full read until it gets to the point of the seek; in practice, that would rarely be fast enough to be useful. In practice, this error case seems even rarer.
Apache Traffic Server (not Apache HTTP Server aka httpd) will error if any mid-read error occurs (as required), and if any initial read error occurs, will fall back to the parent/origin. But as a Caching Proxy, file reads mostly don't apply.
Apache Traffic Control's own Grove Caching Proxy will likewise treat read errors as not-in-cache. But it's written in Go, so the decisions here could potentially change that.
Also, the Go authors probably know this, but for anyone reading who doesn't: I verified,