Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Implement `maxUploadFileSize` #256
Progress towards Production ready node.js HTTP server
When a content-length header is provided (almost always the case) then it is checked against the hub's max configured size. If the size is too large, the http request is immediately rejected with a specific error for the client (blockstack.js) to be able to handle appropriately.
Additionally, while the a request payload is being streamed, the size is monitored and if the content-length is exceeded, the stream is immediately destroyed.
If a content-length header is not provided, then the max hub size limit is used instead. This is to allow for clients that do not necessarily know the content-length in advance. The
TODO: Memory load testing during large file writes to ensure there are no performance regressions.
Near full test coverage. SuperAgent is a PITA and difficult to spoof malicious client behavior. Some error paths are not covered in the http code reporting layer, but covered in the error handling layer right beneath it.
@@ Coverage Diff @@ ## develop #256 +/- ## =========================================== + Coverage 80.68% 81.03% +0.34% =========================================== Files 21 21 Lines 1553 1592 +39 Branches 271 278 +7 =========================================== + Hits 1253 1290 +37 - Misses 225 226 +1 - Partials 75 76 +1
Thanks for reviewing @jcnelson, the comment about chunked-encoding led me down a productive rabbit hole, and found some potential HTTP
I may remove the requirement for the header -- pending additional research and testing.
Confirmed the hub and reader is successfully running. Overall, the PR respects upper file limits but doesn’t respect lower limits.
3 sample files are needed:
File1 upload (Successful):
I removed the requirement for providing a content-length header. The behavior of this header cannot always be controlled. It is a forbidden header -- regular js web clients cannot set the header programmatically. The value sent by the browser is influenced by negotiated content encoding and compression. Additionally, it excludes future uploading features that do not necessarily know the content length in advance.
If the header is provided by a normal-behaving and non-streaming client, then it results in nice errors immediately provided to the client. If the header is not provided, then the error is provided to the client after the hub-configured limit has been exceeded.
However, the possibility of the header not being provided is a situation in which I think we should reasonably expect and allow for.