Streaming multipart parser without tmpfiles

[gwicke]

The multipart parser
(https://github.com/mcavage/node-restify/blob/master/lib/plugins/multipart_parser.js) uses formidable currently. Formidable creates tempfiles for each file upload, and never clears those. Even with a tmpreaper in place, this can be exploited for a DOS by sending requests with a lot of file uploads in a short time.

https://github.com/mscdex/busboy looks like a promising alternative to formidable. It supports streaming and dropping file uploads. The interface for streaming would need to be worked out- maybe by passing in something like req.form?

[techguydave]

Any word on this? I'm looking to use multipart streaming to allow uploading a file to AWS S3 through the API.

[epegzz]

+1 for file streaming support :)

[cxreg]

Is this ever likely to happen?

[gwicke]

FWIW, we have been using https://github.com/mscdex/busboy in other non-restify projects at Wikimedia since, and is has worked well for us (apart from some bugs early on).

[bluesockets]

👍

[prabhu]

I can't bear formidable as well. Busboy seems stable at least for me.

[micahr]

If anyone is interested, we would welcome a PR to the plugins repository to either change the implementation or swap out the module that it uses.

[prabhu]

Some of us do contribute. However, I don't think a simple PR can provide this feature unless there is some refactoring effort along with unit tests. I see the following issues with the current design.

• bodyParser.js loads all the parsers statically from the same directory. Instead there should be way to register the parser that way people can override based on the mime or content type or url or some other value
• multipartBodyParser.js should be like a base class with generic api methods. The default implementation class could use formidable and then the community can create their own parser with their favourite library
• If possible please adopt ES6 and babelify the code to support older node. Cant bear to see function within functions within functions with return (false) and return next() as well as return (object)

Given the popularity of this framework there should be some kind of agreement regards to the changes even before someone could invest their time for a PR.

[DonutEspresso]

@prabhu All of those suggestions sound great - feel free to kick off a discussion in the plugins repo, as plugin development has stopped in the main repo here.