Parser seems to truncate values over 0x7F from body attribs

[gdw2]

(Originally posted here: expressjs/express#1855)

Below is a small illustration. In both cases I'm sending 4 bytes. In the first case (which has a value over 0x7F), the size is incorrectly reported as '1'. In the second case (no values over 0x7F), it is correctly reported as '4' and all the data is intact.

Server:

app.post('/api', multipartMiddleware, function(req, res){
    var body = req.body.name;
    console.log('body',req.body);
    console.log('req body len', body.length);
    res.send("respond with a resource");
});

Python test client:

>>> requests.post('http://localhost:3000/api', files={'file':open('foo','rb')}, data={'name':pack('!HH',0xF0,0x0)})
<Response [200]>
>>> requests.post('http://localhost:3000/api', files={'file':open('foo','rb')}, data={'name':pack('!HH',0x0F,0x0)})
<Response [200]>

Express output:

Express server listening on port 3000
body { name: '\u0000' }
req body len 1
POST /api 200 9ms - 23b

body { name: '\u0000\u000f\u0000\u0000' }
req body len 4
POST /api 200 3ms - 23b

[andrewrk]

This looks like a rather nasty bug. Thanks for the report. Feel free to start working on a fix and make a pull request; otherwise I'll get around to it eventually.

[gdw2]

It looks like I just need to be extra vigilant at setting the encoding (which defaults to utf8) to 'binary' in all the required places, such as...

var form = new multiparty.Form({'encoding':'binary'});

Case closed.

[andrewrk]

ah, I should have known that. Thanks. It's been a while since I've messed with this stuff; my head is full of audio-processing and 3D graphics lately :)