Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

s3.put can truncate body if it's a string that gets utf8 encoded #55

Closed
mattmonson opened this Issue Aug 8, 2012 · 2 comments

Comments

Projects
None yet
2 participants
Contributor

mattmonson commented Aug 8, 2012

s3.put seems to be truncating some of my data when I call it with body as a string that has some unicode characters in it.

I believe the core of the issue is in internals.js in makeRequest (lines 128 & 129):

case 'string':
    options.headers['content-length'] = body.length;

and lines 365 & 366):

if (typeof body === 'string') {
    request.write(body);

When we actually write body it gets utf8 encoded, but the bit that sets the content-length header doesn't take that encoding into account. This works in most scenarios because the utf8 encoding of the string is usually just the same as the string.

I think the fix would be to change line 129 to:

    options.headers['content-length'] = Buffer.byteLength(body);

Am I understanding all of this properly? Is that what's actually going on here? I've confirmed that this proposed change fixes the issue I'm seeing.

Owner

SaltwaterC commented Aug 8, 2012

Good catch. String.prototype.length returns the number of characters while the HTTP request expects the number of bytes in the Content-Lenght header.

Contributor

mattmonson commented Aug 8, 2012

I submitted a pull request with the fix :)

@SaltwaterC SaltwaterC closed this in bddb9a5 Aug 8, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment