support binary data via overrideMimeType() (or other means of not assuming the returned data is utf8) #24

Open
asutherland opened this Issue Mar 6, 2012 · 6 comments

Projects

None yet

6 participants

@asutherland

https://developer.mozilla.org/En/XMLHttpRequest/Using_XMLHttpRequest#Receiving_binary_data_in_older_browsers documents using:

req.overrideMimeType('text/plain; charset=x-user-defined');

as a way to force web browsers to not treat the returned data as utf-8. This is useful if you are doing something like trying to fetch an image in web browsers and also would like that same code to work in node.

Currently, this library forces the data stream to be interpreted as utf-8 which is potentially lossy, and definitely lossy in the case of png images. Specifically, the first byte 0x89 when round-tripped through utf-8 gets converted into the replacement character 0xffd and comes back (utf-8 encoded) as 0xef 0xbf 0xbd.

@driverdan
Owner

Thanks for finding this. There are 2 missing features in node-XHR here. It's missing overrideMimeType and it's missing proper encoding detection.

@pwnall pwnall referenced this issue in dropbox/dropbox-js Nov 26, 2012
Closed

Binary file support in node.js #28

@pwnall

overrideMimeType is mentioned as a hack. I would suggest implementing responseType = "arraybuffer" instead (MDN reference), because node.js supports typed arrays.

@gabrielhurley

+1 on adding support for binary transfers, FWIW. I just ran into this and ended up just hacking around it usng node's http.request directly since there's no way to change XMLHttpRequest's forced utf8 encoding.

A cursory look through the code looks like the two places that need changes are on the call to setEncoding and on the write call, where it needs the second argument for the encoding.

@nkzawa

Is this still an issue? Can I send Buffer or ArrayBuffer?

@gaurav21r

Any Updates on this? I came from here

http://stackoverflow.com/questions/29276660/unable-to-download-file-from-google-drive-using-api-node-js

I see someone from Dropbox also having this issue. The above question is regarding downloading files from Google Drive. I guess any REST API we connect to from Node.JS will always have a requirement of downloading binary content via XHR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment