-
Notifications
You must be signed in to change notification settings - Fork 66
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for Buffers #45
Comments
We'd probably have to do performance tests on those two methods, though I'd predict, that the first one should be a lot faster than the second one. It also would allow for a simple way to check for >127 values. Array.prototype.map.call(buf, function(byte) {
return byte > 127 ? '%' + byte.toString(16) : String.fromCharCode(byte);
}).join(''); However, running this code shows, that we'd have to do a more sophisticated check. var arr = [];
for(var i = 0; i < 128; i++) arr.push(String.fromCharCode(i));
console.log(arr); We should only allow bytes from these ranges to be not encoded with
This leaves us with this snippet: function padLeft(str, pad) {
return pad.substring(0, pad.length - str.length) + str;
}
function encodeByte(byte) {
return (byte >= 49 && byte <= 57)
|| (byte >= 65 && byte <= 90)
|| (byte >= 97 && byte <= 122)
? String.fromCharCode(byte) : '%' + padLeft(byte.toString(16), "00");
}
function encodeBuffer(buf) {
return Array.prototype.map.call(buf, encodeByte).join('');
} This will also be one way only, meaning that we'll only stringify Buffers. Trying to detect binary querystrings can go horribly wrong and would make the API ambiguous. |
personally I would just .toString() it, I don't think shipping binary in a querystring is all that useful |
Seems like it isn't this simple... encodeURIComponent(Buffer([0xEF]));
// => '%EF%BF%BD' |
@silvinci A Buffer can't have >127 values. At this point you might as well call escape(String.fromCharCode.apply(null, buf)) |
@seishun wat. We wanna make sure, that binary data is encoded, when neccessary ( Anyways: Why would you want to send binary data? I can't come up with a real use case for this, however |
Okay, had a brain fart. My snippet works anyway though. Some APIs expect binary data, for example, logging in with Steam. See https://github.com/seishun/node-steam/blob/master/lib/handlers/user.js#L84. |
On a second thought, we should leave the optional encoding out, so a Buffer will always be completely encoded with %. This would make it look more consistent and should be easier to parse for the receiver. Your However, as |
Okay, we have a winner. 1,000,000 rounds took me under 20 s. var buf = new Buffer(256);
for(var b = 0; b < buf.length; b++) buf[b] = b;
console.log(buf.toString('hex').replace(/../g, '%$&').toUpperCase()); // %00%01 ... %FE%FF
|
@seishun I would base64 or hex-encode them personally, closing for now I don't think this is necessary |
Are you suggesting me to change the API? I have no control over that. Otherwise your module would be unnecessary because everyone would use JSON. |
You can always use your own fork or manually encode the Buffer. The code's free to use. :) |
@seishun true, fair enough, just seems like a pretty extreme edge-case to store binary in a querystring |
In case someone finds this issue and needs to implement it: Do not use |
Sometimes you need to send urlencoded binary data. However, the following:
qs.stringify({a: new Buffer([0x00, 0x30, 0x7f, 0x80])})
does not produce meaningful output. One would expect it to return something like:
'a=%000%7F%80'
It should be fairly easy to implement. I would submit a pull request, but there are multiple ways to urlencode a Buffer. The simplest one is probably the following:
escape(buf.toString('binary'))
Unfortunately, the 'binary' encoding is deprecated and is planned to be removed according to the docs, so you might not want to rely on it. The other two ways I know are a bit ugly:
Array.prototype.map.call(buf, function(byte) { return '%' + byte.toString(16); }).join('')
and
buf.toString('hex').match(/../g).map(function(byte) { return '%' + byte; }).join('')
Both produce the same output as the 'escape' method, except they percent-encode every byte, not only those with the value higher than 127.
What do you think? Perhaps someone knows a better method?
The text was updated successfully, but these errors were encountered: