64bit Integer

[kawanet]

The current decoder decodes msgpack's int 64 (0xd3) as a JS's Number which means double.

https://www.npmjs.com/package/node-int64

JavaScript Numbers are represented as IEEE 754 double-precision floats. Unfortunately, this means they lose integer precision for values beyond +/- 2^^53. For projects that need to accurately handle 64-bit ints, such as node-thrift, a performant, Number-like class is needed. Int64 is that class.

It looks fit for us as node-int64 has internal raw Buffer object with big endian byte sequence.

[kawanet]

One problem found: node-int64 supports only signed int64. We need both signed and unsigned.

[kawanet]

int64-buffer supports both Uint64 and Int64: https://www.npmjs.com/package/int64-buffer

[kawanet]

0.1.18 published.

[endel]

Hey @kawanet, I'm trying to use msgpack-lite in the browser and it seems that int64-buffer isn't working there for some reason. Here's what happens:

write-type.js:131 Uncaught TypeError: Cannot read property 'isUint64BE' of undefined

I'll dig into it and hopefully post a solution here.

[kawanet]

Do you use browserify, webpack or the pre-built version of dist/msgpack.min.js?

[endel]

I'm using webpack.

[kawanet]

Thanks. I check it with webpack then. It's tested with browserify but not with webpack in fact.

[kawanet]

It seems it works with webpack as well. Try the most recent version of the modules:

npm install --save 'msgpack-lite@^0.1.18'
npm install --save 'int64-buffer@^0.1.5' # this is not needed

If you still fail, please add another issue to tell your webpack config etc.

[agronholm]

Does this mean msgpack-lite can now encode 64 bit integers without monkey patching?

[kawanet]

Does this mean msgpack-lite can now encode 64 bit integers without monkey patching?

Right. It uses int64-buffer 64bit container object.

var Int64BE = require("int64-buffer").Int64BE;
var msgpack = require("msgpack-lite");

// sample big value
var bigstr = "18446744073709551615"; // 2^64-1
var bignum = parseInt(bigstr); // 18446744073709552000 (NG)
var bigbuf = new Uint64BE(bigstr);
console.warn(Uint64BE.isUint64BE(bigbuf)); // true
console.warn(bigbuf + ""); // 18446744073709551615 (OK)

// encoding
var encoded = msgpack.encode(big); // <Buffer cf ff ff ff ff ff ff ff ff>

// decoding to Double (per default)
var num = msgpack.decode(encoded);
console.warn(Uint64BE.isUint64BE(num)); // false
console.warn(num + ""); // 18446744073709552000 (NG)

// decoding to Uint64BE (with int64 codec)
var codec = msgpack.createCodec({int64: true});
var obj = msgpack.decode(encoded, {codec: codec});
console.warn(Uint64BE.isUint64BE(obj)); // true
console.warn(obj + ""); // 18446744073709551615 (OK)

[agronholm]

Ok, thanks. The referenced project needs to serialize 64-bit integers, which is why I was asking.