Invalid typed array length

Hi,

I've got the following error when doing a SELECT:

buffer.js:98
    const ui8 = new Uint8Array(size);
                ^

RangeError: Invalid typed array length
    at new Uint8Array (native)
    at allocate (buffer.js:98:17)
    at new Buffer (buffer.js:49:12)
    at Parser.append (/home/[...]/node_modules/mysql/lib/protocol/Parser.js:115:14)
    at Parser.write (/home/[...]/node_modules/mysql/lib/protocol/Parser.js:25:8)
    at Protocol.write (/home/[...]/node_modules/mysql/lib/protocol/Protocol.js:39:16)
    at Socket.<anonymous> (/home/[...]/node_modules/mysql/lib/Connection.js:96:28)
    at emitOne (events.js:77:13)
    at Socket.emit (events.js:169:7)
    at Socket.Readable.read (_stream_readable.js:360:10)

I have no idea what it means...

NOTE that it is a very long SELECT (10 milions records) using a cursor (pause / resume).

[dougwilson]

Hi! I'm going to take a look, but I believe using Uint8Array is something new in Node.js. Which version are you using, so I can debug with the same?

Hi,

node v4.2.1
mysql (module) v2.9.0

I can't debug as it is, since the scrypt crash after 10 hours running. But I can try to hightlight the problematic row in MySQL source table. I guess it comes from a specific value in a particuler row since the script treated 20% of the 10 millions rows... I'll keep you posted.

[ecdeveloper]

I'm experiencing exactly the same issue. I'm running a query on a relatively large dataset (about 10M records), and after 3-5 hours it crashes with exactly the same exception. Do you have any updates on your research? I use streaming api. Node version 4.x. mysql module version 2.9.0.

[dougwilson]

If someone can provide me the exact code and instructions on how to reproduce, I can certainly look into it, otherwise I'll need one of you to offer up a fix, since without the ability to reproduce or any good details here, I have no where to even start looking or know if any fix would actually work.

[ecdeveloper]

@dougwilson Ok, no worries. I'm running now some tests, will get back with results soon.

[ecdeveloper]

Ok, so in my case the problem was caused by running out of memory. Due to some hardware problems garbage collector wasn't working properly, and my at some point my script was consuming up to 4GB. I suppose at some certain point it just failed allocating memory for new Uint8Array(size) which caused the exception.