RangeError [ERR_INVALID_OPT_VALUE]: The value "-2048893" is invalid for option "size"

[marcj]

I have 10k queries like that

WITH _tmp as (SELECT `id` FROM user WHERE `id` = 1)
UPDATE `deepkit` as _target, _tmp as _b
SET _target.`priority` = 6 WHERE _target.`id` = _b.`id`;
WITH _tmp as (SELECT `id` FROM user WHERE `id` = 2)
UPDATE `deepkit` as _target, _tmp as _b
SET _target.`priority` = 6 WHERE _target.`id` = _b.`id`;
WITH _tmp as (SELECT `id` FROM user WHERE `id` = 3)
UPDATE `deepkit` as _target, _tmp as _b
SET _target.`priority` = 6 WHERE _target.`id` = _b.`id`;
....

in a benchmark to test system limits. 10k queries are executed in one batch call with multipleStatements: true. Obviously this hit some limits (probably overflow issues) since I get this error:

(node:77669) UnhandledPromiseRejectionWarning: RangeError [ERR_INVALID_OPT_VALUE]: The value "-2048893" is invalid for option "size"
    at Function.allocUnsafe (buffer.js:382:3)
    at ReWritePacket.flushMark (/Users/marc/bude/deepkit-framework/packages/sql/node_modules/mariadb/lib/io/rewrite-packet.js:431:26)
    at ReWritePacket.flush (/Users/marc/bude/deepkit-framework/packages/sql/node_modules/mariadb/lib/io/rewrite-packet.js:396:12)
    at ReWritePacket.growBuffer (/Users/marc/bude/deepkit-framework/packages/sql/node_modules/mariadb/lib/io/rewrite-packet.js:70:12)
    at ReWritePacket.writeDefaultBufferString (/Users/marc/bude/deepkit-framework/packages/sql/node_modules/mariadb/lib/io/rewrite-packet.js:224:24)
    at ReWritePacket.flushMark (/Users/marc/bude/deepkit-framework/packages/sql/node_modules/mariadb/lib/io/rewrite-packet.js:445:10)
    at ReWritePacket.mark (/Users/marc/bude/deepkit-framework/packages/sql/node_modules/mariadb/lib/io/rewrite-packet.js:385:16)
    at BatchRewrite.sendQueries (/Users/marc/bude/deepkit-framework/packages/sql/node_modules/mariadb/lib/cmd/batch-rewrite.js:122:19)
    at BatchRewrite.start (/Users/marc/bude/deepkit-framework/packages/sql/node_modules/mariadb/lib/cmd/batch-rewrite.js:64:10)
    at _addCommandEnablePipeline (/Users/marc/bude/deepkit-framework/packages/sql/node_modules/mariadb/lib/connection.js:1200:11)
(Use `node --trace-warnings ...` to show where the warning was created)

The number behind "value" changes depending on how many queries I try in one batch.

When I limit the SQL to 5120 queries it works, when I do 5121 it breaks. Interestingly when I do first 5000, and then in a next batch call another 5000 it breaks in the second call with the same error. With 5121 I get

The value "-1048697" is invalid for option "size"

and with 10k I get

The value "-2048893" is invalid for option "size"

Running 10k queries of these work fine:

UPDATE `deepkit` SET `priority` = 6 WHERE `id` = 1';
UPDATE `deepkit` SET `priority` = 6 WHERE `id` = 2';

Versions:

• macOS 10.15.6
• Node v14.8.0
• mariadb (this package) 2.4.2
• MySQL 8

[marcj]

I debugged a bit and found that growBuffer (in io/rewrite-packet.js) is called 3 times.

growBuffer 248893
growBuffer 2098892
growBuffer 2098892

well below the 16MB buffer limit. However at the end it goes into

    if (this.pos !== this.markPos) {
      //remove "," character
      afterMark = Buffer.allocUnsafe(this.pos - this.markPos - 1);
      this.buf.copy(afterMark, 0, this.markPos + 1, this.pos);
    }

where this.pos is 5 and this.markPos is 2098897, which results in a big negative number and thus the crash.

[marcj]

When I use query instead of batch everything works fine.

[rusher]

just to be sure, batch is declared this way :
conn.batch('WITH _tmp as (SELECT idFROM user WHEREid= ?) UPDATEdeepkit as _target, _tmp as _b SET _target.priority = 6 WHERE _target.id = _b.id', [[1], [2], [3], ..., [10000]]); ?

I fail to see how multipleStatements is involved here.

[marcj]

I fail to see how multipleStatements is involved here.

It isn't. I misunderstood the API of batch. I thought it's about executing multiple queries at once (where it doesn't matter if you apply prepared statement values), obviously batch is about executing one query with multiple prepare statement value groups instead.

Yet I don't think throwing a cryptic error message is the right thing, especially since it looks more like an algo error and could be captured.

[rusher]

error reproduced and corrected.
This will be corrected in 2.4.3

[marcj]

Thanks!