[misc] exact number implementation compatibility correction

CHANGELOG.md

@@ -1,5 +1,35 @@
 # Change Log
 
+## [3.0.0-beta](https://github.com/mariadb-corporation/mariadb-connector-nodejs/tree/3.0.0-beta) (08 Jun 2021)
+[Full Changelog](https://github.com/mariadb-corporation/mariadb-connector-nodejs/compare/2.5.4...3.0.0-beta)
+
+Migrating from 2.x or mysql/mysql2 driver have some breaking changes, see [dedicated part](https://github.com/mariadb-corporation/mariadb-connector-nodejs/blob/maintenance/3.x/documentation/promise-api.md#migrating-from-2x-or-mysqlmysql2-to-3x) documentation.
+
+* [CONJS-153] support Prepared statement with 10.6 new feature metadata skip
+* [CONJS-165] Adding initial message error value on Error object
+* [CONJS-166] Restrict authentication plugin list
+* [CONJS-167] Permit custom logger configuration
+
+New Connection options
+
+|option|description|type|default|
+|---:|---|:---:|:---:| 
+| **insertIdAsNumber** | Whether the query should return last insert id from INSERT/UPDATE command as BigInt or Number. default return BigInt |*boolean* | false |
+| **decimalAsNumber** | Whether the query should return decimal as Number. If enable, this might return approximate values. |*boolean* | false |
+| **bigIntAsNumber** | Whether the query should return BigInt data type as Number. If enable, this might return approximate values. |*boolean* | false |
+| **logger** | Permit custom logger configuration. For more information, see the [`logger` option](#logger) documentation. |*mixed*|
+| **prepareCacheLength** | Define prepare LRU cache length. 0 means no cache |*int*| 256 |
+
+new Connection methods
+* [`connection.prepare(sql) → Promise`](https://github.com/mariadb-corporation/mariadb-connector-nodejs/blob/master/documentation/promise-api.md#connectionpreparesql---promise): Prepares a query.
+* [`connection.execute(sql[, values]) → Promise`](https://github.com/mariadb-corporation/mariadb-connector-nodejs/blob/master/documentation/promise-api.md#connectionexecutesql-values--promise): Prepare and Executes a query.
+
+This methods are compatible with mysql2 with some differences:
+* permit streaming parameters
+* execute use by default a prepared cache that hasn't infinite length.
+* implement mariadb 10.6 skipping metadata when possible for better performance
+* Doesn't have a unprepare methods. 
+
 ## [2.5.4](https://github.com/mariadb-corporation/mariadb-connector-nodejs/tree/2.5.4) (08 Jun 2021)
 [Full Changelog](https://github.com/mariadb-corporation/mariadb-connector-nodejs/compare/2.5.3...2.5.4)
 

README.md

@@ -25,7 +25,7 @@ See [promise documentation](https://github.com/mariadb-corporation/mariadb-conne
 
 [Callback documentation](https://github.com/mariadb-corporation/mariadb-connector-nodejs/blob/master/documentation/callback-api.md) describe the callback wrapper for compatibility with existing drivers.
 
-See [dedicated part](https://github.com/mariadb-corporation/mariadb-connector-nodejs/blob/master/documentation/promise-api.md#migrating-from-2.x-or-mysql/mysql2-to-3.x) for migration from mysql/mysql2 or from 2.x version.
+See [dedicated part](https://github.com/mariadb-corporation/mariadb-connector-nodejs/blob/master/documentation/promise-api.md#migrating-from-2x-or-mysqlmysql2-to-3x) for migration from mysql/mysql2 or from 2.x version.
 
 
 ## Why a New Client?

documentation/connection-options.md

@@ -28,7 +28,8 @@
 | **insertIdAsNumber** | Whether the query should return last insert id from INSERT/UPDATE command as BigInt or Number. default return BigInt |*boolean* | false |
 | **decimalAsNumber** | Whether the query should return decimal as Number. If enable, this might return approximate values. |*boolean* | false |
 | **bigIntAsNumber** | Whether the query should return BigInt data type as Number. If enable, this might return approximate values. |*boolean* | false |
-| **logger** | Configure logger. For more information, see the [`logger` option](#logger) documentation. |*mixed*|
+| **logger** | Permit custom logger configuration. For more information, see the [`logger` option](#logger) documentation. |*mixed*|
+| **prepareCacheLength** | Define prepare LRU cache length. 0 means no cache |*int*| 256 |
 
 ### JSON or String configuration
 
@@ -389,6 +390,8 @@ mariadb.createConnection({
 | **cachingRsaPublicKey** | Indicate path/content to MySQL server caching RSA public key. use requires Node.js v11.6+ |*string* | |
 | **allowPublicKeyRetrieval** | Indicate that if `rsaPublicKey` or `cachingRsaPublicKey` public key are not provided, if client can ask server to send public key. |*boolean* | false |
 | **restrictedAuth** | if set, restrict authentication plugin to secure list. Default provided plugins are mysql_native_password, mysql_clear_password, client_ed25519, dialog, sha256_password and caching_sha2_password |*Array|String* | |
+| **supportBigNumbers** | (deprecated) DECIMAL/BIGINT data type will be returned as number if in safe integer range, as string if not.|*boolean* | false |
+| **bigNumberStrings** | (deprecated) if set with `supportBigNumbers` DECIMAL/BIGINT data type will be returned as string |*boolean* | false |
 
 
 ## F.A.Q.

documentation/promise-api.md

@@ -52,37 +52,23 @@ const mariadb = require('mariadb');
 
 ## Migrating from 2.x or mysql/mysql2 to 3.x
 
-This is a breaking change from 3.0 compare to previous version and mysql/mysql2 drivers.
+Default behaviour for decoding [BIGINT](https://mariadb.com/kb/en/bigint/) / [DECIMAL](https://mariadb.com/kb/en/decimal/) datatype for 2.x version and mysql/mysql2 drivers return a javascript [Number](https://developer.mozilla.org/fr/docs/Web/JavaScript/Reference/Global_Objects/Number) object.
+BIGINT/DECIMAL values might not be in the safe range, resulting in approximate results. 
 
-Integers in JavaScript use IEEE-754 representation. This means that Node.js cannot exactly represent integers in the ±9,007,199,254,740,991 range.  
-However, MariaDB does support larger integers and exact big decimal.
-This means that when the value set on a BIGINT is not in the [safe](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/isSafeInteger) range, javascript Number Object cannot represent exact value.
-Same with DECIMAL type that might not be exact in IEEE 754 floating number.
-
-2.x/mysql/mysql2 have options that try to address those issues. Those are now removed :
-
-* supportBigNumbers: When an integer was not in the safe range, the driver did interprets the value as a [`Long`](https://www.npmjs.com/package/long) object. Now use javascript standard BigInt
-* supportBigInt: Removed, since BigInt support is now required.
-* bigNumberStrings: When an integer was not in the safe range, the driver did interprets the value as a string.
-
-Those options might return a different object type depending on value.
-
-
-Since 3.x version, Driver automatically return data depending on data types :
+Since 3.x version, driver has reliable default, returning:
 * DECIMAL => javascript String
-* BIGINT => javascript BigInt
+* BIGINT => javascript [BigInt](https://mariadb.com/kb/en/bigint/) object
 
-this permit to ensure returning exact values and reliable data type, but might cause some incompatibility.
-Driver provides 3 options to address this issue.
+For compatibility with previous version or mysql/mysql driver, 3 options have been added to return BIGINT/DECIMAL as number, as previous defaults. 
 
 |option|description|type|default| 
 |---:|---|:---:|:---:| 
 | **insertIdAsNumber** | Whether the query should return last insert id from INSERT/UPDATE command as BigInt or Number. default return BigInt |*boolean* | false |
-| **decimalAsNumber** | Whether the query should return decimal as Number. If enable, this might return approximate values. |*boolean* | false |
-| **bigIntAsNumber** | Whether the query should return BigInt data type as Number. If enable, this might return approximate values. |*boolean* | false |
+| **decimalAsNumber** | Whether the query should return decimal as Number. If enabled, this might return approximate values. |*boolean* | false |
+| **bigIntAsNumber** | Whether the query should return BigInt data type as Number. If enabled, this might return approximate values. |*boolean* | false |
+
+Previous options `supportBigNumbers` and `bigNumberStrings` still exist for compatibility, but are now deprecated.   
 
-If wanting compatibility with previous version those values can be set to true / use [`typeCast`](#typeCast) to convert DECIMAL/BIGINT to expect value.
-
 
 ## Recommendation
 
@@ -552,6 +538,7 @@ connection.query('select * from animals')
 * [`rowsAsArray`](#rowsAsArray)
 * [`nestTables`](#nestTables)
 * [`dateStrings`](#dateStrings)
+* [`bigIntAsNumber`](#bigIntAsNumber)
 * [`decimalAsNumber`](#decimalAsNumber)
 
 Those options can be set on the query level, but are usually set at the connection level, and will then apply to all queries. 

lib/cmd/command.js

@@ -88,7 +88,7 @@ class Command extends EventEmitter {
     if (this.reject) {
       if (this.stack) {
         err = Errors.createError(
-          err.message,
+          err.text ? err.text : err.message,
           err.sql,
           err.fatal,
           info,
@@ -126,7 +126,14 @@ class Command extends EventEmitter {
 
     const affectedRows = packet.readUnsignedLength();
     let insertId = packet.readSignedLengthBigInt();
-    if (opts.insertIdAsNumber && insertId != null) insertId = Number(insertId);
+    if (insertId != null && (opts.supportBigNumbers || opts.insertIdAsNumber)) {
+      if (
+        opts.supportBigNumbers &&
+        (opts.bigNumberStrings || !Number.isSafeInteger(Number(insertId)))
+      ) {
+        insertId = insertId.toString();
+      } else insertId = Number(insertId);
+    }
     info.status = packet.readUInt16();
 
     const okPacket = new OkPacket(affectedRows, insertId, packet.readUInt16());

lib/cmd/decoder/binary-decoder.js

@@ -60,14 +60,26 @@ class BinaryDecoder {
         return packet.readDouble();
       case FieldType.BIGINT:
         const val = column.signed() ? packet.readBigInt64() : packet.readBigUInt64();
-        if (opts.bigIntAsNumber && val != null) {
+        if (val != null && (opts.bigIntAsNumber || opts.supportBigNumbers)) {
+          if (
+            opts.supportBigNumbers &&
+            (opts.bigNumberStrings || !Number.isSafeInteger(Number(val)))
+          ) {
+            return val.toString();
+          }
           return Number(val);
         }
         return val;
       case FieldType.DECIMAL:
       case FieldType.NEWDECIMAL:
         const valDec = packet.readDecimalLengthEncoded();
-        if (opts.decimalAsNumber && valDec != null) {
+        if (valDec != null && (opts.decimalAsNumber || opts.supportBigNumbers)) {
+          if (
+            opts.supportBigNumbers &&
+            (opts.bigNumberStrings || !Number.isSafeInteger(Number(valDec)))
+          ) {
+            return valDec.toString();
+          }
           return Number(valDec);
         }
         return valDec;

lib/cmd/decoder/text-decoder.js

@@ -31,14 +31,26 @@ class TextDecoder {
         return packet.readFloatLengthCoded();
       case FieldType.BIGINT:
         const val = packet.readBigIntLengthEncoded();
-        if (opts.bigIntAsNumber && val != null) {
+        if (val != null && (opts.bigIntAsNumber || opts.supportBigNumbers)) {
+          if (
+            opts.supportBigNumbers &&
+            (opts.bigNumberStrings || !Number.isSafeInteger(Number(val)))
+          ) {
+            return val.toString();
+          }
           return Number(val);
         }
         return val;
       case FieldType.DECIMAL:
       case FieldType.NEWDECIMAL:
         const valDec = packet.readDecimalLengthEncoded();
-        if (opts.decimalAsNumber && valDec != null) {
+        if (valDec != null && (opts.decimalAsNumber || opts.supportBigNumbers)) {
+          if (
+            opts.supportBigNumbers &&
+            (opts.bigNumberStrings || !Number.isSafeInteger(Number(valDec)))
+          ) {
+            return valDec.toString();
+          }
           return Number(valDec);
         }
         return valDec;

lib/cmd/parser.js

@@ -132,7 +132,17 @@ class Parser extends Command {
       bigIntAsNumber:
         cmdOpts.bigIntAsNumber != undefined ? cmdOpts.bigIntAsNumber : connOpts.bigIntAsNumber,
       decimalAsNumber:
-        cmdOpts.decimalAsNumber != undefined ? cmdOpts.decimalAsNumber : connOpts.decimalAsNumber
+        cmdOpts.decimalAsNumber != undefined ? cmdOpts.decimalAsNumber : connOpts.decimalAsNumber,
+      insertIdAsNumber:
+        cmdOpts.insertIdAsNumber != undefined
+          ? cmdOpts.insertIdAsNumber
+          : connOpts.insertIdAsNumber,
+      supportBigNumbers:
+        cmdOpts.supportBigNumbers != undefined
+          ? cmdOpts.supportBigNumbers
+          : connOpts.supportBigNumbers,
+      bigNumberStrings:
+        cmdOpts.bigNumberStrings != undefined ? cmdOpts.bigNumberStrings : connOpts.bigNumberStrings
     };
   }
 
@@ -147,7 +157,7 @@ class Parser extends Command {
    * @returns {*}     null or {Resultset.readResponsePacket} in case of multi-result-set
    */
   readOKPacket(packet, out, opts, info) {
-    const okPacket = Command.parseOkPacket(packet, out, opts, info);
+    const okPacket = Command.parseOkPacket(packet, out, this.opts, info);
     this._rows.push(okPacket);
 
     if (info.status & ServerStatus.MORE_RESULTS_EXISTS) {

lib/config/connection-options.js

@@ -145,16 +145,21 @@ class ConnectionOptions {
       }
       this.permitLocalInfile = this.pipelining ? false : opts.permitLocalInfile || false;
     }
-    this.insertIdAsNumber = opts.insertIdAsNumber || false;
-    this.decimalAsNumber = opts.decimalAsNumber || false;
-    this.bigIntAsNumber = opts.bigIntAsNumber || false;
     this.prepareCacheLength = opts.prepareCacheLength === undefined ? 256 : opts.prepareCacheLength;
     this.restrictedAuth = opts.restrictedAuth;
     if (this.restrictedAuth !== undefined && this.restrictedAuth !== null) {
       if (!Array.isArray(this.restrictedAuth)) {
         this.restrictedAuth = this.restrictedAuth.split(',');
       }
     }
+
+    // for compatibility with 2.x version and mysql/mysql2
+    this.bigIntAsNumber = opts.bigIntAsNumber || false;
+    this.insertIdAsNumber = opts.insertIdAsNumber || false;
+    this.decimalAsNumber = opts.decimalAsNumber || false;
+    this.supportBigNumbers = opts.supportBigNumbers || false;
+    this.bigNumberStrings = opts.bigNumberStrings || false;
+
     if (this.maxAllowedPacket && !Number.isInteger(this.maxAllowedPacket)) {
       throw new RangeError(
         "maxAllowedPacket must be an integer. was '" + this.maxAllowedPacket + "'"