Resolve truncation with null character on Windows

added fix for unwanted truncation on ColumnText and BindText to preserve embedded nulls.

With minor changes by @brodybits:
- indentation consistent with rest of Windows C++ code
- use const declaration for intermediate results
- update test cases & docs
- mark common version, to be merged into multiple plugin versions
- drop extra question

ref: storesafe#709

CHANGES.md

@@ -1,5 +1,9 @@
 # Changes
 
+##### cordova-sqlite-legacy-core 1.0.3-rc1
+
+- Fix to resolve truncation in case of U+0000 character on Windows by @spacepope (J. Hannes Petersen)
+
 ##### cordova-sqlite-legacy-core 1.0.2
 
 - Fix log in case of transaction waiting for open to finish; doc fixes

README.md

@@ -92,7 +92,7 @@ Use the `location` or `iosDatabaseLocation` option in `sqlitePlugin.openDatabase
   - Pre-populated database (Android/iOS/macOS/Windows)
 - Amazon Fire-OS is dropped due to lack of support by Cordova. Android version should be used to deploy to Fire-OS 5.0(+) devices. For reference: [cordova/cordova-discuss#32 (comment)](https://github.com/cordova/cordova-discuss/issues/32#issuecomment-167021676)
 - Windows version using the performant C++ [doo / SQLite3-WinRT](https://github.com/doo/SQLite3-WinRT) component is in an alpha state:
-  - Issue with UNICODE `\u0000` character (same as `\0`)
+  - ~~Issue with UNICODE `\u0000` character (same as `\0`)~~
   - No background processing
 - FTS3 and FTS4 are tested working OK in this version branch (for all target platforms in this version branch Android/iOS/macOS)
 - R-Tree is *not* tested or supported for Android in this version branch.
@@ -226,7 +226,7 @@ Issues fixed in some newer version branches:
 - The Android version cannot work with more than 100 open db files (due to the threading model used).
 - UNICODE `\u2028` (line separator) and `\u2029` (paragraph separator) characters are not supported and known to be broken on iOS, macOS, and Android platforms due to JSON issues reported in [Cordova bug CB-9435](https://issues.apache.org/jira/browse/CB-9435) and [cordova/cordova-discuss#57](https://github.com/cordova/cordova-discuss/issues/57). There *may* be a similar issue with certain other UNICODE characters in the iOS/macOS version (needs further investigation). This is fixed for iOS in [litehelpers / cordova-sqlite-evplus-ext-legacy-build-free](https://github.com/litehelpers/cordova-sqlite-evplus-ext-legacy-build-free) and [litehelpers / Cordova-sqlite-evplus-legacy-attach-detach-free](https://github.com/litehelpers/Cordova-sqlite-evplus-legacy-attach-detach-free) (GPL or special commercial license terms) as well as [litehelpers / cordova-sqlite-evmax-ext-workers-legacy-build-free](https://github.com/litehelpers/cordova-sqlite-evmax-ext-workers-legacy-build-free) (GPL or premium commercial license terms).
 - The BLOB data type is not fully supported by this version branch. SELECT BLOB in Base64 format is supported by [litehelpers / cordova-sqlite-ext](https://github.com/litehelpers/cordova-sqlite-ext) (permissive license terms) and [litehelpers / Cordova-sqlite-evcore-extbuild-free](https://github.com/litehelpers/Cordova-sqlite-evcore-extbuild-free) (GPL or commercial license terms).
-- Truncation in case of UNICODE `\u0000` (same as `\0`) character on Android (default Android-sqlite-connector database implementation) and Windows.
+- Truncation in case of UNICODE `\u0000` (same as `\0`) character on Android (default Android-sqlite-connector database implementation) ~~and Windows~~.
 - Case-insensitive matching and other string manipulations on Unicode characters, which is provided by optional ICU integration in the sqlite source and working with recent versions of Android, is not supported for any target platforms.
 - iOS/macOS platform version uses a thread pool but with only one thread working at a time due to "synchronized" database access
 - Some large query results may be slow, also due to the JSON implementation.

package.json

@@ -1,6 +1,6 @@
 {
   "name": "cordova-sqlite-legacy-core",
-  "version": "1.0.2",
+  "version": "1.0.3-rc1",
   "description": "Native interface to SQLite for PhoneGap/Cordova (legacy core version branch)",
   "cordova": {
     "id": "cordova-sqlite-legacy-core",

plugin.xml

@@ -2,7 +2,7 @@
 <plugin xmlns="http://www.phonegap.com/ns/plugins/1.0"
     xmlns:android="http://schemas.android.com/apk/res/android"
     id="cordova-sqlite-legacy-core"
-    version="1.0.2">
+    version="1.0.3-rc1">
 
     <name>Cordova sqlite storage plugin - legacy core version branch</name>
 

spec/www/spec/db-tx-string-test.js

@@ -77,7 +77,7 @@ var mytests = function() {
 
         it(suiteName + ' string encoding test with UNICODE \\u0000', function (done) {
           if (isWP8) pending('BROKEN on WP(8)'); // [BUG #202] UNICODE characters not working with WP(8)
-          if (isWindows) pending('BROKEN on Windows'); // TBD (truncates on Windows)
+          // if (isWindows) pending('BROKEN on Windows'); // (RESOLVED)
           // XXX BROKEN on Android-sqlite-connector in this version branch:
           if (!isWebSql && !isWindows && isAndroid && !isImpl2) pending('BROKEN on Android-sqlite-connector implementation)');
 

spec/www/spec/db-tx-value-bindings-test.js

@@ -253,7 +253,7 @@ var mytests = function() {
 
         test_it(suiteName + ' stores [Unicode] string with \\u0000 correctly', function () {
           if (isWP8) pending('BROKEN on WP(8)'); // [BUG #202] UNICODE characters not working with WP(8)
-          if (isWindows) pending('BROKEN on Windows'); // TBD (truncates on Windows)
+          // if (isWindows) pending('BROKEN on Windows'); // (RESOLVED)
           // XXX BROKEN on Android-sqlite-connector in this version branch:
           if (!isWebSql && !isWindows && isAndroid && !isImpl2) pending('BROKEN on Android-sqlite-connector implementation)');
 
@@ -321,9 +321,8 @@ var mytests = function() {
         }
 
         test_it(suiteName + ' returns [Unicode] string with \\u0000 correctly', function () {
-          if (isWindows) pending('BROKEN on Windows'); // XXX
           if (isWP8) pending('BROKEN on WP(8)'); // [BUG #202] UNICODE characters not working with WP(8)
-          if (isWindows) pending('BROKEN on Windows'); // XXX
+          // if (isWindows) pending('BROKEN on Windows'); // (RESOLVED)
           if (isWebSql && isAndroid) pending('SKIP on Android Web SQL'); // XXX TBD INCONSISTENT RESULTS Android 4 vs 5
 
           stop();

src/windows/SQLite3-Win-RT/SQLite3/Statement.cpp

@@ -45,7 +45,13 @@ namespace SQLite3
 
   Platform::String^ Statement::ColumnText(int index)
   {
-    return ref new Platform::String(static_cast<const wchar_t*>(sqlite3_column_text16(statement, index)));
+    // To preserve embedded nulls within text data and fix truncation on first null char,
+    // create platform string with corresponding length:
+    // (length is number of bytes in text column divided by size of one wide character.
+    //  see https://bugs.chromium.org/p/chromium/issues/detail?id=422690
+    //  and https://www.sqlite.org/capi3ref.html#sqlite3_column_blob)
+    const unsigned int length = sqlite3_column_bytes16(statement, index) / sizeof(wchar_t);
+    return ref new Platform::String(static_cast<const wchar_t*>(sqlite3_column_text16(statement, index)), length);
   }
 
   int Statement::ColumnInt(int index)
@@ -65,7 +71,13 @@ namespace SQLite3
 
   int Statement::BindText(int index, Platform::String^ val)
   {
-    return sqlite3_bind_text16(statement, index, val->Data(), -1, SQLITE_TRANSIENT);
+    // To preserve embedded nulls within text data and fix truncation on first null char,
+    // pass number of text bytes to sqlite3_bind_text16() instead of -1 as third parameter:
+    // (number of bytes are calculated by multiplying string length with the size of one wide character.
+    //  see https://bugs.chromium.org/p/chromium/issues/detail?id=422690
+    //  and https://www.sqlite.org/capi3ref.html#sqlite3_bind_blob)
+    const int length = val->Length() * sizeof(wchar_t);
+    return sqlite3_bind_text16(statement, index, val->Begin(), length, SQLITE_TRANSIENT);
   }
 
   int Statement::BindInt(int index, int val)