\u0000 in generated keys

[brodycj]

Issue

I have a major client who has sent me a test program that demonstrates a major issue with \u0000 characters and the cordova-sqlite adapter.

The following query returns data when using the cordova-sqlite adapter with cordova-sqlcipher-adapter plugin but not when using cordova-sqlite-storage or a new, customized plugin under development:

let r = await app.db.query('responsesByFormId', { key: 'user-profile', include_docs: true })

Here is a trace of a couple of select statements that I got from the new, customized plugin:

[["BEGIN",[]],["SELECT json FROM 'document-store' WHERE id = ?",["54user-profile\u00004up1\u0000\u0000"]],["SELECT json FROM 'document-store' WHERE id = ?",["54foo\u00004up2\u0000\u0000"]]]

Dealing with \u0000 characters has proven to be a bit problematic across the various implementations and platforms over the years.

From an article in 2014 I had thought that supporting \u0000 characters would not be needed: https://pouchdb.com/2014/10/26/10-things-i-learned-from-reading-and-writing-the-pouchdb-source.html

and this documentation shows how to replace \u0000 characters when using pouchdb-collate: https://github.com/pouchdb/pouchdb/tree/master/packages/node_modules/pouchdb-collate#toindexablestringobj

My guess is that the \u0000 characters come from PouchDB using the collate module internally.

From a quick search I did find a few issues that people have reported and then closed:

• pouchdb-adapter-cordova-sqlite on cordova-sqlite-storage not working with index data on Windows! #6685 - issue with cordova-sqlite-storage on Windows
• \u0000 in document IDs #6881 - \u0000 in document IDs, looks like the OP just closed it and moved on
• ReactNative SQLite error: UNIQUE constraint failed: attach-store.digest #6037 - issue with react-native-sqlite-storage, which led to react-native-sqlite-2, seems to have a statement in the end: "There's also a little patch for avoiding Android's problem truncating string including \u0000."

Any workaround or long-term solution that would stop using the \u0000 characters would be extremely helpful.

Info

• Environment: Hybrid - Cordova
• Platform: Android
• Adapter: cordova-sqlite
• Server: CouchDB, but just using PouchDB locally with cordova-sqlite adapter for the test query above

Reproduce

see description

[brodycj]

I think it is in these places where the \u0000 characters are inserted:

• pouchdb/packages/node_modules/pouchdb-collate/src/index.js

  Lines 116 to 119 in fafcd70

  	function toIndexableString(key) {
  	var zero = '\u0000';
  	key = normalizeKey(key);
  	return collationIndex(key) + SEP + indexify(key) + zero;

• pouchdb/packages/node_modules/pouchdb-collate/src/index.js

  Lines 90 to 106 in fafcd70

  	case 'object':
  	var isArray = Array.isArray(key);
  	var arr = isArray ? key : Object.keys(key);
  	var i = -1;
  	var len = arr.length;
  	var result = '';
  	if (isArray) {
  	while (++i < len) {
  	result += toIndexableString(arr[i]);
  	}
  	} else {
  	while (++i < len) {
  	var objKey = arr[i];
  	result += toIndexableString(objKey) +
  	toIndexableString(key[objKey]);
  	}
  	}

  which in turn calls the toIndexableString function linked above

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[brodycj]

I think this issue is not resolved.