feat: new loader option for UUID primary keys in document tables (#614)

* new loader option for pk type uuid (doc table)

* query file create extension requires superuser account

* UUID doc PK option PR requested changes

* UUID doc PK option PR requested changes 2

* simplify getDefaultSQLforUUID() uuidV arg

* UUID doc PK option PR requested changes 3

* Document doc PK UUID feature

* docs: fix link to documents page

* docs: one more link for the road

docs/connecting.md

@@ -98,11 +98,39 @@ If you don't want to load _every_ table, view, or function your user can access,
 
 Blacklists and whitelists may be comma-separated strings or an array of strings (which will be separated by commas). Either type can use SQL `LIKE` (`_` and `%` placeholders) wildcarding. Consistent with PostgreSQL naming, they are case-sensitive.
 
+### Document table uuid primary key data type
+
+Please see [Primary key default data type](documents#primary-key-default-data-type) for more information about what a UUID is and why you may want to use it. However, to use the loader `documentPkType` option, the connected database will need the extension 'uuid-ossp' installed.
+
+If this extension is not installed, follow these steps, using the 'postgres' account as 'superuser' privileges are required.
+
+```
+$ psql -d postgres -h localhost -p 5432 -U postgres
+psql (10.4 (Ubuntu 10.4-2.pgdg16.04+1))
+Type "help" for help.
+
+postgres=# \c YOUR_DB_NAME;
+You are now connected to database "YOUR_DB_NAME" as user "postgres".
+YOUR_DB_NAME=# CREATE EXTENSION "uuid-ossp";
+CREATE EXTENSION
+YOUR_DB_NAME=# \q
+```
+
+The default/recommended loader `uuidVersion` option is set to 'v4' and will suit most use cases. However, 'v1' and 'v1mc' offer higher performance but are considered less secure, as they may reveal the network card MAC address. Before changing this setting make sure you carry out your own research.
+
+More information: [PostgreSQL extension 'uuid-ossp'](https://www.postgresql.org/docs/10/static/uuid-ossp.html)
+
 ```javascript
 massive(connectionInfo, {
   // change the scripts directory
   scripts: './myscripts',
 
+  // override default 'serial' data type, used for new document tables id/primary key, i.e. 'serial' or 'uuid'
+  documentPkType: 'serial',
+
+  // applies if documentPkType is set to 'uuid'. Override default 'v4' UUID variation, i.e. 'v1', 'v1mc', 'v3', 'v4' or 'v5'
+  uuidVersion: 'v4',
+
   // only load tables, views, and functions in these schemas
   allowedSchemas: ['public', 'auth'],   
 

docs/documents.md

@@ -25,12 +25,24 @@ The JSONB type is a great solution to this problem, and Massive takes care of th
 
 Document tables exist for the sole purpose of storing JSONB data. Query them through Massive's API, and you get a JSON document which you can modify and persist, all seamlessly. You don't even need to create them ahead of time until you know you need them.
 
-Document tables may be extended with new columns and foreign keys. The `id` type can be changed as well (so long as a default is set such as `uuid_generate_v1mc()` for UUID types) without impeding usage of document table functions. Just don't _remove_ any columns or change their names, since Massive depends on those.
-
 Standard table functions still work on document tables, and can be quite useful especially for extended document tables! Fields in the document can be searched with regular `find` and criteria object fields using JSON traversal to look for `body.myField.anArray[1].aField`.
 
 `findDoc` **is still preferred** to JSON queries if at all possible since it uses the `@>` "contains" operator to leverage indexing on the document body to improve performance.
 
+### Primary key default data type
+
+The default primary key data type used for all tables including document tables is 'serial', which by default begins at 1 for the first record created and for every new record increments by 1. However, it is possible to change the primary key data type used for new document tables to 'uuid' (Universal Unique Identifier).
+
+A UUID is a string of 32 hexadecimal digits, in five character groups, separated by hyphens and are generally used in a concurrent or distributed environment.
+
+It is also possible to change the default mechanism used to generate UUIDs, between several standard algorithms, from `uuid_generate_v4()` to `uuid_generate_v1()`, `uuid_generate_v1mc()`, `uuid_generate_v3()` or `uuid_generate_v5()`.
+
+Note: The connected database requires the extension 'uuid-ossp', in order to support the 'uuid' primary key data type.
+
+Please see [loader configuration and filtering](connecting#loader-configuration-and-filtering) for help setting these options.
+
+Document tables may be extended with new columns and foreign keys. The `id` type can be changed as well (so long as a default is set such as `uuid_generate_v1mc()` or `uuid_generate_v4()` etc. for UUID types) without impeding usage of document table functions. Just don't _remove_ any columns or change their names, since Massive depends on those.
+
 ### db.saveDoc
 
 The connected database instance has a `saveDoc` function. Passed a collection name (which can include a non-public schema) and a JavaScript object, this will create the table if it doesn't already exist and write the object to it.

lib/database.js

@@ -27,6 +27,10 @@ const introspectors = {}; // store introspection scripts across reloads
  * table, view, and function visible to the connection's user.
  * @param {String} loader.scripts - Override the Massive script file location
  * (default ./db).
+ * @param {String} loader.documentPkType - Override default data type (serial),
+ *  set for DocumentTable primary key 'id', e.g. 'uuid'
+ * @param {String} loader.uuidVersion - If documentPkType is set to 'uuid', set which UUID version to use,
+ *  e.g. 'uuid_generate_v1', 'uuid_generate_v4', etc. Default is 'uuid_generate_v4'
  * @param {Array|String} loader.allowedSchemas - Table/view schema whitelist.
  * @param {Array|String} loader.whitelist - Table/view name whitelist.
  * @param {Array|String} loader.blacklist - Table/view name blacklist.
@@ -432,6 +436,26 @@ Database.prototype.withTransaction = function (callback, options = {}) {
   });
 };
 
+/**
+ * Create an extension.
+ *
+ * @param {String} extensionName - A valid extension name. Example 'uuid-ossp'
+ * @return {Promise} A promise which resolves when the extension has been created.
+ */
+Database.prototype.createExtension = function (extensionName) {
+  return this.query(`CREATE EXTENSION IF NOT EXISTS "${extensionName}";`);
+};
+
+/**
+ * Drop an extension.
+ *
+ * @param {String} extensionName - A valid extension name. Example 'uuid-ossp'
+ * @return {Promise} A promise which resolves when the extension has been droped.
+ */
+Database.prototype.dropExtension = function (extensionName) {
+  return this.query(`DROP EXTENSION IF EXISTS "${extensionName}";`);
+};
+
 /**
  * Save a document.
  *
@@ -454,6 +478,43 @@ Database.prototype.saveDoc = function (collection, doc) {
   return this.createDocumentTable(collection).then(() => this.saveDoc(collection, doc));
 };
 
+/**
+ * Generate SQL to define UUID primary key default
+ *
+ * @param {String} pkType - Primary key data type, 'serial' or 'uuid' expected
+ * @param {String} uuidV - The UUID variant/version (default = 'v4'), typically 'v1', 'v1mc' or 'v4'
+ * @return {String} SQL to define primary key default.
+ */
+function getDefaultSQLforUUID (pkType, uuidV) {
+  if (pkType !== 'uuid') {
+    return '';
+  }
+
+  let sqlDefault = '';
+
+  switch (uuidV) {
+    case 'v1':
+      sqlDefault = 'DEFAULT uuid_generate_v1()';
+      break;
+    case 'v1mc':
+      sqlDefault = 'DEFAULT uuid_generate_v1mc()';
+      break;
+    case 'v3':
+      sqlDefault = 'DEFAULT uuid_generate_v3()';
+      break;
+    case 'v4':
+      sqlDefault = 'DEFAULT uuid_generate_v4()';
+      break;
+    case 'v5':
+      sqlDefault = 'DEFAULT uuid_generate_v5()';
+      break;
+    default:
+      sqlDefault = 'DEFAULT uuid_generate_v4()';
+  }
+
+  return sqlDefault;
+}
+
 /**
  * Create a new document table and attach it to the Database for usage.
  *
@@ -466,11 +527,16 @@ Database.prototype.createDocumentTable = function (location) {
   const tableName = splits.pop();
   const schemaName = splits.pop() || this.currentSchema;
   const indexName = tableName.replace('.', '_');
+  const documentPkType = this.loader.documentPkType || 'serial';
+  const uuidVersion = this.loader.uuidVersion;
+  const sqlDefault = getDefaultSQLforUUID(documentPkType, uuidVersion);
 
   return this.query(this.loader.queryFiles['document-table.sql'], {
     schema: schemaName,
     table: tableName,
-    index: indexName
+    index: indexName,
+    pkType: documentPkType,
+    pkDefault: sqlDefault
   }).then(() =>
     this.attach(new Writable({
       db: this,

lib/scripts/document-table.sql

@@ -1,5 +1,5 @@
 CREATE TABLE ${schema~}.${table~}(
-  id serial PRIMARY KEY,
+  id ${pkType~} PRIMARY KEY ${pkDefault^},
   body jsonb NOT NULL,
   search tsvector,
   created_at timestamptz DEFAULT now(),

test/database/createDocumentTable.js

@@ -28,6 +28,42 @@ describe('createDocumentTable', function () {
     });
   });
 
+  describe('(UUID config) without schema', function () {
+    before(function () {
+      return db.createExtension('uuid-ossp');
+    });
+
+    after(function () {
+      global.loader.documentPkType = 'serial';
+      global.loader.uuidVersion = '';
+
+      return db.dropTable(tableName, {cascade: true})
+        .then(function () {
+          return db.dropExtension('uuid-ossp');
+        });
+    });
+
+    it('creates a table with UUID primary key on public schema', function () {
+      global.loader.documentPkType = 'uuid';
+      global.loader.uuidVersion = 'v1mc';
+
+      return db.createDocumentTable(tableName).then(() => {
+        assert.isOk(db[tableName]);
+        assert.instanceOf(db[tableName], Writable);
+      });
+    });
+
+    it('saves new document to existing collection (table) without a UUID primary key', function () {
+      return db.saveDoc(tableName, {
+        title: 'Create UUID',
+        created_at: '2015-03-04T09:43:41.643Z'
+      }).then(doc => {
+        assert.equal(doc.title, 'Create UUID');
+        assert.match(doc.id, /^[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i, 'valid uuid format');
+      });
+    });
+  });
+
   describe('with schema', function () {
     const schemaTableName = `${schema}.${tableName}`;
 
@@ -46,4 +82,44 @@ describe('createDocumentTable', function () {
       });
     });
   });
+
+  describe('(UUID config) with schema', function () {
+    const schemaTableName = `${schema}.${tableName}`;
+
+    before(function () {
+      return Promise.all([
+        db.createSchema(schema), db.createExtension('uuid-ossp')
+      ]);
+    });
+
+    after(function () {
+      global.loader.documentPkType = 'serial';
+      global.loader.uuidVersion = '';
+
+      return db.dropSchema(schema, {cascade: true})
+        .then(function () {
+          return db.dropExtension('uuid-ossp');
+        });
+    });
+
+    it('creates a table with UUID primary key on the specified schema', function () {
+      global.loader.documentPkType = 'uuid';
+      global.loader.uuidVersion = 'v1mc';
+
+      return db.createDocumentTable(schemaTableName).then(() => {
+        assert.isOk(db[schema][tableName]);
+        assert.instanceOf(db[schema][tableName], Writable);
+      });
+    });
+
+    it('saves new document to existing collection (table) without a UUID primary key', function () {
+      return db.saveDoc(schemaTableName, {
+        title: 'Create UUID',
+        created_at: '2015-03-04T09:43:41.643Z'
+      }).then(doc => {
+        assert.equal(doc.title, 'Create UUID');
+        assert.match(doc.id, /^[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i, 'valid uuid format');
+      });
+    });
+  });
 });