Merge branch 'master' into token-db-updates

.travis.yml

@@ -1,6 +1,6 @@
 language: node_js
 node_js:
-  - "4"
+  - "6"
 
 env:
   global:

README.md

@@ -15,14 +15,7 @@ Refocus is a platform for visualizing the health and status of systems and/or se
 
 
 - [Features](#features)
-- [Installation](#installation)
-  - [Updates](#updates)
-- [Development](#development)
-  - [Package Scripts](#package-scripts)
-- [Usage](#usage)
-- [How redis is used](#how-redis-is-used)
-- [Running on Heroku](#running-on-heroku)
-  - [Troubleshooting a Heroku deployment](#troubleshooting-a-heroku-deployment)
+- [Quickstart](#Quickstart)
 - [Perspective Debugging](#perspective-debugging)
 - [API Documentation](#api-documentation)
 - [Securing Refocus](#securing-refocus)
@@ -40,73 +33,9 @@ Refocus is a platform for visualizing the health and status of systems and/or se
 - Self-service
 - Easy deployment to Heroku
 
-## Installation
-1. Install [Node.js](https://nodejs.org/).
-1. Install [PostgreSQL](http://www.enterprisedb.com/products-services-training/pgdownload). Be sure to read the "PostgreSQL One Click Installer README" and follow the instructions there to adjust your shared memory as needed.
-1. Install [Redis](http://redis.io/download).
-1. Clone this git repository.
-1. Run `cd refocus`.
-1. Create a new branch: ```git checkout -b <your branch name> master```
-1. Run `npm install`. This downloads and installs project dependencies and executes the post-install steps. Note: don't run this with sudo! You may get some weird errors later.
-1. Install lunchy (`brew install Caskroom/cask/lunchy`). This will help you start redis.
-1. Run `lunchy start redis` to start redis.
-1. Run `npm start` to start your Node.js server at http://localhost:3000.
-1. If you want to unload some of the processing to a background process run `npm run start-clock`. If you intend to deploy this on heroku and have heroku toolbelt installed run `heroku local` to start both the web and the background process.
-
-### Updates
-Whenever you pull down a new version of Refocus from the git repository:
-
-1. Run `npm update` to make sure you have all the latest dependencies.
-1. Run `npm start` to start your Node.js server at http://localhost:3000.
-
-## Development
-- Run `npm run build` and modify the webpack.config.js to take advantage of react hot module reload (react-hmr), for faster front-end development.
-- At times the generated pages don't show due to this error in the browser console: `locals[0] does not appear to be a 'module' object with Hot Module replacement API enabled`. This can happen when the NODE_ENV is blank. To fix the issue, set the NODE_ENV to a non-empty value, then run the build again.
-- If you want any of the packages to send output to stdout, you can start your server with `DEBUG=* node .` or you can spell out which packages you want to show debug output, e.g. `DEBUG=express*,swagger* node .`.
-- Use [nodemon](http://nodemon.io/) to monitor for any changes in your source and automatically restart your server.
-- Use Node.js [Debugger](https://nodejs.org/api/debugger.html).
-- If you are making changes to the code, check for adherence to style guidelines by running `gulp style`.
-- If you are making any changes to the DB schema, create a migration using `node_modules/.bin/sequelize migration:create --name example-name`
-
-### Package Scripts
-Execute any of the scripts in the `scripts` section of [`./package.json`](./package.json) by calling `npm run [SCRIPTNAME]`, e.g. `npm run test` or `npm run lint` or `npm run start`.
-
-## Usage
-
-> TODO
-
-## How redis is used
-On node server startup, two redis clients are instantiated:
-The ```publisher``` is called by the sequelize ORM when a subject or sample is inserted/updated/deleted. It publishes a serialized object keyed by the record type (i.e. ```subject``` or ```sample```) and containing the object which was inserted/updated/deleted.
-The ```subscriber``` parses the message string into an object then uses socket.io to broadcast the changes out to any connected browser clients.
-After installing the server, you can run ```redis-cli``` to issue commands to redis server. Command to show active channels is: ```PUBSUB CHANNELS *```
-
-## Running on Heroku
-- Setup Heroku account and toolbelt (follow this guide https://devcenter.heroku.com/articles/getting-started-with-nodejs#introduction)
-- Run ```heroku create``` to create Heroku project
-- If you are not in a private space run ```PGSSLMODE=require``` to make all db data go over ssl
-- Run ```heroku config:set NODE_ENV=test```
-- Run ```heroku addons:create heroku-postgresql:hobby-dev``` to create a dev db
-- Run ```git push heroku <your branch>:master``` which will push to Heroku and start up a dyno.
-- Run ```heroku open``` and view the app running in Heroku
-- Run ```heroku run bash``` then run ```mocha``` to execute the test suite
-- If you are running the app in more than one dyno, you will need to force the client to communicate with the server only using websockets. To do so, set the config variable SOCKETIO_TRANSPORT_PROTOCOL to websocket or run ```heroku config:set SOCKETIO_TRANSPORT_PROTOCOL=websocket```
-If you are running on Heroku and you want to use Google Analytics, store your tracking id in a Heroku config variable called `GOOGLE_ANALYTICS_ID`.
-
-### Troubleshooting a Heroku deployment
-- Log errors in suspected areas. Use the logging in the error handler
-- For errors 'Relation __ does not exist', the db is not set up properly. Try resetting the database. Run ```heroku run bash``` to enter shell script mode, then run ```gulp resetdb```, to reset the db
-- Run ```heroku restart```, to restart the app
-- Run ```heroku logs --tail``` to see the heroku logs, as they update
-
-## Setup Production Environment on Localhost
-If not already setup, follow Installation instructions to setup Refocus. Execute the following commands to setup production environment and corresponding config variables:
-
-```
-export NODE_ENV=production
-export DATABASE_URL='postgres://postgres:postgres@localhost:5432/focusdb'
-npm start
-```
+## Quickstart
+
+See the [Quickstart](https://salesforce.github.io/refocus/docs/01-quickstart.html) guide to get going with Refocus!
 
 ## Securing Refocus
 1. After installation, log in (UI or API) as `admin@refocus.admin` with password `password` and change the password for that account.

db/createOrDropDb.js

@@ -9,59 +9,58 @@
 /**
  * ./db/createOrDropDb.js
  *
- * Creates or deletes the configured db, depending on
- * the option after the filename.
+ * Call this script from the command line to create or drop the db. Requires
+ * a command line argument to indicate whether to create (--init, -i) or drop
+ * (--drop, -d) the database.
  */
-
 const commandLineArgs = require('command-line-args');
 const pgtools = require('pgtools');
-const conf = require('../config');
-const env = conf.environment[conf.nodeEnv];
-const DB_URL = env.dbUrl;
-
-// example DB_URL: 'postgres://username:password@host:port/name'
-const dbConfig = {
-  name: DB_URL.split('/').pop(),
-  user: DB_URL.split(':')[1].slice(2),
-  password: DB_URL.split(':')[2].split('@')[0],
-  host: DB_URL.split('@').pop().split(':')[0],
-  port: DB_URL.split(':').pop().split('/')[0],
-};
-
-/**
- * Creates or drops db
- * @param {boolean} bool - If true, creates the db with dbConfig properties.
- * Bool defaults to false, to delete the configured db.
- * @param {Object} pgtool - Npm module for creating and dropping db.
- * @parm {Object} dbConfigObj - Js Object with db config, as input to pgtool.
-*/
-function createOrDropDb(bool, pgtool, dbConfigObj) {
-  (bool ? pgtool.createdb : pgtool.dropdb)(
-  dbConfigObj,
-  dbConfigObj.name,
-  (err, res) => {
-    if (err) {
-      console.error('ERROR', err.name); // eslint-disable-line
-      process.exit(1); // eslint-disable-line
-    }
-
-    console.log(`${res.command} "${dbConfigObj.name}"... OK`); // eslint-disable-line
-    process.exit(0); // eslint-disable-line
-  });
-}
-
+const u = require('./utils');
 const cli = commandLineArgs([
   { name: 'init', alias: 'i', type: Boolean },
   { name: 'drop', alias: 'd', type: Boolean },
 ]);
 
-const options = cli.parse();
-const keys = Object.keys(options);
+/**
+ * Must be called with one and only one arg. Throws error if invalid arg count.
+ *
+ * @param {Array} args - Array of keys from the command line args.
+ * @throws {Error} - No args or more than one arg.
+ */
+function validateArgCount(args) {
+  if (args.length !== 1) { // eslint-disable-line no-magic-numbers
+    throw new Error();
+  }
+} // validateArgCount
 
-if (keys.indexOf('init') >= 0) {
-  createOrDropDb(true, pgtools, dbConfig);
-} else if (keys.indexOf('drop') >= 0) {
-  createOrDropDb(false, pgtools, dbConfig);
-}
+try {
+  // Parse the command line options
+  const options = cli.parse();
+  const keys = Object.keys(options);
 
+  // Make sure we only have one command line arg. More or less is an error.
+  validateArgCount(keys);
 
+  // Create or drop the database.
+  if (keys.includes('init')) {
+    u.createOrDropDb(pgtools.createdb)
+    .then((res) => {
+      u.clog('createOrDropDb', '', res);
+    })
+    .catch((err) => {
+      u.clog('createOrDropDb', '', err.message);
+    });
+  } else {
+    u.createOrDropDb(pgtools.dropdb)
+    .then((res) => {
+      u.clog('createOrDropDb', '', res);
+    })
+    .catch((err) => {
+      u.clog('createOrDropDb', '', err.message);
+    });
+  }
+} catch (err) {
+  u.clog('createOrDropDb', '',
+      'Script must be called with one command line arg: ' +
+      '--init OR -i OR --drop OR -d.');
+}

db/createOrUpdateDb.js

@@ -9,34 +9,69 @@
 /**
  * ./db/createOrUpdateDb.js
  *
- * If database and table schema already exist, just execute migratedb to
- * perform the necessary migrations to bring the db up to the present.
- * Otherwise, create the database, run resetdb to create the tables and
- * indexes, and do "pseudo-migrations" to bring the migration table up to date.
+ * This is called from the npm "checkdb" script, which is run as part of npm
+ * prestart. If the database and table schema already exist, it just executes
+ * migratedb to perform the necessary migrations to bring the db up to the
+ * present. Otherwise, it creates the database, runs resetdb to create the
+ * tables and indexes, and does "pseudo-migrations" to bring the migration
+ * table up to date.
  */
-const models = require('./index');
+const Sequelize = require('sequelize');
+require('sequelize-hierarchy')(Sequelize);
+const conf = require('../config');
+const env = conf.environment[conf.nodeEnv];
+const seq = new Sequelize(env.dbUrl, {
+  logging: env.dbLogging,
+});
 const pgtools = require('pgtools');
-const utils = require('./utils');
+const u = require('./utils');
+
+/**
+ * Create the database, run resetdb to create the tables and indexes, and do
+ * "pseudo-migrations" to bring the migration table up to date.
+ */
+function createAndReset() {
+  u.createOrDropDb(pgtools.createdb)
+  .then((res) => {
+    u.clog('createOrUpdateDb', 'createAndReset', res);
+    return u.reset();
+  })
+  .then((res) => {
+    u.clog('createOrUpdateDb', 'createAndReset', res);
+    process.exit(u.ExitCodes.OK); // eslint-disable-line no-process-exit
+  })
+  .catch((err) => {
+    u.clog('createOrUpdateDb', 'createAndReset', err.message);
+    process.exit(u.ExitCodes.ERROR); // eslint-disable-line no-process-exit
+  });
+} // createAndReset
 
-models.sequelize.query(`select count(*) from
+seq.query(`select count(*) from
   information_schema.tables where table_schema = 'public'`)
 .then((data) => {
   if (data[0][0].count === '0') { // eslint-disable-line
-    require('./reset.js'); // eslint-disable-line global-require
+    // The database exists but the table schemas do not exist.
+    u.reset()
+    .then((res) => {
+      u.clog('createOrUpdateDb', '', res);
+      process.exit(u.ExitCodes.OK); // eslint-disable-line no-process-exit
+    })
+    .catch((err) => {
+      u.clog('createOrUpdateDb', '', err.message);
+      process.exit(u.ExitCodes.ERROR); // eslint-disable-line no-process-exit
+    });
   } else {
+    // The database AND the table schemas exist.
     require('./migrate.js'); // eslint-disable-line global-require
   }
 })
 .catch((err) => {
-  const dbConfig = utils.dbConfigObjectFromDbURL();
-  console.log('create db now', dbConfig.name); // eslint-disable-line
-  pgtools.createdb(dbConfig, dbConfig.name, (err2, res) => {
-    if (err2) {
-      console.error('ERROR', err2); // eslint-disable-line
-      process.exit(1); // eslint-disable-line
-    }
-
-    console.log(`${res.command} "${dbConfig.name}"... OK`); // eslint-disable-line
-    require('./reset.js'); // eslint-disable-line global-require
-  });
+  if (err.name === 'SequelizeConnectionError' &&
+    err.message === `database "focusdb" does not exist`) {
+    // Database does not exist.
+    createAndReset();
+  } else {
+    // Some other error.
+    u.clog('createOrUpdateDb', '', err.name + ': ' + err.message);
+  }
 });

db/index.js

@@ -12,77 +12,12 @@
  * Database Model Loader
  */
 'use strict'; // eslint-disable-line strict
-
-const fs = require('fs');
-const path = require('path');
-const Sequelize = require('sequelize');
-require('sequelize-hierarchy')(Sequelize);
-const conf = require('../config');
-const dbutils = require('./utils');
-
-const env = conf.environment[conf.nodeEnv];
-const LAST_THREE_CHARS = -3;
-const seq = new Sequelize(env.dbUrl, {
-  logging: env.dbLogging,
-});
-
-/**
- * Tests whether the file is a javascript file.
- *
- * @param {Object} file - The file to test.
- * @returns {Boolean} True if file is a file (not a directory) and its name
- *  ends with .js.
- */
-function isJavascriptFile(file) {
-  return fs.statSync(file).isFile() && // eslint-disable-line no-sync
-    file.slice(LAST_THREE_CHARS) === '.js';
-}
-
-/**
- * Imports all of the the model definitions from the db/model directory then
- * executes any designated post-import tasks (like setting up associations and
- * scopes, which depend on the existence of other models).
- * Initializes the "Admin" User and Profile.
- *
- * @param {String} modelDirName - The directory where the model definitions
- *  are located.
- * @returns {Object} Object containing each of the models.
- */
-function doImport(modelDirName) {
-  const dir = path.join(__dirname, modelDirName);
-  const imported = {};
-  const filteredFileArray = fs.readdirSync(dir) // eslint-disable-line no-sync
-  .map((f) => path.join(dir, f))
-  .filter((f) => isJavascriptFile(f));
-  for (let i = 0; i < filteredFileArray.length; i++) {
-    const m = seq.import(filteredFileArray[i]);
-    imported[m.name] = m;
-  }
-
-  /*
-   * Add any missing tables and indexes based on the imported model
-   * definitions.
-   */
-  seq.sync();
-
-  const keys = Object.keys(imported);
-  for (let i = 0; i < keys.length; i++) {
-    const m = keys[i];
-    if (imported[m].postImport) {
-      imported[m].postImport(imported);
-    }
-  }
-
-  dbutils.initializeAdminUserAndProfile(seq);
-  return imported;
-}
-
-/*
- * Set up the "db" object and prepare it to be exported. It will contain the
- * imported models, the "sequelize" instance and the Sequelize class.
- */
-const db = doImport(conf.db.modelDirName);
-db.sequelize = seq;
-db.Sequelize = Sequelize;
-
+const u = require('./utils');
+
+u.doImport();
+u.seq.sync(false);
+u.initializeAdminUserAndProfile();
+const db = u.seq.models;
+db.sequelize = u.seq;
+db.Sequelize = u.Sequelize;
 module.exports = db;