Hybrid rendering application for the Performance Platform using Backbone and D3
Switch branches/tags
add-csv-files-for-q1-2016-txex-data amend-about-text backdrop-migration bugfix/fix-master cleanup/schema cleanup/stubs config/initial-blood-donor-police-uk-dashboards config/update-blood-donation-appllications-dashboard dashboards-from-stagecraft-with-fallback dep-updates-wip deployed-to-integration deployed-to-preview deployed-to-production deployed-to-staging enhancement/js-source-maps feature/all-flat feature/big-screen-link feature/flat-data-for-bar-chart-with-numbers feature/flat-data-for-column feature/flat-data-for-completion-rate feature/keepaliveagent feature/log-request-id feature/module-groups feature/performance-emails feature/request-timing feature/request-timings feature/revision-upgrades feature/serve-tx-csv feature/74439456-register-to-vote-changes fix-501-on-dashboards-pages google-crawler-500 homepage-redesign-july iterate-dashboard-template journey-refactor json-link-get-current-date-range master mobile-sort more-granular-timing-attempt new-homepage no-data-no-crash node-v-upgrade non-dev-setup-docs not-migrate-config parse-config-file-as-mustache production prototype/carers-referrers prototype/complaints prototype/date-picker prototype/new-graphs-inc-transaction-time-and-exit-rate prototype/tax-credit-dashboard ralph-style-changes reduce-jenkins-to-travis refactor-scss refactor/backdrop-queries refactor/data-sources refactor/template-reorganisation release remove-about-page remove-breadcrumbs remove-data-from-hompage remove-survey remove-unused-arg rename-hmrc-prototype replace-download-files revert-928-revert-927-cheapseats save-stagecraft-dashboard-list-branch staging temp-branch-for-schema-transfer update-logit-drain upgrades working-dev-tx-dashboard-creator
Nothing to show
Clone or download
tlwr Merge pull request #1101 from alphagov/159387369-import-transaction-e…
…xporer-metrics-into-production

Use updated data from The Spreadsheet™
Latest commit ed3777f Sep 19, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
app don't crash if there's no data to put in the table May 30, 2018
assets Use updated data from The Spreadsheet™ Sep 19, 2018
config Remove big screen view link from spotlight Dec 15, 2017
docs Update the defaults documentation May 11, 2015
etc change permissions on deploy script May 30, 2018
log Use Winston for logging to a file in production Nov 1, 2013
schema Fix JSHint errors Mar 12, 2015
spec Remove big screen view link from spotlight Dec 15, 2017
styles Remove survey from top of pages Mar 19, 2018
tests Fix typo in functional test Mar 22, 2017
.dockerignore Paas migration Aug 9, 2017
.gitignore Try a different path Feb 22, 2018
.jshintignore Try a different path Feb 22, 2018
.nvmrc Align version to manifest May 31, 2018
.python-version Loosen python requirements Jan 26, 2018
.travis.yml Fix travis deployment sudo requirement. Sep 7, 2018
CONTRIBUTING.md Add suggestion for images in pull requests Feb 9, 2015
Gruntfile.js Try a different path Feb 22, 2018
LICENSE Initial commit Oct 9, 2013
README.md Bug has been fixed Oct 27, 2017
app.json Set key value to string, so it works properly in heroku dashboard Aug 8, 2014
manifest.production.yml update logit service name May 30, 2018
manifest.staging.yml The staging branch should run with NODE_ENV=staging Mar 29, 2018
nightwatch.json Major version bumps Dec 19, 2017
package-lock.json Rollback to when things used to work Feb 7, 2018
package.json Upgrade minor version. Sep 7, 2018
request-lifecycle.md Add documentation about how spotlight handles requests Dec 1, 2017
start-app.sh Remove separate VM config Jun 9, 2014

README.md

Build status

Dependency status

Spotlight

Hybrid rendering app for the GOV.UK Performance Platform using Backbone and D3. JavaScript is shared between the client and server, and the app makes use of progressive enhancement to provide a great experience in every browser.

Building and running the app

Development

Just Spotlight: The simplest way to get started is to run just this app, against production data.

Firstly, it is recommended that you set up Node Version Manager on your host. See the (nvm) README for installation instructions.

Next checkout the Spotlight repo and create an .nvmrc file in its root directory containing the version of node specified in the 'engines' entry in package.json e.g. 6.11.2.

Now install the specified version of node using nvm:

nvm install 6.11.2

To check you have the correct version of node installed:

nvm which

Found '/Users/<username>/<path to>/spotlight/.nvmrc' with version <6.11.2>
/Users/<username>/.nvm/versions/node/v6.11.2/bin/node

Now tell nvm to use the version of node specified in the .nvmrc file:

nvm use

You can then run the app as follows:

npm install
npm start

Now you should be able to connect to the app at http://localhost:3057.

The app uses node-supervisor and grunt-contrib-watch to monitor changes, automatically restart the server and recompile Sass.

By default, this will look at production data, but perhaps you want to connect to a different data source. You can do that by creating your own config file in /config/config.development_personal.json that mimics /config/config.development.json with a different backdropUrl property. It'll be ignored by Git.

Full stack: if you're using our development environment then you can run all our apps in one go and use a real database for development. As a bonus, this will let you test the image fallbacks using the screenshot-as-a-service app.

First, you need to set up the Performance Platform development environment.

Once you have a machine with the required system-level dependencies, you can run the application with:

cd /var/apps/pp-puppet/development
bowl performance

Running tests

Command line

The Jasmine tests are divided into ones that work on the client (test/spec/client), and ones that work on the server (test/spec/server and test/spec/shared). The client tests are run using Jasmine v2.x, while the server tests are using Jasmine v1.x. It used to be that both were written for Jasmine v1, but after upgrading node versions our client tests needed to be upgraded.

npm test runs both client and server tests, as well as linting the codebase:

  • npm run jasmine_node executes server Jasmine tests in Node.js
  • npm run jasmine executes client Jasmine tests in PhantomJS
  • npm run shell:cheapseats executes feature tests using cheapseats with a small subset of dashboards, for speed
  • npm run shell:cheapseats_full_run runs cheapseats with all dashboards
  • npm run test:functional executes functional tests using [nightwatch][https://github.com/beatfactor/nightwatch]
Functional tests

As part of the CI (travis) npm run test:functional:ci is run. This spins up an instance of spotlight, nightwatch and phantomjs to run the tests in a headless environment.

To assist with debugging the functional tests can also be run in a selenium webdriver using the following command npm run test:functional:ff

If you want to run against firefox,chrome and phantom you can also do npm run test:functional:all.

All the functional tasks except ci will require a server to be running already.

Debugging locally

Install node-inspector where the app runs with sudo npm install -g node-inspector@0.5.0 and run it with node-inspector.

Start the app with node --debug app/server.js and visit http://spotlight.perfplat.dev:8080/debug to view the console.

Production

npm run build:production to create a production release.

NODE_ENV=production node app/server.js to run the app in production mode.

Heroku

If you want to deploy the app to Heroku, follow these instructions.

Create an app on Heroku

Using the web interface, or the CLI:

heroku create <app-name>

Set the app to use the node-grunt buildpack

The app runs on Heroku using a custom buildpack for Grunt.js support.

This means it will run the grunt commands we need to compile the app when deploying code.

 heroku config:set BUILDPACK_URL=https://github.com/mbuchetics/heroku-buildpack-nodejs-grunt.git

Set configuration vars

heroku config:set NODE_ENV=development # makes app run in development mode
heroku config:set npm_config_production=true # does not install dev dependencies

Deploy the code

If the code you're deploying is not in master, then you'll need to make sure you specify your local branch to push to master. Otherwise it will just deploy your local master (and probably not work as expected).

git push heroku <your-branch-name>:master
heroku open # opens the freshly deployed app in a browser

Or just...

Deploy

If you want the Heroku app to be password-protected, set config variables as follows, before pushing the code.

heroku config:set BASIC_AUTH_USER=xxxx
heroku config:set BASIC_AUTH_PASS=xxxx
heroku config

Logging

You might also want to enable some logging in your Heroku app to assist with debugging. You can use logentries to do that:

heroku addons:add logentries

You can then access the logs from your app's dashboard on Heroku (under the "Add-ons" section).

Contributing

For Javascript, follow the styleguide (apart from the sections on GOV.UK modules as we don't use these)

Functionality should work without Javascript where possible.

All content should work well with screenreaders (at least Voiceover and JAWS). 'Work well' means

  • a screenreader user can orientate themselves effectively and use the page.
  • async updates are reported to the user (an 'accessibility' module exists for this).

Notes for Developers

Tables

Tables are used in the following places:

  1. To display a list of dashboards on the services, web-traffic and other dashboards pages
  2. To display data on a dashboard, e.g. a web-traffic dashboard
  3. To be displayed instead of a graph if javascript is disabled.

Configuration

As there are more tables underpinning graphs than any other tables in spotlight, table columns are configured as though they are the axes on a graph.

Columns defined in the x-axis will appear before those in the y-axis.

For example, axes defined as:

axes: {
    x: {
      key: 'key_X',
      label: 'Label X'
    },
    y: [
      {
        key: 'key_Y_one',
        label: 'Label Y1',
        format: 'integer'
      },
      {
        key: 'key_Y_two',
        label: 'Label Y2',
        format: 'integer'
      }
    ]
}

Will be displayed as:

Label X Label Y1 Label Y2
Value X Value Y One Value Y Two

The y-axis only accepts a list of column configuration.

The x-axis will accept a list or a single value. If a list is provided, the columns will appear in reverse order in the table. For example, x-axis columns defined as:

x: [
  {
    key: 'key_X_one',
    label: 'Label X1'
  },
  {
    key: 'key_X_two',
    label: 'Label X2'
  }
]

Will be displayed as:

Label X2 Label X1
Value X Two Value X One

Axes configuration is most commonly found in the module visualisation settings, and can be edited via the admin app.

Configuration for tables displaying lists of dashboards can be found in their respective controllers.