Developer notes: Javascript toolchain

While it takes advantage of some Open Tree of Life APIs, Tree Illustrator is implemented almost entirely in client-side code. This has some interesting advantages in terms of local deployment, testing, and serving multiple versions as static files, directly from GitHub.

However, we recommend using a few tools when making and testing changes. This page will describe these tools in brief, and (more importantly) how to get them up and running.

Quick start (tested on OS X, Debian and Ubuntu)

You'll need fairly recent versions of Git, Node.js and its included npm (Node Package Manager). You'll also need python, or another tool that can serve static local files as a website (more on this below).

To install "from scratch" on a bare-bones Debian/Ubuntu system:

$ sudo apt install git
$ sudo apt install nodejs-legacy
$ sudo apt install npm

Next, clone the Tree Illustrator repo on GitHub, then move to its stylist subdirectory (assuming our project directory is ~/tree-illustrator):

$ cd ~
$ git clone https://github.com/OpenTreeOfLife/tree-illustrator.git
Cloning into 'tree-illustrator'...
$ cd tree-illustrator/stylist

Next we install the node modules for our project:

$ npm install

This checks the current directory for a package.json file and installs whatever dependencies it finds there. The result is a local node_modules/ folder with all dependencies installed in a recursive tree.

NOTE that you might get some errors when installing dependencies, but you can safely ignore these for our purposes:

npm WARN deprecated d3.layout.cloud@1.2.0: This package has been renamed to d3-cloud. Please install d3-cloud instead
messages during install of canvas module, relating to
- missing package cairo
- problems installing node-gyp
- problem detecting 'freetype' library (in binding.gyp)
npm WARN optional Skipping failed optional dependency /chokidar/fsevents:

Since these requirements are occasionally updated, it's not a bad idea to re-run the npm install command after changing repo branches or pulling a newer version from GitHub.

My recommended development workflow uses three terminals to launch a local browser, edit code, and monitor JS bundling after any changes. In the first terminal, we'll run :

$ cd ~/tree-illustrator/stylist
$ npm run watch

After a few seconds, you should see something like this: 4076987 bytes written to dist/stylist-bundle.js (3.23 seconds). Otherwise, troubleshoot any error that appears here (and re-run the above command if it has quit). Typically, any errors here are bugs in your latest JavaScript edits, but you might also see problems from a failed npm install.

`` This starts a "watch" script that monitors your Javascript source files. After any saved changes, this will attempt to bundle all your JS scripts and modules into a single stylist-bundle.js file that's actually served in the browser.

In a second terminal, launch a local webserver to test your changes (here we use python):

$ cd ~/tree-illustrator
$ python -m SimpleHTTPServer

You should be able to see the running app at http://localhost:8000/stylist/stylist.html. To use a different port, just modify the above command like python -m SimpleHTTPServer 8888.

You can use a third terminal to explore and edit source code and markup. Any changes should result in a fresh JS bundle in the 'watchify' terminal, or a visible error message if there are JS errors. Naturally, you'll need to reload the browser to test any changes!

Bundling and CommonJS modules

What's all this "bundling" about? We're using CommonJS modules to manage complexity in the JavaScript and take advantage of third-party packages. There are other tools that handle this; our choice was somewhat driven by the conventions used in Vega.

Speaking of which, here are the top-level dependencies in our current package.json, explained:

vega provides repeatable, grammar-based visualization for trees and figures
vega-logging provides additional logging features for Vega visualizations
d3-cloud is redundant, as it's included in the vega module; here we use its modern name, just to suppress an error message during npm install
jszip is used for in-browser (de)compression of archived illustrations, for storage as local files and for transport to shared storage in Open Tree's phylesystem storage API
file-saver is used to support familiar loading/saving of local files, for offline storage and sharing
blob-polyfill supports a modern WC3 Blob interface in older browsers