The Monarch Institute philosophy is based on the premise that we want to make all the data count. Monarch isn’t just another database that slurps data from the typical places and renders it in a different format. We are driven to truly integrate biological information using semantics, and present it in a novel way, leveraging phenotypes to bridge the knowledge gap. Our niche is the use of computational reasoning to enable phenotype comparison both within and across species, with the ultimate goal of improving biomedical research. More project information is available on our website https://monarchinitiative.org/page/about
The vast majority of the work we do is in GitHub, though spread across dozens of repositories and not even necessarily all under this GitHub organization (Monarch Initiative). The GitHub organizations that contain the overall work done by the Monarch Initiative team include:
- Monarch Initiative
- SciGraph
- Biolink
- INCA tools
- NCATS Translator - Tangerine team
- OBO Phenotype
- BioDatasets
- GeneOntology
- PrefixCommons
- OBO Foundry
This repo contains the source and configuration files used to implement the Monarch Initiative web application and associated tools.
You can build and run the web application yourself via the instructions below in Quickstart, or you can use the production version of the Monarch Initiative web interface at http://monarchinitiative.org.
The audience for this README is primarily developers and integrators of the Monarch technology. The Monarch web application has its own user-level documentation as described in Monarch Documentation below.
- Recent Changes
- Quickstart
- Detailed Launch Instructions
- Launching via supervisor
- Monarch Documentation
- Making changes
- Scripts
- Non-bundled use of
require.js - New UI Tools and Bundling Instructions
- Installing NodeJS and NPM for Monarch
- Legacy RingoJS installation and launch instructions
The Monarch Initiative web application server has been migrated from the RingoJS server-side Javascript framework to the NodeJS framework. Previous production versions of Monarch prior to October 2015 were based upon RingoJS, which uses the Rhino Javascript engine within a Java Virtual Machine. The same Javascript source code that implements Monarch is now running under NodeJS. The instructions herein refer to the NodeJS implementation unless otherwise clarified as RingoJS-specific.
Monarch now uses up-to-date NPM modules to provide common libraries such as jQuery, Mustache, D3, and more. Previously, these libraries were duplicated into the Monarch code base and were very out of date with respect to the NPM-available versions. This change required the adoption of require.js, a JS library that manages module dependencies in the browser. See Non-bundled use of require.js in this document for more details. Note that require.js is only used in the non-bundled mode.
The Monarch web application was designed to support the integration of diverse JavaScript libraries and HTML fragments. This was to encourage experimentation with different visualization frameworks and technology. The Monarch web application associates a given route (e.g., /page/about) with a handler that generates the correct webpage by assembling pieces via the server-side pup-tent library. This allows different web pages within the Monarch app to have completely different Javascript and CSS resources, and has been useful in the development of Monarch's features.
Recently, we have been evolving the codebase to support modern web front-end tooling, including the use of preprocessors (e.g., LESS, ESLint), the bundling and minification of JS and CSS resources, and a more rapid development cycle. In the short term, we expect this will result in more effective and pleasant development experience, as well as a more efficient web application. In the longer term, this will enable us to build parts of the Monarch UI as a single-page app.
Details on how to use the new tech are later in this document at New UI Tools and Bundling Instructions.
As of January, 2016, we have changed the default build behavior to use the new bundling via WebPack. The existing, unbundled behavior may be obtained by defining the environment variable USE_BUNDLE=0 prior to running the application, or by using the convenience task: npm run oldstart
You will need to have NodeJS and NPM installed. At the time of this writing, we are supporting the following versions, which can be reported via the npm version command:
> npm version
...
node: '4.5.0',
...
About Node Versioning: Node 4.x is the immediate successor to the 0.12.x version of NodeJS. The version number jumped from 4.4.1 to 4.0.0 as a result of the NodeJS committee adjusting their version-numbering system recently. More information is in the V4.0 Release Notes.
We are currently holding at 4.5.0 and HapiJS 11.0.2 due to GCC version issues on some of our CentOS deployment nodes. When we are able to update these nodes to GCC 4.8, then we can update package.json to reflect Node 4.2.x and HapiJS 11.1.x, which are the current stable and supported versions of these packages.
These versions do not have to be exact, and NPM version 2.15.x will likely work as well.
Currently, we have been successfully using the nvm tool to configure and manage our NodeJS environment; nvm enables a user to associate a paritcular NodeJS and NPM version with their Unix shell, allowing for each switching between NodeJS versions across different projects. If you need help in getting the Monarch-required NodeJS and NPM versions installed, please read the platform-specific instructions on installing NodeJS and NPM, see Installing NodeJS and NPM for Monarch below.
About Node Versioning: Node 4.x is the immediate successor to the 0.12.x version of NodeJS. The version number jumped from 0.12.2 to 4.0.0 as a result of the NodeJS committee adjusting their version-numbering system recently. More information is in the V4.0 Release Notes.
After downloading the Monarch GitHub repository (via git clone or as a .zip or .tar.gz), change your working directory to the downloaded source code directory and then type:
./install.sh
This will install the required modules, including NPM-based modules such as gulp and mustache, as well as the necessary NodeJS runtime tools.
After installation completes, start the server:
> npm run start
You're done! You now have a running Monarch Initiative web application.
Try it out by connecting your browser to:
Or view a particular disease, e.g:
Typically, the web application is started with:
> cd monarch-app
> npm run start
which can take an optional environment name parameter of: 'dev', 'stage', or 'production'. The 'dev' environment is the default. Note that you will need to separate the environment name parameter from the npm command by using -- (otherwise, the argument will be consumed by npm instead of the web application. For example:
> npm run start -- stage
This start-server.sh script is a thin veneer upon the underlying command:
NODE_PATH=./lib/monarch node lib/monarch/web/webapp_launcher.js
The webapp_launcher.js can take an optional --port portNumber parameter in addition to an optional environment name. For example:
> export NODE_PATH=./lib/monarch
> node lib/monarch/web/webapp_launcher.js dev --port 8888
On some deployments, it may be necessary to fully specify the path to the node executable. For example:
/usr/local/bin/node lib/monarch/web/webapp_launcher.js stage --port 8080
The Monarch application contains a variety of documentation, available via the 'Documentation' menu on the navbar in the locally running app at http://127.0.0.1:8080 or in the production app at http://monarchinitiative.org.
The Monarch API documentation can be viewed directly from the monarch-app source directory by opening the file doc/index.html in a web browser.
See CONTRIBUTIONS.md to learn about the Monarch developer workflow and code submission process.
See README-developers.md for information on the Monarch architecture, internals, and means of extending and integrating Monarch.
See bin/README.md.
When env variable USE_BUNDLE is set to the value '0', which we will call non-bundled or Legacy Mode, the Monarch UI uses RequireJS to load various JS files and modules, and to ensure they are loaded in the correct order.
The file requireconfig.js is included via monarch_base.mustache by RequireJS and this file
contains a set of require statements that include other JS files used by the Monarch front-end.
The default (as of Jan 15, 2016) is that Monarch operates in Bundled Mode unless USE_BUNDLE=0.
By default, when you run the Monarch server via ./start-server.sh, Monarch will operate in bundled mode. This means that when the web server delivers a page (e.g., /page/about), it will invoke a particular handler in lib/monarch/web/webapp.js and that handler (pageByPageHandler, in this example) will generate a custom HTML page by expanding a set Mustache templates and streaming the result back to the web browser. The custom HTML page includes CSS and JS file references that support the particular page being delivered.
One of the things that Monarch has relied on up to this point is the ability for different handlers to present different CSS and JS files to the browser, enabling us to experiment with diverse libraries and frameworks as we extend and integrate Monarch. Unfortunately, this technique is at odds with many of the goals of delivering a high-performance modern website, which encourages bundling, minification and caching of assets (CSS, JS) in the browser.
We have improved our front-end page generation pipeline to accommodate the current on-the-fly page composition technique as well as to allow for the use of modern tooling to bundle and minify CSS and JS assets for efficient delivery and caching. This bundled mode is enabled by default when Monarch is invoked by ./start-server.sh.
The bundle behavior is currently controlled via the environment variable USE_BUNDLE, which directs webapp.js to include dist/app.bundle.js and dist/app.bundle.css into the dynamically generated page. This is in contrast to non-bundled mode, where a variety of page-specific JS and CSS resources are included.
Here's one way to experiment with the bundled execution:
> npm run build # This takes a minute or so, generates dist/app.bundle.* and other dist/*
> npm run start # Runs webapp_launcher.js
Alternatively:
> rm -rf dist/*
> webpack --config webpack.build.js --bail
> NODE_PATH=./lib/monarch node lib/monarch/web/webapp_launcher.js
One drawback to the use of bundled mode above is that any time a developer
edits a front-end resource (e.g., a CSS or JS file), they will have to
regenerate the bundle using the above commands, which is slow and tedious. That is why most rapid development will occur usig the webpack-dev-server, which allows a developer to make a change to a CSS/JS file and immediately see the result delivered to their browser, without restarting any servers. The only time the slow-bundle needs to occur is during a production build (or during the unit tests prior to a pull request).
To exercise the rapid development tools:
> npm run dev # Starts webapp, webpack-dev-server, browsersync, nodemon
Note that your default web browser will open up automatically and you will be pointed to the BrowserSync proxy URL: http://localhost:3000/. If you want to see the application without BrowserSync, use the url: http://localhost:8081/. This 8081 URL is also the one to use when running behave tests:
> cd tests/behave
> source bin/activate
> TARGET=http://localhost:8081/ behave
BrowserSync is a free and open source tool and library that is useful for web designers and QA folk for debugging and testing UIs under a variety of browsers and conditions. We at Monarch are primarily using it to provide us with automatic browser reload when we change an asset, but it has many other features.
WebPack is a free and open source tool and library that provides bundling, preprocessing and an efficient module system based upon Browserify and NodeJS require statements. WebPack is very extensible, and the Monarch tooling includes several preprocessors as part of the build chain:
- LESS Preprocessor
- Babel ES6
- Babel ESLint
- Uglify
- JSON, CSS, JS, and other asset loaders
webpack-dev-server is a NodeJS server that integrates with WebPack to deliver bundled assets incrementally to a web browser during active development. It enables a developer to see the effect of their changes immediately in the browser, and then later they can perform the slower and more thorough npm run build operation.
Although your development machine may be running NodeJS and NPM currently, it is likely not the exact same version that Monarch is currently supporting (NodeJS 4.5.0 and NPM 3.10.8). The instructions below may help achieve the proper NodeJS and NPM versions. If you are not familiar with NodeJS and NPM, then this may help. Otherwise, use your ordinary technique for achieving NodeJS 4.5.0 and NPM 3.10.8 and skip the remainder of this section.
If you are not running a current version of NodeJS, use the instructions below in:
Once you have a correct NodeJS version running and selected as the current default for your shell (e.g., nvm use 4.5.0), then you can ensure that the correct NPM is installed by using the command:
> npm install -g npm@latest
Verify that you are running the correct versions by:
> npm version
which should output something like:
> ...
> npm: '3.10.8',
> ...
One of the easiest ways to install an alternative version of Node is to use the nvm tool available at https://github.com/creationix/nvm. If you have nvm installed, you can use the following command to installed NodeJS v4.5.0:
> nvm install v4.5.0
This will download, compile and install the 4.5.0 version of NodeJS into the ~/.nvm directory, making it available for the next command:
> nvm use v4.5.0
This command will change your current NVM environment so that it sees a v4.5.0 version of NodeJS.
If you have MacOSX, and you have HomeBrew installed, then the following command will be sufficient to install nvm:
> brew install nvm
Follow the instructions printed to your console after the above brew install nvm. The most important part of the instructions are:
You should create NVM's working directory if it doesn't exist:
mkdir ~/.nvmAdd the following to ~/.bash_profile or your desired shell configuration file:
export NVM_DIR=~/.nvm source $(brew --prefix nvm)/nvm.sh
The instructions on the nvm GitHub Site provide a way to install NVM easily; these have been adapted below.
Download install.sh via curl:
> cd /tmp
> curl -O https://raw.githubusercontent.com/creationix/nvm/v0.29.0/install.sh
Alternatively, download install.sh via wget:
> cd /tmp
> curl -O https://raw.githubusercontent.com/creationix/nvm/v0.29.0/install.sh
Run the install.sh script
> bash install.sh
According to the nvm site, this script will modify your .bashrc or .bash_profile automatically. See Manual Install for more information if the above install.sh does not work.
There is a tool called n that may be useful for installing the proper NodeJS versions if the nvm-based solutions above do not work. You may have also adopted n for a different project, in which case it may be used for Monarch.
The n tool is available from https://github.com/tj/n where you can find the necessary instructions.
The RingoJS version of Monarch is currently deprecated as we are committed to making the Monarch NodeJS experience great. It is likely that the RingoJS support for Monarch will be eliminated in the near future. Until then, the following instructions should assist in running the RingoJS version.
-
Install RingoJS - http://ringojs.org
./installRingo.sh
-
Run app
./ringo-server.sh