Metabolic Pathway Visualization
There backend comprises several components.
The primary component is the Flask web server which handles URL routing for
webpages and all other requests from the frontend. The Flask server is
implemented in the
metapaths.py file. This code is responsible for serving
the pages that the user visits as well as managing the search tasks.
When a user submits a pathway search, the Flask server utilizes Celery to manage that search task. Celery is a distributed task queue and allows the main server process to continue handling requests while the search is executed asynchronously by Celery workers.
The backend is also supported by two MySQL databases:
These databases can be recreated by sourcing the provided database dumps in the
resources directory (see in-depth instructions in the Installation
metadb database contains a single table,
provides a mapping of KEGG IDs to compound names. This information is requested
by the frontend and used to populate fields and label nodes in the
hubdb database contains a table for each pair of hub compounds for which
there are pathways. For example, each row in the
describes a path from
C00024. The information in this database is
used to populate the hub visualization when a user clicks to expand a hub link.
The HTML for webpages resides in the
templates directory and is served by
the Flask server. The frontend makes extensive use of
Bootstrap for the UI components and
Select2 for the searchable drop-down select boxes.
The visualization is created using a force-directed graph layout from
Instructions are applicable for Debian/Ubuntu and Mac OSX.
$indicates command should be executed from the shell
mysql>indicates command should be executed from the MySQL server REPL
$ sudo pip install Flask
$ sudo pip install celery
$ sudo pip install mysql-python
MySQL Server: https://dev.mysql.com/downloads/mysql/
$ git clone https://github.com/KavrakiLab/metapaths.git
- Complete the initial post-installation setup of MySQL Server. The default
rootand the password is
meta. If you configure with a different username and password, you must update the
metapaths.py. Start and login to the MySQL server with:
$ mysql -u root -p
mysql> CREATE DATABASE metadb; mysql> USE metadb; mysql> SOURCE ./metapaths/resources/metadb_2017-04-30.sql;
mysql> CREATE DATABASE hubdb; mysql> USE hubdb; mysql> SOURCE ./metapaths/resources/hubdb_2017-04-30.sql;
Once all the dependencies are available and the databases have been setup, we can run the main Flask server.
If using Ubuntu or Debian, the RabbitMQ server will begin running right after
installation, so no further action is needed. On Mac OSX, you will need to run
rabbitmq-server to start up the broker.
Next, we need to intialize Celery.
$ cd metapaths $ celery worker -A metapaths.celery --loglevel=info
This creates a celery work for the metapaths application. The log level can be configured as desired. You can run the Celery worker in its own shell and use another shell to run the Flask server, or the worker can be run in the background and its output and logging can be redirected into a log file like this:
$ cd metapaths $ celery worker -A metapaths.celery --loglevel=info &> celery_worker.log &
Lastly, we can start the Flask server:
$ cd metapaths $ export FLASK_APP=metapaths.py $ flask run
The Flask server will initialize and the webapp will become available at localhost:5000
$ sudo /etc/init.d/apache2 reload
If we make changes to the python scripts, we can update the server using the following process:
$ killall ../bin/python2.7 $ ../bin/celery multi start worker -A metapaths.celery --loglevel=info --pidfile=../run/%n.pid --logfile=../log/%n%I.log export FLASK_APP=metapaths.py $ nohup flask run & $ ../bin/celery multi start worker --loglevel=info --pidfile=../run/%n.pid --logfile=../log/%n%I.log $ sudo /etc/init.d/apache2 reload
The first line kills all existing celery workers. Then the next two lines recreates a celery worker for metapaths and restarts the Flask server. The last line restarts apache.
Deploying to a non-local server
The whole metapaths web app can be deployed to the cloud using apache and mod_wsgi. Complete instructions can be found here.
This guide can be used to setup apache and mod_wsgi, but the other installation instructions regarding dependencies, database setup, and celery will also need to be followed.
A Note On Usage
Celery and RabbitMQ are necessary for the search execution functionality of the webapp. However, it is possible to just run the Flask web app and still use all of the visualization features.
This can be done in two ways:
By uploading properly formatted JSON using the upload feature on the webapp
For examples of the JSON accepted by this feature, see the
resourcesdirectory. This is the same JSON format as is given when pathways are exported from the web app.
By placing a text file of pathways in the
searches/outputdirectory and clicking "View more pathways" on the web app.
The format of these text files is simple; each line represents a pathway and contains a comma delimited list of KEGG compound and RPair IDs. For a sample of this format, inspect any of the files in the
Note that this is not exactly the same format as that output by the AtomMetaNet algorithms. However, the
path_convert.pytool (located in the
searchesdirectory) can be used to convert certain supported formats into this format. For example, to convert LPAT:
python path_convert.py lpat lpat_output.txt
To convert Hub output:
python path_convert.py hub hub_output.txt
path_convert.pyoverwrites the original input file with the converted format.