If any of these steps is incorrect, please email me at pjvh@umich.edu and I'll see what I can do to improve things.
If everything goes well, you should be able to build a PheWeb with the following commands:
pip3 install git+https://github.com/statgen/pheweb.git
mkdir ~/my-new-pheweb && cd ~/my-new-pheweb
pheweb phenolist glob --simple-phenocode /data/my-analysis/*/*.epacts.gz
pheweb process-assoc-files
pheweb serve --port 5000
# open http://localhost:5000 in your web browser
Here are more detailed instructions:
Run
pip3 install pheweb
.If that doesn't work, use a virtualenv like this:
python3 -m venv ~/venv3 # Choose whatever path you like. ~/venv3/bin/activate pip3 install pheweb
Make a data directory. It should be in a location where you can afford to store twice as much data as the size of your input files.
- All
pheweb ...
commands should be run while in this directory. Alternatively, you may set the environment variablePHEWEB_DATADIR="/path/to/data/dir
.
- All
In your data directory, make a file
config.py
if you want to configure any options. Options you can set:minimum_maf
: any variant that has at least this minor allele frequency in some phenotype will be shown. (default:minimum_maf = 0
)cache
: a directory where files used by all datasets can be stored. If you don't want one, setcache = False
. (default:cache = "~/.pheweb/cache/"
)
Make sure you have tabix, bgzip, wget, and g++. If you can't just run
tabix
,bgzip
,wget
, andg++
from the command line, find a way to install them or add them to your$PATH
.
You should have one file for each phenotype. It can be gzipped if you want. It should be tab-delimited and have a header row. Variants must be sorted by chromosome and position, with chromosomes in the order [1-22,X,Y,MT].
- If you are using EPACTS, your files should work just fine. If they
don't, email me. EPACTS files won't have
REF
orALT
, but PheWeb will parse theirMARKER_ID
column to get those.
The file must have columns for:
- chromosome
- named
#CHROM
orCHROM
(all column names are not case-sensitive) - must be a number between 1 and 22 or
X
orY
orM
orMT
- named
- position
- named
POS
,BEG
, orBEGIN
- must be an integer
- named
- reference allele
- named
REF
- named
- alternate allele
- named
ALT
- named
- minor allele frequency
- named
MAF
- must be a real number between 0 and 1 (numbers may be in
scientific notation, like
5.4e-12
)
- named
- p-value
- named
PVAL
orPVALUE
- must be decimal number between 0 and 1 or
.
orNA
(both representing unknown)
- named
You may also have columns for:
- effect size
- named
BETA
- must be a real number
- named
- standard error of effect size
- named
SEBETA
- must be a real number
- named
If you need Odds Ratio, I can add that.
Inside of your data directory, you need a file named pheno-list.json
that looks like this:
[
{
"assoc_files": ["/home/watman/ear-length.epacts.gz"],
"phenocode": "ear-length"
},
{
"assoc_files": ["/home/watman/eats-kimchi.X.epacts.gz","/home/watman/eats-kimchi.autosomal.epacts.gz"],
"phenocode": "eats-kimchi"
},
...
]
phenocode
must only contain letters, numbers, or any of _-~
.
That example file only includes the columns assoc_files
(a list of
paths to association files) and phenocode
(a string representing
your phenotype that is valid in a URL). If you want, you can also
include:
phenostring
: a string that is more descriptive thanphenocode
and will be shown in several placescategory
: a string that will group together phenotypes in the PheWAS plot and also be shown in several placesnum_cases
,num_controls
, and/ornum_samples
: numbers of strings which will be shown in several places- anything else you want, but you'll have to modify templates to show it.
There are four ways to make a pheno-list.json
:
If you have a csv (or tsv, optionally gzipped) with a header that has EXACTLY the right column names, just import it by running
pheweb phenolist import-phenolist "/path/to/my/pheno-list.csv"
.If you have multiple association files for each phenotype, you may put them all into a single column with
|
between them.For example, your file
pheno-list.csv
might look like this:phenocode,assoc_files eats-kimchi,/home/watman/eats-kimchi.autosomal.epacts.gz|/home/watman/eats-kimchi.X.epacts.gz ear-length,/home/watman/ear-length.all.epacts.gz
If you have one association file per phenotype, you can use a shell-glob and a regex to get assoc-files and phenocodes for them.
Suppose that your assocation files are at paths like:
/home/watman/eats-kimchi.epacts.gz
/home/watman/ear-length.epacts.gz
Then you could run
pheweb phenolist glob-files "/home/watman/*.epacts.gz"
to getassoc-files
.To get
phenocodes
, you can use a regex that captures the phenocode from the file path. In most cases (including this one), just use:pheweb phenolist extract-phenocode-from-fname --simple
If you have multiple association files for some phenotypes, you can follow the directions in 2 and then run
pheweb phenolist unique-phenocode
.For example, if your association files are at:
/home/watman/autosomal/eats-kimchi.epacts.gz
/home/watman/X/eats-kimchi.epacts.gz
/home/watman/all/ear-length.epacts.gz
then you can run:
pheweb phenolist glob-files "/home/watman/*/*.epacts.gz" pheweb phenolist extract-phenocode-from-fname --simple pheweb phenolist unique-phenocode
If you want to do more advanced things, like merging in more information from another file, email pjvh@umich.edu and I'll write documentation for
pheweb phenolist
.
No matter what you do, please run pheweb phenolist verify
when you
are done to check that it worked correctly. At any point, you may run
pheweb phenolist view
or pheweb phenolist print-as-csv
to view
the current file.
Run
pheweb process-assoc-files
.If something breaks, read the error message.
- If you can understand the error message, modify your input files to avoid it.
- If the problem is something that PheWeb should support by default, feel free to email it to me at pjvh@umich.edu.
- If you can't understand the error message, please email your error message to pjvh@umich.edu and hopefully I can get back to you quickly.
Then re-run
pheweb process-assoc-files
.
Run pheweb serve
.
- If port 5000 is already taken, choose a different port (for example,
5432) and run
pheweb serve --port 5432
instead.
Next you need to find a way to for your computer to access the server. You have two options:
Run PheWeb exposed to anybody on the internet. This might be dangerous, but I never worry much about it.
You need a port that can get through your firewall. 80 or 5000 probably work, though 80 will require you to run something like
sudo $(which python3) $(which pheweb) serve --port 80
.Find an IP adddress or hostname that refers to your server. If you ssh into your server with
ssh watman@foobar.example.com
, this isfoobar.example.com
. If you don't know this, runcurl http://httpbin.org/ip
on your server to get its IP address. (If it returns something like"origin": "12.34.5.678"
, your server's IP is12.34.5.678
).Now run
pheweb serve --port <myport> --host <myhost>
. For example, if you're using the default port (5000), andcurl http://httpbin.org/ip
returns"origin": "12.34.5.678"
, then runpheweb serve --port 5000 --host 12.34.5.678
.When the server starts, it should say something like
Running on http://12.34.5.678:5000/ (Press CTRL+C to quit)
. Open that URL in the web browser on your computer.Run PheWeb with the default settings, then use an SSH tunnel to connect to it from your computer.
For example, if you normally ssh in with
ssh watman@foobar.example.com
, then the command you should run (from your local computer) isssh -N -L localhost:5000:localhost:5000 watman@foobar.example.com
. Now open http://localhost:5000 in your web browser.
At this point your PheWeb should be working how you want it to, and everything should be good except maybe the URL you're using.
To start, run Flask behind gunicorn. To do that, run pheweb make-wsgi
,
to produce a file wsgi.py
. Then you can run
gunicorn -b 0.0.0.0:5000 -w4 wsgi
, to start a webserver.
For maximum speed and safety, you should run gunicorn behind something like Apache2 or Nginx. More information about this is here. If you choose Apache2, I have some documentation for you here.