Downloading and Parsing Hail LD matrices

Documentation	Build Status	Code Coverage

This is a Julia package that helps users download and read blocks of LD (linkage disequilibrium) matrices stored as HailBlockMatrix format into memory. This package is inspired by an existing pipeline curated by Aaron Sossin.

We tested this package for processing the following

• gnomAD LD matrices
• Pan-UKBB LD matrices

Installation

This resource uses AWSS3.jl to download files from Amazon servers. Thus, one needs to register an account with AWS first.

To install this package, download Julia. Within Julia, execute the following

using Pkg
Pkg.add(url="https://github.com/biona001/EasyLD.jl")

Requirements:

• A working local installation of Python package hail
• Java version 8 or 11, according to hail documentation.

We currently rely on the Hail python package to read the matrix and use the PyCall package to pass the result into Julia, but in the future, we will consider developing a native Julia parser to avoid the reliance on Python and Hail altogether.

Trouble shooting

• PyCall.jl cannot find hail and numpy even though I installed them? Julia's PyCall.jl by default installs its own local copy of python that is different than the system default. Thus, after installing PyCall.jl, within Julia I had to do ENV["PYTHON"] = "PATH OF PYTHON EXECUTABLE" and rebuild PyCall.jl via using Pkg; Pkg.build("PyCall"). This will point the python version in PyCall.jl to be the default python on your system.
• Intel MKL FATAL ERROR: Cannot load libmkl_intel_thread.dylib Try following suggestions in this post. In particular, the following worked for me:

# within Julia
using Conda
Conda.rm("mkl")
Conda.add("nomkl")
Conda.add("hail")

• OutOfMemoryError: Java heap space: Try increasing the java heap size by adding export _JAVA_OPTIONS="-Xmx24g" to your .bash_profile file. Here the -Xmx24g implies maximum heap size of 24 GB, which you are free to adjust.
• Can't find AWS credentials! This package internally downloads LD matrices from AWS, thus one needs to register an AWS account first and obtain AWS credentials. Usually, this means you should have a folder .aws in your home directory with the credential file.