segfault (possibly due to BNPY)

[sahilseth]

Hi,

I am trying to test hatchet, however after installation of gurobi and bnpy-dev, the first step fails with a segmentation fault. I dont see a core dump being created in the pwd.

demo-WES]$ ${CLUBB} demo-wes.bb -by ${BNPY} -o demo-wes.seg -O demo-wes.bbc -e 12 -tB 0.03 -tR 0.15 -d 0.08
[2019-Apr-23 14:02:41]# Parsing and checking input arguments
[2019-Apr-23 14:02:41]# Reading the combined BB file
[2019-Apr-23 14:02:41]# Format data to cluster
[2019-Apr-23 14:02:41]# Clustering bins by RD and BAF across tumor samples
[2019-Apr-23 14:02:41]## Loading BNPY
Segmentation fault (core dumped)

It fails in a similar fashion with WGS demo as well.

I have bnpy-dev from bitbucket, and gurobi 811.

Thanks!

[simozacca]

The error may be associated either to BNPY installation or to the runtime creation of the temporary directory used by BNPY. Please answer and check the following debugging list to try to identify the issue:

• Is your version of bnpy-dev the most updated in the main branch of this repository https://bitbucket.org/michaelchughes/bnpy-dev/src/master/?
• How did you install BNPY? Did you install any of the suggested alternatives described here: https://bitbucket.org/michaelchughes/bnpy-dev/wiki/Installation.md? Also, what is your version of the required numpy and scipy in the current python environment?
• Could you please try to remove ANY FOLDER from the current directory where you are executing the demo? Especially, please make sure to remove _TMPDIR/
• Do you have permission to create and remove directories in you current running directory?
• Could you please report the value of the variable ${BNPY} that you appropriately defined in the script demo-WES.sh?

Thank you

[sahilseth]

Thanks for getting back!

It seems to be the most up-to-date

cd /home/sseth/apps/bnpy/github_bnpy-dev
origin  https://michaelchughes@bitbucket.org/michaelchughes/bnpy-dev (fetch)
git pull
Already up-to-date.
git branch
* master

I did not install the Fast C++ libraries for HMM inference

numpy                     1.15.1           py27h1d66e8a_0
scipy                     1.1.0            py27hfa4b5c9_1

I do have write permissions.

BNPY="/home/sseth/apps/bnpy/github_bnpy-dev"

I removed the TMPDIR, and reran, with the same error.

Thanks!

[simozacca]

According to the BNPY documentation (https://bitbucket.org/michaelchughes/bnpy-dev/wiki/Installation.md), BNPY requires numpy (version 1.8 or later). As such, we should first try to get the required version of Numpy. If you do not want or cannot change the packages of your current python environment, I recommend to use a virtual python environment, inside which you can install all the required packages with appropriate versions without affecting the default python enviroment. You will simply activate you virtual environment before running any component of HATCHet. There are two options:

1. virtualenv, the simplest option (read more at https://virtualenv.pypa.io/en/latest/ or https://docs.python-guide.org/dev/virtualenvs/)
2. Anaconda, the cleanest option (read more at https://uoa-eresearch.github.io/eresearch-cookbook/recipe/2014/11/20/conda/)

Moreover, as this appears to be a BNPY-specific issue I would suggest the following two things to do:

1. Run the few commands suggested at https://bitbucket.org/michaelchughes/bnpy-dev/wiki/Installation.md to make sure everything works
2. Try to import BNPY in your python environment to make sure it works properly. You can perform this test by using the same python commands used by cluBB component, i.e.

import os, sys
bnpydir = "/home/sseth/apps/bnpy/github_bnpy-dev/"

tmp = os.path.splitext(output)[0] + "_TMPDIR/"
if os.path.exists(tmp):
    shutil.rmtree(tmp)
os.makedirs(tmp)
os.environ["BNPYOUTDIR"] = tmp
sys.path.append(bnpydir)
import bnpy

[sahilseth]

thanks for the pointers. Indeed it was an issue with the version of libs. It works well on a new env, with updated libs.

Thanks!

[sahilseth]

I was about to raise an issue with bnpy-dev on bitbucket, but realize they have moved to github. Would hatchet move to use the new bnpy version instead?

[simozacca]

Yes, HATCHet is going to support both the versions, both the old one in bitbucket (bnpy-dev) and the new one (bnpy) maintained on github.
We are testing to make sure that new version of BNPY is working as expected (we deeply tested the old one), and we will add this feature very soon.

For now I would recommend to keep using the bitbucket version, which is fully compatible.
As such, I close this issue for now but please feel free to re-open it in case of need.