fresh pbipa install fails to run

[splaisan]

Dear support,

I am trying to get pbipa to work on my server using fasta data derived from a SRA Sequel run SRR16576536
I freshly installed with conda from instructions on the pbipa page after reordering the repos as advised in a separate ticket and this time the install went well.

Thanks for your help

ipa validate works

INFO: /opt/miniconda3/envs/ipa/bin/ipa validate
INFO: ipa.py ipa (wrapper) version=1.5.0 ... Checking dependencies ...
INFO: Dependencies
/opt/miniconda3/envs/ipa/bin/python3
/opt/miniconda3/envs/ipa/bin/ipa2-task
/opt/miniconda3/envs/ipa/bin/falconc
/opt/miniconda3/envs/ipa/bin/minimap2
/opt/miniconda3/envs/ipa/bin/nighthawk
/opt/miniconda3/envs/ipa/bin/pancake
/opt/miniconda3/envs/ipa/bin/pblayout
/opt/miniconda3/envs/ipa/bin/racon
/opt/miniconda3/envs/ipa/bin/samtools
/opt/miniconda3/envs/ipa/bin/ipa_purge_dups
/opt/miniconda3/envs/ipa/bin/ipa_purge_dups_split_fa
snakemake version=6.3.0
ipa2-task 1.5.0 (commit c875fce13bdacbafc2f4f750c6438f4453e1354d)
 Machine name: 'Linux'
Copyright (C) 2004-2021     Pacific Biosciences of California, Inc.
This program comes with ABSOLUTELY NO WARRANTY; it is intended for
Research Use Only and not for use in diagnostic procedures.

falconc version=1.13.1+git.f9d1b5651e891efe379bd9727a0fa0931b875d7b, nim-version=1.5.1
minimap2 version=2.24-r1122
Nighthawk 0.1.0 (commit SL-release-10.1.0-7-gbe5dfb1*)
pancake 1.3.0 (commit SEQII-release-10.1.0-432-gf2693fd*)
pblayout 1.0.0 (commit SL-release-10.1.0-152-g66936d1*)
racon version=v1.4.20
samtools 1.15
Using htslib 1.14
ipa_purge_dups Version: 1.2.5

Operating system
ubuntu20 (all updates applied)

Package name
pbipa 1.5

Conda environment

# packages in environment at /opt/miniconda3/envs/ipa:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
amply                     0.1.4                      py_0    conda-forge
appdirs                   1.4.4              pyhd3eb1b0_0  
attrs                     21.4.0             pyhd3eb1b0_0  
boto                      2.49.0                   py37_0  
boto3                     1.20.24            pyhd3eb1b0_0  
botocore                  1.23.24            pyhd3eb1b0_0  
brotlipy                  0.7.0           py37h27cfd23_1003  
bzip2                     1.0.8                h7b6447c_0  
c-ares                    1.18.1               h7f8727e_0  
ca-certificates           2022.2.1             h06a4308_0  
certifi                   2021.10.8        py37h06a4308_2  
cffi                      1.15.0           py37hd667e15_1  
charset-normalizer        2.0.4              pyhd3eb1b0_0  
coincbc                   2.10.5               hab63836_1    conda-forge
configargparse            1.4                pyhd3eb1b0_0  
cryptography              36.0.0           py37h9ce1e76_0  
datrie                    0.8.2            py37h27cfd23_0  
docutils                  0.18.1           py37h06a4308_2  
filelock                  3.4.2              pyhd3eb1b0_0  
gitdb                     4.0.7              pyhd3eb1b0_0  
gitpython                 3.1.18             pyhd3eb1b0_1  
htslib                    1.14                 h9093b5e_0    bioconda
idna                      3.3                pyhd3eb1b0_0  
importlib-metadata        4.8.2            py37h06a4308_0  
importlib_metadata        4.8.2                hd3eb1b0_0  
ipython_genutils          0.2.0              pyhd3eb1b0_1  
jmespath                  0.10.0             pyhd3eb1b0_0  
jsonschema                3.2.0              pyhd3eb1b0_2  
jupyter_core              4.9.1            py37h06a4308_0  
k8                        0.2.5                h9a82719_1    bioconda
krb5                      1.19.2               hac12032_0  
ld_impl_linux-64          2.35.1               h7274673_9  
libblas                   3.9.0           11_linux64_openblas    conda-forge
libcblas                  3.9.0           11_linux64_openblas    conda-forge
libcurl                   7.80.0               h0b77cf5_0  
libdeflate                1.7                  h7f98852_5    conda-forge
libedit                   3.1.20210910         h7f8727e_0  
libev                     4.33                 h7f8727e_1  
libffi                    3.3                  he6710b0_2  
libgcc-ng                 11.2.0              h1d223b6_13    conda-forge
libgfortran-ng            7.5.0               ha8ba4b0_17  
libgfortran4              7.5.0               ha8ba4b0_17  
libgomp                   11.2.0              h1d223b6_13    conda-forge
liblapack                 3.9.0           11_linux64_openblas    conda-forge
libnghttp2                1.46.0               hce63b2e_0  
libopenblas               0.3.17               hf726d26_1  
libssh2                   1.9.0                h1ba5d50_1  
libstdcxx-ng              9.3.0               hd4cf53a_17  
libzlib                   1.2.11            h36c2ea0_1013    conda-forge
minimap2                  2.24                 h7132678_1    bioconda
nbformat                  5.1.3              pyhd3eb1b0_0  
ncurses                   6.3                  h7f8727e_2  
networkx                  2.6.3              pyhd3eb1b0_0  
openssl                   1.1.1m               h7f8727e_0  
pb-falconc                1.13.1               h6e72b04_0    bioconda
pbipa                     1.5.0                h259a17a_2    bioconda
pcre                      8.45                 h295c915_0  
pip                       21.2.2           py37h06a4308_0  
psutil                    5.8.0            py37h27cfd23_1  
pulp                      2.6.0            py37h89c1867_0    conda-forge
pycparser                 2.21               pyhd3eb1b0_0  
pyopenssl                 22.0.0             pyhd3eb1b0_0  
pyparsing                 3.0.4              pyhd3eb1b0_0  
pyrsistent                0.18.0           py37heee7806_0  
pysocks                   1.7.1                    py37_1  
python                    3.7.11               h12debd9_0  
python-dateutil           2.8.2              pyhd3eb1b0_0  
python_abi                3.7                     2_cp37m    conda-forge
pyyaml                    6.0              py37h7f8727e_1  
racon                     1.4.20               h9a82719_1    bioconda
ratelimiter               1.2.0                   py_1002    conda-forge
readline                  8.1.2                h7f8727e_1  
requests                  2.27.1             pyhd3eb1b0_0  
s3transfer                0.5.0              pyhd3eb1b0_0  
samtools                  1.15                 h3843a85_0    bioconda
setuptools                58.0.4           py37h06a4308_0  
six                       1.16.0             pyhd3eb1b0_1  
smart_open                1.8.0                    py37_0  
smmap                     4.0.0              pyhd3eb1b0_0  
snakemake-minimal         6.3.0              pyhdfd78af_0    bioconda
sqlite                    3.37.2               hc218d9a_0  
stopit                    1.1.2                      py_0    conda-forge
tk                        8.6.11               h1ccaba5_0  
toposort                  1.7                pyhd8ed1ab_0    conda-forge
traitlets                 5.1.1              pyhd3eb1b0_0  
typing-extensions         3.10.0.2             hd3eb1b0_0  
typing_extensions         3.10.0.2           pyh06a4308_0  
urllib3                   1.26.8             pyhd3eb1b0_0  
wheel                     0.37.1             pyhd3eb1b0_0  
wrapt                     1.13.3           py37h7f8727e_2  
xz                        5.2.5                h7b6447c_0  
yaml                      0.2.5                h7b6447c_0  
zipp                      3.7.0              pyhd3eb1b0_0  
zlib                      1.2.11            h36c2ea0_1013    conda-forge

Describe the bug
this simple run fails almost immediately

ipa local --nthreads 8 --njobs 1 -i SRR16576536.fa

Error message

INFO: /opt/miniconda3/envs/ipa/bin/ipa local --nthreads 8 --njobs 1 -i SRR16576536.fa
INFO: ipa.py ipa (wrapper) version=1.5.0 ... Checking dependencies ...
INFO: Dependencies
/opt/miniconda3/envs/ipa/bin/python3
/opt/miniconda3/envs/ipa/bin/ipa2-task
/opt/miniconda3/envs/ipa/bin/falconc
/opt/miniconda3/envs/ipa/bin/minimap2
/opt/miniconda3/envs/ipa/bin/nighthawk
/opt/miniconda3/envs/ipa/bin/pancake
/opt/miniconda3/envs/ipa/bin/pblayout
/opt/miniconda3/envs/ipa/bin/racon
/opt/miniconda3/envs/ipa/bin/samtools
/opt/miniconda3/envs/ipa/bin/ipa_purge_dups
/opt/miniconda3/envs/ipa/bin/ipa_purge_dups_split_fa
snakemake version=6.3.0
ipa2-task 1.5.0 (commit c875fce13bdacbafc2f4f750c6438f4453e1354d)
 Machine name: 'Linux'
Copyright (C) 2004-2021     Pacific Biosciences of California, Inc.
This program comes with ABSOLUTELY NO WARRANTY; it is intended for
Research Use Only and not for use in diagnostic procedures.

falconc version=1.13.1+git.f9d1b5651e891efe379bd9727a0fa0931b875d7b, nim-version=1.5.1
minimap2 version=2.24-r1122
Nighthawk 0.1.0 (commit SL-release-10.1.0-7-gbe5dfb1*)
pancake 1.3.0 (commit SEQII-release-10.1.0-432-gf2693fd*)
pblayout 1.0.0 (commit SL-release-10.1.0-152-g66936d1*)
racon version=v1.4.20
samtools 1.15
Using htslib 1.14
ipa_purge_dups Version: 1.2.5


To run this yourself:
/opt/miniconda3/envs/ipa/bin/python3 -m snakemake -j 1 -d RUN -p -s /opt/miniconda3/envs/ipa/etc/ipa.snakefile --configfile RUN/config.yaml --reason

Starting snakemake ...
Traceback (most recent call last):
  File "/opt/miniconda3/envs/ipa/lib/python3.7/site-packages/snakemake/__init__.py", line 594, in snakemake
    snakefile, overwrite_first_rule=True, print_compilation=print_compilation
  File "/opt/miniconda3/envs/ipa/lib/python3.7/site-packages/snakemake/workflow.py", line 1094, in include
    if is_local_file(snakefile):
  File "/opt/miniconda3/envs/ipa/lib/python3.7/site-packages/snakemake/common.py", line 65, in is_local_file
    from smart_open import parse_uri
ImportError: cannot import name 'parse_uri' from 'smart_open' (/opt/miniconda3/envs/ipa/lib/python3.7/site-packages/smart_open/__init__.py)
ERROR: Command '/opt/miniconda3/envs/ipa/bin/python3 -m snakemake -j 1 -d RUN -p -s /opt/miniconda3/envs/ipa/etc/ipa.snakefile --configfile RUN/config.yaml --reason' returned non-zero exit status 1.
Exiting.

[isovic]

Hi @splaisan,

This looks like an issue with Snakemake.

I found this issue with a similar problem:
nanoporetech/pipeline-structural-variation#18

The solution suggested in this thread is to downgrade the smart_open package with:

conda install smart_open=3

Does this work?

Best regards,
Ivan.

[isovic]

Alternatively, try upgrading Snakemake (and perhaps also smart_open).

[splaisan]

HI @isovic, Thanks a lot for your help
I installed conda install smart_open=3 and it now executes (if it dies later onI will edit this note).
The installed snakemake (6.3.0) seems to work now (busy computing)
I had already done this in my previous ipa attempt but had forgotten it, it may be wise to add this info to the ipa install page or modify the conda build accordingly.

[isovic]

That's great, thanks for the feedback!
This is a good suggestion, I'll add a note there.
Best regards,
Ivan.

[isovic]

According to your suggestion I edited the README page here:
https://github.com/PacificBiosciences/pbipa/blob/master/README.md

Thanks again!
Ivan