Conda package

[sebastian-luna-valero]

Hello,

Have you considered creating a conda package for ngsplot?

I am happy to add it to bioconda. Would you be able to assist if I find any issues in the process?

Best regards,
Sebastian

CC @Acribbs @jencyw

[apeltzer]

I'm currently looking at this too, but it seems to me to be not that straightforward:

• Packaging the ngs.plot application alone is probably not too difficult
• Packaging the databases is, as they are only on GDrive and not on a versioned page

If the authors would be willing to push things to bioconductor, we could simply rely on that and get the packages from there. However, I don't see that happening right now...

Cross-ref here nf-core/chipseq#15

[ny-shao]

Hi Sebastian and Alex,

Both Li and I are interested in it. If you need help just let us know.
I don't think we need to integrate the database with the default installation, as the other utilities like HOMER don't ship their database with the database in bioconda.

Ning-Yi

[apeltzer]

I talked with bioconda developers and there is a way to achieve things, if you'd be willing to e.g. host the database with a stable release information / repository. Google Drive is fine for just distributing, but its not a stable release URL (since you can overwrite things). If you could e.g. add some kind of versioning for the databases, I guess we could be using them also from there - that would be something you could help us with.

We can then produce separate packages for the databases that pull the required databases on user request and have the main application in the "main" package.

E.g. adding something like "2018-06-04_hg19.db" or something similar and not overwriting that file (or just produce something like "hg19_v1.0.db" and later add "hg19_v1.1.db" - this would allow us to have a direct versionizing schema.

[apeltzer]

PR accepted this is available via bioconda now. I will work on submitting a couple of packages for the Databases now!

[sebastian-luna-valero]

Thank you very much @apeltzer

[LokiLuciferase]

Hi everyone,

I am encountering the following error when attempting to install ngsplot via bioconda:

conda create -p ./test_ngsplot python=2.7 bioconda::r-ngsplot=2.63 bioconda::r-ngsplotdb-hg19=3.00

[...]

Preparing transaction: done
Verifying transaction: done
Executing transaction: \ Extracting information from package... contains 111 tables.
Will install new genome hg19: Ensembl=> v75.0; ngs.plot=> v3.0. Continue?(y/n): Traceback (most recent call last):
  File "/scratch/lukas.lueftinger/conda_envs/test_ngsplot/bin/ngsplotdb.py", line 669, in <module>
    args.func(root_path, args)
  File "/scratch/lukas.lueftinger/conda_envs/test_ngsplot/bin/ngsplotdb.py", line 376, in install
    ans = raw_input("Continue?(y/n): ")
EOFError: EOF when reading a line

Conda environment creation concludes with exitcode 0, however the relevant ngsplot database is not installed.

The error occurs in all environments I tried up until now, including:

• CentOS 7.4.1708 + conda 4.4.10
• CentOS 7.5.1804 + conda 4.5.1
• a singularity-hub build based on https://hub.docker.com/r/continuumio/miniconda/

best regards,
Lukas

[apeltzer]

Thanks for the report, we found something similar to be the case and I'll work on it the next days to get this fixed asap.

[apeltzer]

Cross linking our issue here nf-core/chipseq#43

[apeltzer]

Looks for me as if the installation requires interaction with the installation script (a confirmation via -yswitch). We didn't configure this in the build script until now, so I will do that now and then the installation should work properly. Weird however, that this does not cause any error so far.

[apeltzer]

echo "y" | ngsplotdb.py install packageshould fix the confirmation issue for now. However, the installation does not seem to work either unfortunately.

echo y | ngsplotdb.py install ~/Desktop/ngsplotdb_hg19_75_3.00.tar.gz
Extracting information from package... contains 111 tables.
Will install new genome hg19: Ensembl=> v75.0; ngs.plot=> v3.0. Continue?(y/n): Installing new genome...Traceback (most recent call last):
  File "/home/alex/.conda/envs/ngsplotdb/bin/setTableDefaults.py", line 36, in ?
    in_f = file(sys.argv[1])
NameError: file
Error in read.table(default.tbl, header = TRUE, sep = "\t", blank.lines.skip = TRUE,  : 
  no lines available in input
Execution halted
 Done

[LokiLuciferase]

Thanks for the quick response!

I personally have worked around the problem by hosting the database files in my own github repo, and, after environment building, activating the environment and installing the files manually using the following, (and passing the -y flag directly to ngsplotdb.py):

$ ngsplotdb.py -y install data/ngsplotdb_hg19_75_3.00.tar.gz 
Extracting information from package... contains 111 tables.
Will install new genome hg19: Ensembl=> v75.0; ngs.plot=> v3.0. Installing new genome...Using library: /scratch/lukas.lueftinger/conda_envs/test_ngsplot/lib/R/library
 Done

which seems to work:

$ ngsplotdb.py list
ID   Assembly Species      EnsVer   NPVer    InstalledFeatures            
hg19 GRCh37   homo_sapiens 75.0     3.0      cgi,exon,genebody,tss,tes

best,
Lukas

[apeltzer]

Thanks, didn't know that this switch even existed as its not documented by the authors.
I added that to the recipe and that should run through CircleCI testing right now. Lets hope that works :-)

[Runuply]

when I tried to install the reference, I still encountered the error.

$ ngsplotdb.py -y install ngsplotdb_mm10_75_3.00.tar 
Read package file ngsplotdb_mm10_75_3.00.tar error. The downloaded file may be corrupted.

[lishuai800]

I tried several times with the previous solutions but still could not fix it.
version 1:
ngsplotdb.py -y install ~/Downloads/software_install_package/ngsplot_data/ngsplotdb_Tair10_21_3.00.tar.gz
Extracting information from package... contains 27 tables.
Will install new genome Tair10: Ensembl=> v21.0; ngs.plot=> v3.0. Installing new genome...
Traceback (most recent call last):
File "/home/shuai/bin/ngsplot/bin/ngsplotdb.py", line 669, in
args.func(root_path, args)
File "/home/shuai/bin/ngsplot/bin/ngsplotdb.py", line 372, in install
install_pkg(root_path, pkg_file, gn_inst)
File "/home/shuai/bin/ngsplot/bin/ngsplotdb.py", line 426, in install_pkg
add_dbtbl(root_path, new_gn)
File "/home/shuai/bin/ngsplot/bin/ngsplotdb.py", line 479, in add_dbtbl
subprocess.call(["setTableDefaults.py", gnlist_n, default_n])
File "/usr/lib/python2.7/subprocess.py", line 172, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib/python2.7/subprocess.py", line 394, in init
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

version2:
echo "y" | ngsplotdb.py install ~/Downloads/software_install_package/ngsplot_data/ngsplotdb_Tair10_21_3.00.tar.gz
Extracting information from package... contains 27 tables.
Will install new genome Tair10: Ensembl=> v21.0; ngs.plot=> v3.0. Continue?(y/n): Installing new genome...
Traceback (most recent call last):
File "/home/shuai/bin/ngsplot/bin/ngsplotdb.py", line 669, in
args.func(root_path, args)
File "/home/shuai/bin/ngsplot/bin/ngsplotdb.py", line 384, in install
install_pkg(root_path, pkg_file, gn_inst)
File "/home/shuai/bin/ngsplot/bin/ngsplotdb.py", line 426, in install_pkg
add_dbtbl(root_path, new_gn)
File "/home/shuai/bin/ngsplot/bin/ngsplotdb.py", line 479, in add_dbtbl
subprocess.call(["setTableDefaults.py", gnlist_n, default_n])
File "/usr/lib/python2.7/subprocess.py", line 172, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib/python2.7/subprocess.py", line 394, in init
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

[lishuai800]

I tried several times with the previous solutions but still could not fix it.
version 1:
ngsplotdb.py -y install ~/Downloads/software_install_package/ngsplot_data/ngsplotdb_Tair10_21_3.00.tar.gz
Extracting information from package... contains 27 tables.
Will install new genome Tair10: Ensembl=> v21.0; ngs.plot=> v3.0. Installing new genome...
Traceback (most recent call last):
File "/home/shuai/bin/ngsplot/bin/ngsplotdb.py", line 669, in
args.func(root_path, args)
File "/home/shuai/bin/ngsplot/bin/ngsplotdb.py", line 372, in install
install_pkg(root_path, pkg_file, gn_inst)
File "/home/shuai/bin/ngsplot/bin/ngsplotdb.py", line 426, in install_pkg
add_dbtbl(root_path, new_gn)
File "/home/shuai/bin/ngsplot/bin/ngsplotdb.py", line 479, in add_dbtbl
subprocess.call(["setTableDefaults.py", gnlist_n, default_n])
File "/usr/lib/python2.7/subprocess.py", line 172, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib/python2.7/subprocess.py", line 394, in init
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

version2:
echo "y" | ngsplotdb.py install ~/Downloads/software_install_package/ngsplot_data/ngsplotdb_Tair10_21_3.00.tar.gz
Extracting information from package... contains 27 tables.
Will install new genome Tair10: Ensembl=> v21.0; ngs.plot=> v3.0. Continue?(y/n): Installing new genome...
Traceback (most recent call last):
File "/home/shuai/bin/ngsplot/bin/ngsplotdb.py", line 669, in
args.func(root_path, args)
File "/home/shuai/bin/ngsplot/bin/ngsplotdb.py", line 384, in install
install_pkg(root_path, pkg_file, gn_inst)
File "/home/shuai/bin/ngsplot/bin/ngsplotdb.py", line 426, in install_pkg
add_dbtbl(root_path, new_gn)
File "/home/shuai/bin/ngsplot/bin/ngsplotdb.py", line 479, in add_dbtbl
subprocess.call(["setTableDefaults.py", gnlist_n, default_n])
File "/usr/lib/python2.7/subprocess.py", line 172, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib/python2.7/subprocess.py", line 394, in init
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

I figured it out that it was the reason that I am used a conda py27 env. and downloaded and extracted the tar.gz file as introduced on the website. there might be some conflicts. However, when I used conda to install the r-ngsplot and now it works well for the install genome step.

[ny-shao]

Hi Runyply, Could you try to run the ngsplotdb.py by python2.7? It was written in py2.7, but the first line of the script will call the default python interpreter in the $PATH. So it looks like the issue of which python you are using. Ning-Yi Shao

…

On Mon, Dec 16, 2019 at 3:50 PM Runuply ***@***.***> wrote: when I tried to install the reference, I still encountered the error. $ ngsplotdb.py -y install ngsplotdb_mm10_75_3.00.tar Read package file ngsplotdb_mm10_75_3.00.tar error. The downloaded file may be corrupted. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#74?email_source=notifications&email_token=AAIKUKISGJOBMHN3GTCQVK3QY7SZ5A5CNFSM4D5VI5C2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHABQTY#issuecomment-566237263>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIKUKMA5LBEMBWTFWRTLCLQY7SZ5ANCNFSM4D5VI5CQ> .

[poflawless]

Have you solved the issue? I encountered the same error for install TAIR10 even if using the python2.7, Could I know how to fix it?