Added recipe for SonicParanoid 1.0.13 #11271

salvoc81 · 2018-10-09T13:18:11Z

I have read the guidelines for bioconda recipes.
This PR adds a new recipe.
AFAIK, this recipe is directly relevant to the biological sciences (otherwise, please submit to the more general purpose conda-forge channel).
This PR updates an existing recipe.
This PR does something else (explain below).

salvoc81 · 2018-10-10T10:34:27Z

I am not sure what is causing the problem. If I remove the compiler() function it builds correctly on both linux and osx using the default conda-build coming with anaconda.
It would be very appreciated if somebody could give some advice.

npavlovikj · 2018-10-10T17:11:30Z

@salvoc81 , your syntax for the compiler is wrong, and therefore the error. For C compiler you should use {{ compiler('c') }} and for C++, {{ compiler('cxx') }}. For more information on the compilers, please look at https://conda.io/docs/user-guide/tasks/build-packages/compiler-tools.html.

npavlovikj · 2018-10-11T00:22:17Z

@salvoc81 , as you can see, adding "cxx" fixed your initial error, but you are getting another one now, "conda_build.exceptions.CondaBuildException: Empty package".

Considering this, there are few issues with your recipe.

You need a build section, https://conda-forge.org/docs/meta.html.
Only the compiler needs to be in the build section, and the rest of the build section should go in host, including pip.
What are you testing is an executable named "sonicparanoid". Is there this type of executable, or there are only python scripts? If the latter, you should modify the test section accordingly.
Looks like there is a "LICENSE.md" file in your download, so you should add that in the license section.
If the sub-folder "sonicparanoid" has python scripts that are not installed, you should copy them the ${PREFIX}/bin.

You can use this recipe for start, https://github.com/bioconda/bioconda-recipes/blob/bc656c004ca3399595959403ca1870e83ac1e50b/recipes/rseqc/meta.yaml, and you can always search the repo for other examples as well.
I don't have time to test the recipe now, but I hope this helps you get going.
Also, it is a good idea to test the changes locally first, https://bioconda.github.io/contribute-a-recipe.html#test-locally, instead of pushing them directly to the repo.

…n; added LICENSE.md

salvoc81 · 2018-10-11T02:06:31Z

Thanks a lot for your help @npavlovikj . The build under Linux seems successful although now it is failing in mulled-build saying that there is no gcc installed in the system

Regarding your comments:

1. You need a build section, https://conda-forge.org/docs/meta.html.

Do you mean requirements: build: or the build: section before requirements?
Ah, sorry about that - I meant the build section before requirements with the build number, e.g.:
build:
number: 0

3. What are you testing is an executable named "sonicparanoid". Is there this type of executable, or there are only python scripts? If the latter, you should modify the test section accordingly.

Yes, the build is supposed to generate 4 executables, one of which is named "sonicparanoid"
In that case, you should check for all 4 executables, e.g.:
test:
commands:
- exec1
- exec2
- exec3
- exec4

4. Looks like there is a "LICENSE.md" file in your download, so you should add that in the license section.

Done correctly, I hope
Yep, looks right :)

Also, it is a good idea to test the changes locally first, https://bioconda.github.io/contribute-a-recipe.html#test-locally, instead of pushing them directly to the repo.
I have been doing the test locally on OSX machines but unluckily the outcome of the tests is different from the one I get when pushing to the repo.
Also when I try to run bootstrap.py it fails with the following error after the installation of miniconda:


installation finished.
Warning: 'defaults' already in 'channels' list, moving to the top

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

`$ /tmp/miniconda/miniconda/bin/conda install -y git pip --file https://raw.githubusercontent.com/bioconda/bioconda-utils/cb3-migration/bioconda_utils/bioconda_utils-requirements.txt`

  environment variables:
                 CIO_TEST=<not set>
               CONDA_ROOT=/tmp/miniconda/miniconda
                     PATH=/tmp/miniconda/miniconda/bin:/Users/salvocos/anaconda3/bin:/usr/local/
                          opt/sphinx-doc/bin:/Users/salvocos/.cargo/bin:/usr/local/bin:/usr/bin:
                          /bin:/opt/X11/bin:/sbin:/usr/sbin:/usr/local/sbin:/Users/salvocos/Libr
                          ary/Android/sdk/platform-tools:/Library/TeX/Distributions/Programs/tex
                          bin:/Users/salvocos/.cargo/bin
       REQUESTS_CA_BUNDLE=<not set>
            SSL_CERT_FILE=<not set>

     active environment : None
       user config file : /Users/salvocos/.condarc
 populated config files : /tmp/miniconda/miniconda/.condarc
          conda version : 4.5.4
    conda-build version : not installed
         python version : 3.6.5.final.0
       base environment : /tmp/miniconda/miniconda  (writable)
           channel URLs : https://conda.anaconda.org/conda-forge/osx-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://conda.anaconda.org/bioconda/osx-64
                          https://conda.anaconda.org/bioconda/noarch
                          https://repo.anaconda.com/pkgs/main/osx-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/osx-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/osx-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/pro/osx-64
                          https://repo.anaconda.com/pkgs/pro/noarch
          package cache : /tmp/miniconda/miniconda/pkgs
                          /Users/salvocos/.conda/pkgs
       envs directories : /tmp/miniconda/miniconda/envs
                          /Users/salvocos/.conda/envs
               platform : osx-64
             user-agent : conda/4.5.4 requests/2.18.4 CPython/3.6.5 Darwin/17.7.0 OSX/10.13.6
                UID:GID : 501:20
             netrc file : None
           offline mode : False


V V V V V V V V V V V V V V V V V V V V V V V V V V V V V V V

CondaHTTPError: HTTP 404 NOT FOUND for url <https://raw.githubusercontent.com/bioconda/bioconda-utils/cb3-migration/bioconda_utils/bioconda_utils-requirements.txt>
Elapsed: 00:00.217378

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.

A reportable application error has occurred. Conda has prepared the above report.
Upload successful.
Traceback (most recent call last):
  File "./bootstrap.py", line 110, in <module>
    sp.check_call(['.circleci/setup.sh'], env=env)
  File "/Users/salvocos/anaconda3/lib/python3.6/subprocess.py", line 291, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['.circleci/setup.sh']' returned non-zero exit status 1.

npavlovikj · 2018-10-11T02:51:39Z

@salvoc81 , if you look at the Linux and OSX logs, you will see that nothing was built:
02:42:31 BIOCONDA INFO Nothing to be done for recipe recipes/sonicparanoid
02:42:31 BIOCONDA INFO BUILD SUMMARY: successfully built 0 of 1 recipes

The build number should be 0 since there is not package available for "SonicParanoid" yet. If there is a package publicly available and you want to modify the current version, then you increment the build number to 1, etc. If your are building new version of "SonicParanoid", then you start from build number 0 again. Can you please set the build number to 0 and try again?

salvoc81 · 2018-10-11T02:54:09Z

The build number should be 0 since there is no package available for "SonicParanoid" yet. If there is a package publicly available and you want to modify the current version, then you increment the build number to 1, etc. If your are building new version of "SonicParanoid", then you start from build number 0 again. Can you please set the build number to 0 and try again?

I see, that is way it was also working locally :(
Thank you

npavlovikj · 2018-10-11T03:00:03Z

@salvoc81 , when I test the updated recipe locally, I am getting the following errors:

File "/opt/conda/conda-bld/sonicparanoid_1539226107544/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.5/site-packages/sonicparanoid/process_output.py", line 47
annotate: bool = False
^
SyntaxError: invalid syntax

File "/opt/conda/conda-bld/sonicparanoid_1539226107544/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.5/site-packages/sonicparanoid/ortholog_detection.py", line 3394
segOverlapCutoff: float = 0.25
^
SyntaxError: invalid syntax

File "/opt/conda/conda-bld/sonicparanoid_1539226107544/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.5/site-packages/sonicparanoid/mmseqs_parser_cython.py", line 79
bsName: str = os.path.basename(inputPath)
^
SyntaxError: invalid syntax

File "/opt/conda/conda-bld/sonicparanoid_1539226107544/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.5/site-packages/sonicparanoid/sonic_paranoid_extract.py", line 29
sonicparanoid_usage: str = '\nProvide a table with ortholog groups generated using SonicParanoid.\n'
^
SyntaxError: invalid syntax

File "lib/python3.5/site-packages/sonicparanoid/mmseqs_parser_cython.py", line 79
bsName: str = os.path.basename(inputPath)
^
SyntaxError: invalid syntax

File "lib/python3.5/site-packages/sonicparanoid/process_output.py", line 47
annotate: bool = False
^
SyntaxError: invalid syntax

File "lib/python3.5/site-packages/sonicparanoid/ortholog_detection.py", line 3394
segOverlapCutoff: float = 0.25
^
SyntaxError: invalid syntax

File "lib/python3.5/site-packages/sonicparanoid/sonic_paranoid_extract.py", line 29
sonicparanoid_usage: str = '\nProvide a table with ortholog groups generated using SonicParanoid.\n'
^
SyntaxError: invalid syntax

Do you have any idea why?

@npavlovikj
It seems like the errors are related to static typing although these should supported from py3.5
https://docs.python.org/3/library/typing.html
Did you try to install the 'mypy' module? Although I am not sure that would fix...
Maybe you could try a with more recent version of python.

…ion; added test for all executables.

salvoc81 · 2018-10-11T05:42:54Z

Hello @npavlovikj . Basically now it fails during the tests.
This is because 'sonicparanoid' the first time it is executed executes a make file to compile the c and c++ source files for another program it uses. The error, as you can see from the log says make: g++: Command not found.
Because I cannot use the compiler(cxx) inside the run section as per linter rules, how do you think I could fix this problem?

Thanks a lot for all your help!

npavlovikj · 2018-10-11T05:56:52Z

@salvoc81 , ideally, there should be a conda package for every dependency required. That way, you don't build the dependencies within the package, but add the appropriate conda package in the "run" section. Is "mmseqs2" the program you are talking about? If that is the case, there is already an existing conda package, https://anaconda.org/bioconda/mmseqs2, you can add in the "run" section.

EDIT: Ah, I see the new package is "quick_multi_paranoid". I would say go ahead and create a different recipe for it. If it is part of the "sonicparanoid" folder, you can use the same "url" and "sha256", just build only "qp" and "qa".

@npavlovikj 👍 Right, that is what I was about to write...
The problem is that the program builds specific C programs for each run (a bit nerdy I know...), hence gcc would still be required to execute.
It would be great if it would be possible to have gcc/g++ for the execution.

EDIT: I have checked again and it might be possible to solve it writing another formula for it.
Will try to do this next Monday,
Thanks a lot for all your help.

salvoc81 · 2018-10-18T00:30:28Z

Hello @npavlovikj and thanks a lot for your help. In the I just solved the problem by adding a class that builds the source code inside the setup.py.

npavlovikj · 2018-10-18T04:04:23Z

I am glad to help @salvoc81 , and I am glad you made the recipe work :)

salvoc81 added 3 commits October 9, 2018 22:15

Added recipe for SonicParanoid 1.0.13

fb35974

removed 'x.x' for numpy, added basic test

a96c932

fixed linter related errors

95c8156

salvoc81 added help wanted GCC problems labels Oct 10, 2018

salvoc81 added 2 commits October 11, 2018 08:28

compiler set to cxx and added host section (same build error!)

1c882bf

removed host section

154467c

salvoc81 added 2 commits October 11, 2018 09:41

moved numpy and cython to host section; restored build: noarch: pytho…

72b287e

…n; added LICENSE.md

removed build: noarch

a348004

added build and extra sections

d2ddb11

salvoc81 added 3 commits October 11, 2018 12:21

set build number to 0; removed skip build; added python3 minimum vers…

7e5cc7c

…ion; added test for all executables.

restored sh python module as requirement

fb6fb9a

added Make to run; updated release to latest commit

ad4563c

salvoc81 added 5 commits October 17, 2018 14:55

remove c compiler from meta.yaml

8c91c9a

modified build.sh to compile source codel added make in build section

f71f22f

updated to latest commit

2d2ad68

updated to latest version with make included in setup.py

2d3f689

reverted build.sh to the simplest version (python setup.py install)

29f79a8

salvoc81 merged commit 942da89 into master Oct 18, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added recipe for SonicParanoid 1.0.13 #11271

Added recipe for SonicParanoid 1.0.13 #11271

salvoc81 commented Oct 9, 2018

salvoc81 commented Oct 10, 2018

npavlovikj commented Oct 10, 2018

npavlovikj commented Oct 11, 2018 •

edited

salvoc81 commented Oct 11, 2018 •

edited by npavlovikj

npavlovikj commented Oct 11, 2018 •

edited

salvoc81 commented Oct 11, 2018 •

edited by npavlovikj

npavlovikj commented Oct 11, 2018 •

edited by salvoc81

salvoc81 commented Oct 11, 2018

npavlovikj commented Oct 11, 2018 •

edited by salvoc81

salvoc81 commented Oct 18, 2018

npavlovikj commented Oct 18, 2018

Added recipe for SonicParanoid 1.0.13 #11271

Added recipe for SonicParanoid 1.0.13 #11271

Conversation

salvoc81 commented Oct 9, 2018

salvoc81 commented Oct 10, 2018

npavlovikj commented Oct 10, 2018

npavlovikj commented Oct 11, 2018 • edited

salvoc81 commented Oct 11, 2018 • edited by npavlovikj

npavlovikj commented Oct 11, 2018 • edited

salvoc81 commented Oct 11, 2018 • edited by npavlovikj

npavlovikj commented Oct 11, 2018 • edited by salvoc81

salvoc81 commented Oct 11, 2018

npavlovikj commented Oct 11, 2018 • edited by salvoc81

salvoc81 commented Oct 18, 2018

npavlovikj commented Oct 18, 2018

npavlovikj commented Oct 11, 2018 •

edited

salvoc81 commented Oct 11, 2018 •

edited by npavlovikj

npavlovikj commented Oct 11, 2018 •

edited

salvoc81 commented Oct 11, 2018 •

edited by npavlovikj

npavlovikj commented Oct 11, 2018 •

edited by salvoc81

npavlovikj commented Oct 11, 2018 •

edited by salvoc81