conda builds not working anymore

[jpfeuffer]

https://abibuilder.cs.uni-tuebingen.de/jenkins/job/openms/job/ntly/job/conda/475/console

no idea why.

[jpfeuffer]

Issue nr2: our builds do not show the correct outcome

Mamba failed to solve:
 - pandas
 - boost-cpp 1.74.0.*
 - python 3.7.* *_cpython
 - numpy 1.17.*
 - autowrap >=0.22.10
 - qt 5.12.*
 - wheel
 - libgcc-ng >=12
 - cython >=0.25.2
 - xerces-c 3.2.*
 - pip
 - libstdcxx-ng >=12
 - libsvm 3.21.*
 - setuptools
 - libopenms 3.0.0dev

with channels:

The reported errors are:
- Encountered problems while solving:
-   - nothing provides requested libopenms 3.0.0dev
- 
Returning non-final recipe for pyopenms-3.0.0dev-py37_0; one or more dependencies was unsatisfiable:
19:01:25 CONDA_BUILD WARNING Returning non-final recipe for pyopenms-3.0.0dev-py37_0; one or more dependencies was unsatisfiable:
Reloading output folder: /home/builder/mambaforge/envs/bioconda/conda-bld
Transaction

[poshul]

Where do you see those errors? I see https://abibuilder.cs.uni-tuebingen.de/jenkins/job/openms/job/ntly/job/conda/477/ looking successful.

[timosachsenberg]

its in the console output

[timosachsenberg]

fixed with:
OpenMS/bioconda-recipes#4

[jpfeuffer]

TODO:

• rewrite Jenkins script to save the exit code of the builds and report accordingly.
• build more python versions

[timosachsenberg]

btw I asked Björn and he said that he thinks you need to specify python versions you want to build... there is no shortcut for building a range of python verison

[jpfeuffer]

Can you check if you can read that from somewhere in the repo. Due to issues with the nightly wheels and different versions on Ubuntu,macOS,Win, I would like to fix a supported version file somewhere

[jpfeuffer]

For the time being add 3.9, 3.10, 3.11

[jpfeuffer]

Reduced to "major". It works now but the build statuses are still wrong.

[jpfeuffer]

Weird, tells me sth different:
openms-meta built: 1
pyopenms built: 1
upload: 0

[jpfeuffer]

yes, as discussed because bioconda build is bugged

[jpfeuffer]

06:55:31 BIOCONDA ERROR BUILD FAILED: the built package /home/builder/mambaforge/envs/bioconda/conda-bld/linux-64/pyopenms-3.0.0dev-py310h4b47abe_0.tar.bz2 cannot be found
it looks in the wrong folder

[timosachsenberg]

but it seems to upload it???

[jpfeuffer]

yes because it's bugged. bioconda build does not do the upload.

[jpfeuffer]

Someone NEEDS to file a bug report.

[jpfeuffer]

Or you parse the logs and ignore this specific error.

[bernt-matthias]

Wild guess: conda's has a new package format? Packages now end with .conda.

[jpfeuffer]

Hmm good guess, but it failed since years. It never worked to build inside a container when specifying you own repository.

Your guess will probably lead to additional trouble since we did not know about it.

[poshul]

Timo has submitted a bug report. In the meantime I just created a soft-link between the directory that conda expects it to be in, and where it actually is. This resolves the issue for now.

[jpfeuffer]

This actually broke the builds completely. Seems like the env needs to be empty and due to the soft link, it is not anymore.

[poshul]

Okay, I've rolled that back. With Timo's update the Job should pass now even if the upload fails. Is that sufficient as a stopgap until we get a response from the bioconda folks?

[jpfeuffer]

But the update from Timo now only checks upload. And upload will always work because there is an artifact from a previous build. So we don't see any failures ever. There must be another way:

1. Parse logs
2. make upload fail if build (really) fails (e.g. by checking artifact date or resetting build folder [be careful with that, conda is very picky about folders used as environments])

[poshul]

bioconda/bioconda-utils#838

[poshul]

Still need to get bioconda/bioconda-utils#838 merged