{bio}[foss/2023a] CellRank v2.0.2

[pavelToman]

(created using eb --new-pr)

• depends on {bio}[foss/2023a] scVelo v0.3.1, loompy v3.0.7 #20002
• waiting for Update loompy builddep - use poetry 1.5.1 to not have a conflict with other modules #20148

[pavelToman]

@boegelbot please test @ generoso

[boegelbot]

@pavelToman: Request for testing this PR well received on login1

PR test command 'EB_PR=20115 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs /opt/software/slurm/bin/sbatch --job-name test_PR_20115 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

• exit code: 0
• output:

Submitted batch job 13115

Test results coming soon (I hope)...

- notification for comment with ID 1999236699 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

[boegelbot]

Test report by @boegelbot
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
cns1 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/boegelbot/23415f5ee9ed5c29da3dcb79106e4c41 for a full test report.

[pavelToman]

@boegelbot please test @ jsc-zen3

[boegelbot]

@pavelToman: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=20115 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_20115 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

• exit code: 0
• output:

Submitted batch job 3782

Test results coming soon (I hope)...

- notification for comment with ID 1999258677 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

[verdurin]

Test report by @verdurin
FAILED
Build succeeded for 6 out of 7 (1 easyconfigs in total)
easybuild-c7.novalocal - Linux CentOS Linux 7.9.2009, x86_64, Intel Xeon Processor (Skylake, IBRS), Python 3.6.8
See https://gist.github.com/verdurin/c73fbcf0068342c895e47fd1cad2c7cf for a full test report.

[verdurin]

Here's the error I saw:

== FAILED: Installation ended unsuccessfully (build directory: /dev/shm/CellRank/2.0.2/foss-2023a): build failed (first 300 chars): `/eb/maint/software/Python/3.11.3-GCCcore-12.3.0/bin/python -m pip check` failed:
botocore 1.34.51 has requirement urllib3<2.1,>=1.25.4; python_version >= "3.10", but you have urllib3 2.1.0.
poetry 1.5.1 has requirement urllib3<2.0.0,>=1.26.0, but you have urllib3 2.1.0.

Do I need to rebuild something?

[boegelbot]

Test report by @boegelbot
FAILED
Build succeeded for 0 out of 3 (1 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.3, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18
See https://gist.github.com/boegelbot/dad208f84ff3f62ba577577862b749d5 for a full test report.

[pavelToman]

Here's the error I saw:

== FAILED: Installation ended unsuccessfully (build directory: /dev/shm/CellRank/2.0.2/foss-2023a): build failed (first 300 chars): `/eb/maint/software/Python/3.11.3-GCCcore-12.3.0/bin/python -m pip check` failed:
botocore 1.34.51 has requirement urllib3<2.1,>=1.25.4; python_version >= "3.10", but you have urllib3 2.1.0.
poetry 1.5.1 has requirement urllib3<2.0.0,>=1.26.0, but you have urllib3 2.1.0.

Do I need to rebuild something?

Could it be caused by Poetry? https://github.com/easybuilders/easybuild-easyconfigs/blob/develop/easybuild/easyconfigs/p/poetry/poetry-1.7.1-GCCcore-12.3.0.eb
It imports urllib3 2.1.0. It comes from scVelo and loompy dependencies I hope.

[verdurin]

Test report by @verdurin
FAILED
Build succeeded for 0 out of 1 (1 easyconfigs in total)
easybuild-c7.novalocal - Linux CentOS Linux 7.9.2009, x86_64, Intel Xeon Processor (Skylake, IBRS), Python 3.6.8
See https://gist.github.com/verdurin/5804ecb714839c996c60d5dd14b401e9 for a full test report.

[verdurin]

Yes, I think it is poetry.

[pavelToman]

• Update loompy builddep - use poetry 1.5.1 to not have a conflict with other modules #20148

[casparvl]

Since there was some discussion on #20148 (and I don't think that PR is needed):

== FAILED: Installation ended unsuccessfully (build directory: /dev/shm/CellRank/2.0.2/foss-2023a): build failed (first 300 chars): `/eb/maint/software/Python/3.11.3-GCCcore-12.3.0/bin/python -m pip check` failed:
botocore 1.34.51 has requirement urllib3<2.1,>=1.25.4; python_version >= "3.10", but you have urllib3 2.1.0.
poetry 1.5.1 has requirement urllib3<2.0.0,>=1.26.0, but you have urllib3 2.1.0.

basically means that the poetry version in this PR (i.e. #20115) requires an urllib3 that is older than what we already have in the environment. I don't know which of your dependencies provides urllib3, but it provides version 2.1.0, while poetry-1.5.1 only supports versions smaller than 2.0.0. The same goes for botocore (not sure where that comes in? I don't see it in this PR as a seperate extension).

The solution here would probably be to pick a newer version of poetry that does support urllib3-2.1.0. Probably, you also need to pick a newer botocore version (if that exists), but as said, I don't immediately see where that is coming from. Maybe just upgrade the poetry version first, and take it from there...

[casparvl]

Ah, botocore comes in through PyTorch-Lightning. urllib3 comes in through wandb. I'm not sure what the most elegant way is to resolve this conflict...

• Option 1: downgrade the urllib3 version used in wandb
• Option 2: upgrade the version of botocore in PyTorch-Lightning
• Option 3: install a newer botocore with CellRank (I think this will take priority over the one that comes with PyTorch-Lightning, but you'll see it from the pip check) and hope that PyTorch-Lightning is ok with that newer version
• Option 4: install an older version of ulrllib3 with CellRank and hope that wandb is ok with that newer version

Welcome to dependency hell :) I'm not a big fan of Option 1 and 2 because they change existing recipes btw, but that's my personal opinion.

@verdurin any opinion on this?

[verdurin]

I think I would go with option 3 first, then option 4 if that doesn't work.

[pavelToman]

wandb not using urllib3 anymore: 9f6ea86
Could you try to reinstall it and it should be ok?

@laraPPr has tried "option 3" and got:
python -m pip check failed: aiobotocore 2.12.1 has requirement botocore<1.34.52,>=1.34.41, but you have botocore 1.34.52.
python -m pip check failed: aiobotocore 2.12.1 has requirement botocore<1.34.52,>=1.34.41, but you have botocore 1.34.81.

[verdurin]

Test report by @verdurin
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
easybuild-c7.novalocal - Linux CentOS Linux 7.9.2009, x86_64, Intel Xeon Processor (Skylake, IBRS), Python 3.6.8
See https://gist.github.com/verdurin/9598f6c5627734dd34efcac8931c2bf5 for a full test report.

[verdurin]

My local build is working now that I've rebuilt wandb.

[boegel]

@boegelbot please test @ jsc-zen3
CORE_CNT=16

[boegel]

lgtm

[boegel]

Test report by @boegel
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
node3113.skitty.os - Linux RHEL 8.8, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz, Python 3.6.8
See https://gist.github.com/boegel/46b832d1901fce6d343694d825458dc7 for a full test report.

[boegelbot]

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=20115 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_20115 --ntasks="16" ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

• exit code: 0
• output:

Submitted batch job 3987

Test results coming soon (I hope)...

- notification for comment with ID 2060573848 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

[boegelbot]

Test report by @boegelbot
SUCCESS
Build succeeded for 3 out of 3 (1 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.3, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18
See https://gist.github.com/boegelbot/cca710796257d80fb5826eb9cefb3f29 for a full test report.

[boegel]

Going in, thanks @pavelToman!