Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

{bio}[foss/2022a] AFNI v23.0.03 w/ Python 3.10.4 #17309

Conversation

sassy-crick
Copy link
Collaborator

@sassy-crick sassy-crick commented Feb 13, 2023

(created using eb --new-pr)

Requires:

@boegelbot
Copy link
Collaborator

@sassy-crick: Tests failed in GitHub Actions, see https://github.com/easybuilders/easybuild-easyconfigs/actions/runs/4162756804
Output from first failing test suite run:

ERROR: test_conflicts (test.easyconfigs.easyconfigs.EasyConfigTest)
Check whether any conflicts occur in software dependency graphs.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 326, in test_conflicts
    self.assertFalse(check_conflicts(self.ordered_specs, modules_tool(), check_inter_ec_conflicts=False),
  File "test/easyconfigs/easyconfigs.py", line 278, in ordered_specs
    EasyConfigTest.resolve_all_dependencies()
  File "test/easyconfigs/easyconfigs.py", line 216, in resolve_all_dependencies
    cls._parsed_easyconfigs, modules_tool(), retain_all_deps=True)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 463, in resolve_dependencies
    raise_error_missing_deps(totally_missing, extra_msg="no easyconfig file or existing module found")
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 326, in raise_error_missing_deps
    raise EasyBuildError(error_msg)
EasyBuildError: 'Missing dependencies: motif/2.3.8-foss-2022a (no easyconfig file or existing module found)'

======================================================================
ERROR: test_dep_graph (test.easyconfigs.easyconfigs.EasyConfigTest)
Unit test that builds a full dependency graph.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 313, in test_dep_graph
    dep_graph(fn, self.ordered_specs)
  File "test/easyconfigs/easyconfigs.py", line 278, in ordered_specs
    EasyConfigTest.resolve_all_dependencies()
  File "test/easyconfigs/easyconfigs.py", line 216, in resolve_all_dependencies
    cls._parsed_easyconfigs, modules_tool(), retain_all_deps=True)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 463, in resolve_dependencies
    raise_error_missing_deps(totally_missing, extra_msg="no easyconfig file or existing module found")
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 326, in raise_error_missing_deps
    raise EasyBuildError(error_msg)
EasyBuildError: 'Missing dependencies: motif/2.3.8-foss-2022a (no easyconfig file or existing module found)'

======================================================================
ERROR: test_dep_versions_per_toolchain_generation (test.easyconfigs.easyconfigs.EasyConfigTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 853, in test_dep_versions_per_toolchain_generation
    for ec in self.ordered_specs:
  File "test/easyconfigs/easyconfigs.py", line 278, in ordered_specs
    EasyConfigTest.resolve_all_dependencies()
  File "test/easyconfigs/easyconfigs.py", line 216, in resolve_all_dependencies
    cls._parsed_easyconfigs, modules_tool(), retain_all_deps=True)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 463, in resolve_dependencies
    raise_error_missing_deps(totally_missing, extra_msg="no easyconfig file or existing module found")
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 326, in raise_error_missing_deps
    raise EasyBuildError(error_msg)
EasyBuildError: 'Missing dependencies: motif/2.3.8-foss-2022a (no easyconfig file or existing module found)'

----------------------------------------------------------------------
Ran 16394 tests in 753.258s

FAILED (errors=3)
ERROR: Not all tests were successful

bleep, bloop, I'm just a bot (boegelbot v20200716.01)
Please talk to my owner @boegel if you notice me acting stupid),
or submit a pull request to https://github.com/boegel/boegelbot fix the problem.

@boegelbot
Copy link
Collaborator

@sassy-crick: Tests failed in GitHub Actions, see https://github.com/easybuilders/easybuild-easyconfigs/actions/runs/4164053635
Output from first failing test suite run:

ERROR: test_conflicts (test.easyconfigs.easyconfigs.EasyConfigTest)
Check whether any conflicts occur in software dependency graphs.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 326, in test_conflicts
    self.assertFalse(check_conflicts(self.ordered_specs, modules_tool(), check_inter_ec_conflicts=False),
  File "test/easyconfigs/easyconfigs.py", line 278, in ordered_specs
    EasyConfigTest.resolve_all_dependencies()
  File "test/easyconfigs/easyconfigs.py", line 216, in resolve_all_dependencies
    cls._parsed_easyconfigs, modules_tool(), retain_all_deps=True)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 463, in resolve_dependencies
    raise_error_missing_deps(totally_missing, extra_msg="no easyconfig file or existing module found")
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 326, in raise_error_missing_deps
    raise EasyBuildError(error_msg)
EasyBuildError: 'Missing dependencies: motif/2.3.8-foss-2022a (no easyconfig file or existing module found)'

======================================================================
ERROR: test_dep_graph (test.easyconfigs.easyconfigs.EasyConfigTest)
Unit test that builds a full dependency graph.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 313, in test_dep_graph
    dep_graph(fn, self.ordered_specs)
  File "test/easyconfigs/easyconfigs.py", line 278, in ordered_specs
    EasyConfigTest.resolve_all_dependencies()
  File "test/easyconfigs/easyconfigs.py", line 216, in resolve_all_dependencies
    cls._parsed_easyconfigs, modules_tool(), retain_all_deps=True)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 463, in resolve_dependencies
    raise_error_missing_deps(totally_missing, extra_msg="no easyconfig file or existing module found")
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 326, in raise_error_missing_deps
    raise EasyBuildError(error_msg)
EasyBuildError: 'Missing dependencies: motif/2.3.8-foss-2022a (no easyconfig file or existing module found)'

======================================================================
ERROR: test_dep_versions_per_toolchain_generation (test.easyconfigs.easyconfigs.EasyConfigTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 853, in test_dep_versions_per_toolchain_generation
    for ec in self.ordered_specs:
  File "test/easyconfigs/easyconfigs.py", line 278, in ordered_specs
    EasyConfigTest.resolve_all_dependencies()
  File "test/easyconfigs/easyconfigs.py", line 216, in resolve_all_dependencies
    cls._parsed_easyconfigs, modules_tool(), retain_all_deps=True)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 463, in resolve_dependencies
    raise_error_missing_deps(totally_missing, extra_msg="no easyconfig file or existing module found")
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 326, in raise_error_missing_deps
    raise EasyBuildError(error_msg)
EasyBuildError: 'Missing dependencies: motif/2.3.8-foss-2022a (no easyconfig file or existing module found)'

----------------------------------------------------------------------
Ran 16397 tests in 762.697s

FAILED (errors=3)
ERROR: Not all tests were successful

bleep, bloop, I'm just a bot (boegelbot v20200716.01)
Please talk to my owner @boegel if you notice me acting stupid),
or submit a pull request to https://github.com/boegel/boegelbot fix the problem.

@sassy-crick sassy-crick changed the title {bio}[foss/2022a] AFNI v23.0.03 w/ Python 3.10.4 {bio}[foss/2022a] AFNI v23.0.03 w/ Python 3.10.4 WIP Feb 13, 2023
@sassy-crick
Copy link
Collaborator Author

I mark that as Work In Progress as during the installation, some Python packages will be installed in the Python directory, both lib and bin! That messed up my Python install which I only found by chance. So I need to dig into that more.

@sassy-crick
Copy link
Collaborator Author

Hopefully fixed now, so WIP removed

@sassy-crick sassy-crick changed the title {bio}[foss/2022a] AFNI v23.0.03 w/ Python 3.10.4 WIP {bio}[foss/2022a] AFNI v23.0.03 w/ Python 3.10.4 Feb 13, 2023
@zao
Copy link
Contributor

zao commented Feb 17, 2023

Test report by @zao
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
eb-rocky9.zao.se - Linux Rocky Linux 9.1 (Blue Onyx), x86_64, AMD Ryzen 9 3900X 12-Core Processor, Python 3.9.14
See https://gist.github.com/29bd405e466e7d85a8f28fcca1f792e0 for a full test report.


builddependencies = [
('CMake', '3.24.3'),
]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sassy-crick wondering whether the buildopts from the previous versions are still needed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@verdurin from what I could see it is not required any more:

[  0%] Building C object src/CMakeFiles/pcor_objs.dir/configured_pcor/afni_pcor_byte.c.o
cd /dev/shm/jsassman/AFNI/23.0.03/foss-2022a/easybuild_obj/src && /apps/sw-eb/software/GCCcore/11.3.0/bin/gcc -DDYNAMIC_suffix=\".so\" -DREPLACE_XT -DDTYPE=byte -I/dev/shm/jsassman/AFNI/23.0.03/foss-2022a/afni-AFNI_23.0.03/src/niml -I/dev/shm/jsassman/AFNI/23.0.03/foss-2022a/easybuild_obj/_deps/nifti_clib-build/nifti2 -I/dev/shm/jsassman/AFNI/23.0.03/foss-2022a/easybuild_obj/_deps/nifti_clib-src/nifti2 -I/dev/shm/jsassman/AFNI/23.0.03/foss-2022a/easybuild_obj/_deps/nifti_clib-build/znzlib -I/dev/shm/jsassman/AFNI/23.0.03/foss-2022a/easybuild_obj/_deps/nifti_clib-src/znzlib -I/dev/shm/jsassman/AFNI/23.0.03/foss-2022a/easybuild_obj/_deps/nifti_clib-build/nifticdf -I/dev/shm/jsassman/AFNI/23.0.03/foss-2022a/easybuild_obj/_deps/nifti_clib-src/nifticdf -I/dev/shm/jsassman/AFNI/23.0.03/foss-2022a/easybuild_obj/_deps/nifti_clib-build/niftilib -I/dev/shm/jsassman/AFNI/23.0.03/foss-2022a/easybuild_obj/_deps/nifti_clib-src/niftilib -I/dev/shm/jsassman/AFNI/23.0.03/foss-2022a/afni-AFNI_23.0.03/src -I/dev/shm/jsassman/AFNI/23.0.03/foss-2022a/easybuild_obj/src/configured_pcor -I/dev/shm/jsassman/AFNI/23.0.03/foss-2022a/afni-AFNI_23.0.03/src/3DEdge/src -I/dev/shm/jsassman/AFNI/23.0.03/foss-2022a/afni-AFNI_23.0.03/src/f2c -I/dev/shm/jsassman/AFNI/23.0.03/foss-2022a/easybuild_obj/src -I/dev/shm/jsassman/AFNI/23.0.03/foss-2022a/afni-AFNI_23.0.03/src/rickr -O2 -ftree-vectorize -march=native -fno-math-errno -fopenmp -fPIC -DUSE_TRACING -DHAVE_ZLIB -DREAD_WRITE_64 -DLINUX2 -O3 -DNDEBUG -fPIC -MD -MT src/CMakeFiles/pcor_objs.dir/configured_pcor/afni_pcor_byte.c.o -MF CMakeFiles/pcor_objs.dir/configured_pcor/afni_pcor_byte.c.o.d -o CMakeFiles/pcor_objs.dir/configured_pcor/afni_pcor_byte.c.o -c /dev/shm/jsassman/AFNI/23.0.03/foss-2022a/easybuild_obj/src/configured_pcor/afni_pcor_byte.c

That said, please feel free to test it to make sure nothing is slipping through. I don't know the program so I might get it wrong.

One thing which I cannot work out is I find two files in my local PYTHONPATH which should not be there:
afnipy.egg-link easy-install.pth
The content of them refers to the build directory. Do you or @zao have any idea if they are actually needed?

$ cat python3.10/site-packages/afnipy.egg-link
/dev/shm/jsassman/AFNI/23.0.03/foss-2022a/afni-AFNI_23.0.03/src/python_scripts
$ cat python3.10/site-packages/easy-install.pth
/dev/shm/jsassman/AFNI/23.0.03/foss-2022a/afni-AFNI_23.0.03/src/python_scripts

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zao would know much better than I would.

@verdurin
Copy link
Member

Test report by @verdurin
SUCCESS
Build succeeded for 4 out of 4 (1 easyconfigs in total)
easybuild-c7.novalocal - Linux CentOS Linux 7.9.2009, x86_64, Intel Xeon Processor (Skylake, IBRS), Python 3.6.8
See https://gist.github.com/2b54f0892a918fa2c228c64271462821 for a full test report.

@tlangfor
Copy link

@sassy-crick Have you fixed the issue of files being installed into Python's directories? I ran into this same issue when trying to build a 2020b version on my system.

@sassy-crick
Copy link
Collaborator Author

@tlangfor Apologies for all the delays, busy year.
I had another look at it and hopefully I finally nailed it. The install I tested is running the afni_system_check.py -check_all command fine, some warnings messages are due to missing data-directories which, from what I understand, users can download themselves.
Please try it out and see if that is all working for you now.
Also, please see the latest PR #19654 as well if that helps.

@sassy-crick
Copy link
Collaborator Author

FFmpeg downgraded to 4.4.2 so the checks are passed. Apparently there is also the 5.0.1 version. Please advise which one to use. Thanks

@sassy-crick
Copy link
Collaborator Author

@boegelbot please test @ generoso

@boegelbot
Copy link
Collaborator

@sassy-crick: Request for testing this PR well received on login1

PR test command 'EB_PR=17309 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs /opt/software/slurm/bin/sbatch --job-name test_PR_17309 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 12674

Test results coming soon (I hope)...

- notification for comment with ID 1900861360 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Collaborator

Test report by @boegelbot
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
cns2 - Linux Rocky Linux 8.5, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/boegelbot/d5da8746e18d942bac8395e618668179 for a full test report.

@sassy-crick
Copy link
Collaborator Author

Closed as PR #19654 is newer and just got merged.

@sassy-crick sassy-crick closed this Apr 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants