Abacus: version and feature update #42974
Open
+220
−17
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@QuantumMisaka @Bitllion @caic99 can you review this PR? This PR modifies the following package(s), for which you are listed as a maintainer:
|
caic99
approved these changes
Mar 5, 2024
There was a problem hiding this comment.
@yizeyi18: When I try to build, I get:
bin/spack install -v abacus
==> Error: Package 'spack.pkg.builtin.nvpl-blas' not found.
Use 'spack create' to create a new package.
Did you mean one of the following packages?
py-liblas
netlib-xblas
liblasThis seems to be common error I also get from another update - how to build?
|
@bernhardkaindl I'm also confused by that...Actually, after updating my fork and before editing abacus the package, this error jumps out. Maybe any bug in CudaPackages class? For a temporary fix, try |
bernhardkaindl
previously approved these changes
Mar 25, 2024
There was a problem hiding this comment.
@yizeyi18 Thanks for checking and mentioning that you can build it after spack clean -m on your laptop!
I don't have a CUDA compiler installed, and my next try ended in an error that it could not find my CUDA compiler.
I then went on to try bin/spack install -v abacus~cuda, which resulted in:
35 -- Looking for a CUDA compiler
36 -- Looking for a CUDA compiler - NOTFOUND
>> 37 CMake Error at /home/t_bernhardk/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-11.4.0/cmake-3.27.9-qdzb3zo5s3c6v
f4oj7p4jevv4qdda4cy/share/cmake-3.27/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
38 Could NOT find IntelMKL (missing: MKL_INTEL MKL_INTEL_THREAD MKL_CORE
39 MKL_SCALAPACK MKL_BLACS_INTELMPI MKL_INCLUDE_DIR)
40 Call Stack (most recent call first):
41 /home/t_bernhardk/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-11.4.0/cmake-3.27.9-qdzb3zo5s3c6vf4oj7p4jevv4q
dda4cy/share/cmake-3.27/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
42 cmake/FindIntelMKL.cmake:24 (find_package_handle_standard_args)
43 CMakeLists.txt:385 (find_package)I guess I'd have to install intel-mkl externally to build it, or select it to be downloaded and installed by spack?
I am just trying to understand the process with the aim of finding out how CI could test the build of packages like abacus correctly, thanks!
|
Hi @bernhardkaindl, EDIT: One spec sample of Sample spec
$ spack spec -l abacus
Input spec
--------------------------------
- abacus
Concretized
--------------------------------
- nbwv4cw abacus@3.5.4%gcc@13.2.0~cuda+elpa~ipo+lcao+libxc~mathlib+mpi+openmp~rocm~tests build_system=cmake build_type=Release generator=make arch=linux-ubuntu22.04-skylake
[+] uhgc4yl ^cereal@1.3.2%gcc@13.2.0~ipo build_system=cmake build_type=Release generator=make patches=2dfa0bf arch=linux-ubuntu22.04-skylake
[e] 5jtet3y ^cmake@3.22.1%gcc@13.2.0~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-ubuntu22.04-skylake
[+] pw5jd2b ^elpa@2023.05.001%gcc@13.2.0~autotune~cuda+mpi+openmp~rocm build_system=autotools arch=linux-ubuntu22.04-skylake
[+] sjr3v5h ^libtool@2.4.7%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-skylake
[+] tdejqfr ^m4@1.4.19%gcc@13.2.0+sigsegv build_system=autotools patches=9dc5fbd,bfdffa7 arch=linux-ubuntu22.04-skylake
[e] nodstrc ^diffutils@3.8%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-skylake
[+] wq3ghrz ^libsigsegv@2.14%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-skylake
[e] zg3y6s6 ^python@3.10.6%gcc@13.2.0+bz2+crypt+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic patches=0d98e93,7d40923,f2fd060 arch=linux-ubuntu22.04-skylake
[+] yvepezs ^fftw@3.3.10%gcc@13.2.0+mpi+openmp~pfft_patches build_system=autotools precision=double,float arch=linux-ubuntu22.04-skylake
[+] eonvvfe ^gcc-runtime@13.2.0%gcc@13.2.0 build_system=generic arch=linux-ubuntu22.04-skylake
[e] 6cnijfi ^gmake@4.3%gcc@13.2.0~guile build_system=autotools patches=599f134 arch=linux-ubuntu22.04-skylake
[+] xqbit2c ^libxc@6.2.2%gcc@13.2.0~cuda~kxc~lxc+shared build_system=autotools arch=linux-ubuntu22.04-skylake
[+] 4kvryat ^autoconf@2.69%gcc@13.2.0 build_system=autotools patches=35c4492,7793209,a49dd5b arch=linux-ubuntu22.04-skylake
[+] 344ajhj ^automake@1.16.5%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-skylake
[+] x6tczzn ^perl@5.38.0%gcc@13.2.0+cpanm+opcode+open+shared+threads build_system=generic patches=714e4d1 arch=linux-ubuntu22.04-skylake
[+] bbwnq6p ^berkeley-db@18.1.40%gcc@13.2.0+cxx~docs+stl build_system=autotools patches=26090f4,b231fcc arch=linux-ubuntu22.04-skylake
[+] aynbov5 ^bzip2@1.0.8%gcc@13.2.0~debug~pic+shared build_system=generic arch=linux-ubuntu22.04-skylake
[+] 2cxlzy6 ^gdbm@1.23%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-skylake
[+] epwc5ug ^readline@8.2%gcc@13.2.0 build_system=autotools patches=bbf97f1 arch=linux-ubuntu22.04-skylake
[+] 5szvra6 ^netlib-scalapack@2.2.0%gcc@13.2.0~ipo~pic+shared build_system=cmake build_type=Release generator=make patches=072b006,1c9ce5f,244a9aa arch=linux-ubuntu22.04-skylake
[+] wgxa4ex ^openblas@0.3.23%gcc@13.2.0~bignuma~consistent_fpcsr+fortran~ilp64+locking+pic+shared build_system=makefile symbol_suffix=none threads=openmp arch=linux-ubuntu22.04-skylake
[+] 5nthj2e ^openmpi@4.1.5%gcc@13.2.0~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~internal-pmix~java~legacylaunchers~lustre~memchecker~openshmem~orterunprefix+romio+rsh~singularity+static+vt+wrapper-rpath build_system=autotools fabrics=none schedulers=none arch=linux-ubuntu22.04-skylake
[+] sg627sl ^hwloc@2.9.1%gcc@13.2.0~cairo~cuda~gl~libudev+libxml2~netloc~nvml~oneapi-level-zero~opencl+pci~rocm build_system=autotools libs=shared,static arch=linux-ubuntu22.04-skylake
[+] f2n3i6i ^libpciaccess@0.17%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-skylake
[+] scbnhrk ^util-macros@1.19.3%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-skylake
[+] rftdk7o ^libxml2@2.10.3%gcc@13.2.0~python build_system=autotools arch=linux-ubuntu22.04-skylake
[+] 7gfjnwe ^libiconv@1.17%gcc@13.2.0 build_system=autotools libs=shared,static arch=linux-ubuntu22.04-skylake
[+] hri3lgq ^xz@5.4.1%gcc@13.2.0~pic build_system=autotools libs=shared,static arch=linux-ubuntu22.04-skylake
[+] ghcbbga ^ncurses@6.4%gcc@13.2.0~symlinks+termlib abi=none build_system=autotools arch=linux-ubuntu22.04-skylake
[+] 43w6aco ^numactl@2.0.14%gcc@13.2.0 build_system=autotools patches=4e1d78c,62fc8a8,ff37630 arch=linux-ubuntu22.04-skylake
[e] 6nqbdun ^openssh@8.9p1%gcc@13.2.0+gssapi build_system=autotools arch=linux-ubuntu22.04-skylake
[+] xm23fkm ^pkgconf@1.9.5%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-skylake
[+] oaj4xlz ^pmix@4.2.4%gcc@13.2.0~docs+pmi_backwards_compatibility~python~restful build_system=autotools arch=linux-ubuntu22.04-skylake
[+] snzahvl ^libevent@2.1.12%gcc@13.2.0+openssl build_system=autotools arch=linux-ubuntu22.04-skylake
[+] ernmtd7 ^openssl@3.1.2%gcc@13.2.0~docs+shared build_system=generic certs=mozilla arch=linux-ubuntu22.04-skylake
[+] 2zfwgve ^ca-certificates-mozilla@2023-05-30%gcc@13.2.0 build_system=generic arch=linux-ubuntu22.04-skylake
[+] saszwn4 ^zlib-ng@2.1.3%gcc@13.2.0+compat+opt build_system=autotools patches=299b958,ae9077a,b692621 arch=linux-ubuntu22.04-skylake |
alalazo
requested changes
Mar 28, 2024
Comment on lines
+30
to
+61
| version("3.5.4", sha256="ab7cbdd07a951da116cc4fe4dfa23e7ac41dda9f35c35d16c267920f267f4722") | ||
| version("3.5.3", sha256="f56066473dbd562f170f40809738076c0862321499ae7fcbd71508581f9ba7bf") | ||
| version("3.5.2", sha256="b4823db244bc68cfa2cff0b4d33140051f56925b19c411f136ce27fb8e1ed3be") | ||
| version("3.5.1", sha256="0867b74ef866033d0120f8b1040fdae8f1dc72a113ffdac6b472b2c8bf1eaf0e") | ||
| version("3.5.0", sha256="0bc43af9bdb5b6a7bc30a72d680bb5054932ecedeb6fd3f4cdd3c4be0651c251") | ||
| version("3.4.4", sha256="654590f97b2ff3ec49f800da667f24fc34c3ff42b392248b9c88104f8d3bf010") | ||
| version("3.4.3", sha256="2c40d2d688ba599d2dcfa916c0e44a5d1a79d04213a9351830aaf231e6e321fe") | ||
| version("3.4.2", sha256="251594ad864580181a26149819572631b757480ee9ce49a473d48855e55a084e") | ||
| version("3.4.1", sha256="20f9a0e99320b1d4cf477ee55a17d1d88c2aa0b019c9a714d38a27a9cc700953") | ||
| version("3.4.0", sha256="01ea63f921596f574d7eb93762724dbd45b01416bec8e3f719834ed7d2e7ed57") | ||
| version("3.3.4", sha256="f9ef0baa624e39eb4f8a4fd7533d1bdd8f61ee3764a62ac980f51238aa102e38") | ||
| version("3.3.3", sha256="8de5904656f7f83c5f5df32b0f05ec2d28c2fbb832576532f7714ea545620327") | ||
| version("3.3.2", sha256="dc3049f68efa72d4e8c3e2c34e7553dbb057242cea3fbca062d5dd484e42bc53") | ||
| version("3.3.1", sha256="29e1a78e1e370e45c5730df4a4b5ae52de74d0cf86771ca9f62dde34643f0f2d") | ||
| version("3.3.0", sha256="9979572b0a60f74e1de432afeec072e16e0994b6a8f5fd5fb9985c3ff34d7816") | ||
| version("3.2.5", sha256="8b96f9e509dedfb3cb43f802715a3ea8bde5d499a525171e314eebe6993c5897") | ||
| version("3.2.4", sha256="65a84a4a02a085c42f86d54b487bf3f1501a45f940370a8a7527c881145f78d6") | ||
| version("3.2.3", sha256="e01d2dd82ec1732499ea92a8ba48660593bc4ba4e165c1347b772561a005866d") | ||
| version("3.2.2", sha256="80c11f6e7fa0b58cd3ef61ed101e519733eed32d7ddd125d9dbccc363f33ffa0") | ||
| version("3.2.1", sha256="05389d66cb4e019764d2f1c8b51cfd445174c229629abe00dd4d31bfdc8ade08") | ||
| version("3.2.0", sha256="cf7de312f6ddf1957621d6acc198ca4aa4e20924df7b4fda69454758e4614ea4") | ||
| version("3.1.4", sha256="4aaf150d74e1c01726f012d23aa1e40e119cac9b8330ae425a29ec3b1040ef1c") | ||
| version("3.1.3", sha256="ca056628472c8959f93f78e8e4fe10e1ea51a0b3333cef0c8c841dd22f9c0d1b") | ||
| version("3.1.2", sha256="0fcebc659754deecfa912391a14ee9749c84ec45849e8b231ad9c765602826d4") | ||
| version("3.1.1", sha256="a268a28e8dce2da98260265cb8edd0a16dd4a307a5016022c5cd2598e9df93c9") | ||
| version("3.1.0", sha256="f038e20c712276c5a7c8f26ed6af6ff5d7c57110accd45882d11b1d64dee3f90") | ||
| version("3.0.5", sha256="81ff1b8f10909eeda2d888f97f615e5b92aa26c101a32f36f346f78aa7638ba9") | ||
| version("3.0.4", sha256="b01714e734b0aa5b0b52f3d7871e4bc9f6c81f25a444b1287e3f2e98a4f5d289") | ||
| version("3.0.3", sha256="a4ae83cdb2c5029f152064f163d1b66b265980793d76b79908c073027401d2fc") | ||
| version("3.0.2", sha256="b0c3aad9cac18d0ca73c78c69326031407af6a4bfcc953b3a27506c298bc59a3") | ||
| version("3.0.1", sha256="812941146c31ab53c9a7695abbca6bfb36ae55878e38f8d0f13a17f3d9c36dc0") | ||
| version("3.0.0", sha256="1c1299f53788beb2f6b6180d8484d584b6f293c81cea25f45b9632dd608ba4f9") |
There was a problem hiding this comment.
Do we need all these versions? Or can we get by with the latest patch for each minor?
There was a problem hiding this comment.
I followed llvm's version-including strategy; patching minors is also acceptable. As a fast-releasing package, I suppose including versions directly.
Comment on lines
+134
to
+136
| variant( | ||
| "tests", default=False, description="Build ABACUS unit tests", when="build_system=cmake" | ||
| ) |
There was a problem hiding this comment.
Can we avoid the variant and use self.run_tests instead? Or is this variant installing something in the prefix?
There was a problem hiding this comment.
Nothing installed. What is self.run_tests?
There was a problem hiding this comment.
It's a variable that is set to True when the package is installed asking for build-time tests. You can grep the builtin repo to see how it's used. One example would be hdf5, for instance.
Comment on lines
+137
to
+142
| variant( | ||
| "benchmarks", | ||
| default=False, | ||
| description="Enable ABACUS's builtin benchmark tests", | ||
| when="+tests", | ||
| ) |
Comment on lines
173
to
176
| depends_on( | ||
| "openblas threads=openmp", | ||
| when="^[virtuals=blas] openblas" or "^[virtuals=lapack] openblas", | ||
| ) |
There was a problem hiding this comment.
This would evaluate to the first arg, so you can simply use:
Suggested change
| depends_on( | |
| "openblas threads=openmp", | |
| when="^[virtuals=blas] openblas" or "^[virtuals=lapack] openblas", | |
| ) | |
| depends_on( | |
| "openblas threads=openmp", | |
| when="^[virtuals=blas] openblas", | |
| ) |
As far as I know openblas provides lapack and blas together (it cannot build on top of other blas libraries).
There was a problem hiding this comment.
In some corner cases, openblas could be concretized as only a lapack provider, e. g.:
one corner case example
yizeyi18@DESKTOP-5UK5DD4:~/soft/spack$ spack spec -l abacus ^netlib-xblas
Input spec
--------------------------------
- abacus
- ^netlib-xblas
Concretized
--------------------------------
- q44hmr5 abacus@develop%gcc@13.2.0~cuda+elpa~ipo+lcao+libxc~mathlib+mpi+openmp~rocm~tests build_system=cmake build_type=Release generator=make arch=linux-ubuntu22.04-skylake
[+] uhgc4yl ^cereal@1.3.2%gcc@13.2.0~ipo build_system=cmake build_type=Release generator=make patches=2dfa0bf arch=linux-ubuntu22.04-skylake
[e] 5jtet3y ^cmake@3.22.1%gcc@13.2.0~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-ubuntu22.04-skylake
- xccstl6 ^elpa@2023.05.001%gcc@13.2.0~autotune~cuda+mpi+openmp~rocm build_system=autotools arch=linux-ubuntu22.04-skylake
[+] sjr3v5h ^libtool@2.4.7%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-skylake
[+] tdejqfr ^m4@1.4.19%gcc@13.2.0+sigsegv build_system=autotools patches=9dc5fbd,bfdffa7 arch=linux-ubuntu22.04-skylake
[e] nodstrc ^diffutils@3.8%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-skylake
[+] wq3ghrz ^libsigsegv@2.14%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-skylake
[e] zg3y6s6 ^python@3.10.6%gcc@13.2.0+bz2+crypt+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic patches=0d98e93,7d40923,f2fd060 arch=linux-ubuntu22.04-skylake
[+] eonvvfe ^gcc-runtime@13.2.0%gcc@13.2.0 build_system=generic arch=linux-ubuntu22.04-skylake
[e] 6cnijfi ^gmake@4.3%gcc@13.2.0~guile build_system=autotools patches=599f134 arch=linux-ubuntu22.04-skylake
[e] c4hwk6l ^intel-oneapi-mkl@2023.2.0%gcc@13.2.0~cluster+envmods~ilp64+shared build_system=generic mpi_family=none threads=none arch=linux-ubuntu22.04-skylake
[e] vkj5jcu ^intel-oneapi-mpi@2021.10.0%gcc@13.2.0+envmods~external-libfabric~generic-names~ilp64 build_system=generic arch=linux-ubuntu22.04-skylake
[+] xqbit2c ^libxc@6.2.2%gcc@13.2.0~cuda~kxc~lxc+shared build_system=autotools arch=linux-ubuntu22.04-skylake
[+] 4kvryat ^autoconf@2.69%gcc@13.2.0 build_system=autotools patches=35c4492,7793209,a49dd5b arch=linux-ubuntu22.04-skylake
[+] 344ajhj ^automake@1.16.5%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-skylake
[+] x6tczzn ^perl@5.38.0%gcc@13.2.0+cpanm+opcode+open+shared+threads build_system=generic patches=714e4d1 arch=linux-ubuntu22.04-skylake
[+] bbwnq6p ^berkeley-db@18.1.40%gcc@13.2.0+cxx~docs+stl build_system=autotools patches=26090f4,b231fcc arch=linux-ubuntu22.04-skylake
[+] aynbov5 ^bzip2@1.0.8%gcc@13.2.0~debug~pic+shared build_system=generic arch=linux-ubuntu22.04-skylake
[+] 2cxlzy6 ^gdbm@1.23%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-skylake
[+] epwc5ug ^readline@8.2%gcc@13.2.0 build_system=autotools patches=bbf97f1 arch=linux-ubuntu22.04-skylake
[+] ghcbbga ^ncurses@6.4%gcc@13.2.0~symlinks+termlib abi=none build_system=autotools arch=linux-ubuntu22.04-skylake
[+] xm23fkm ^pkgconf@1.9.5%gcc@13.2.0 build_system=autotools arch=linux-ubuntu22.04-skylake
[+] saszwn4 ^zlib-ng@2.1.3%gcc@13.2.0+compat+opt build_system=autotools patches=299b958,ae9077a,b692621 arch=linux-ubuntu22.04-skylake
- cyx2ffx ^netlib-scalapack@2.2.0%gcc@13.2.0~ipo~pic+shared build_system=cmake build_type=Release generator=make patches=072b006,1c9ce5f,244a9aa arch=linux-ubuntu22.04-skylake
- omuynna ^netlib-xblas@1.0.248%gcc@13.2.0+fortran+plain_blas build_system=autotools arch=linux-ubuntu22.04-skylake
[+] wgxa4ex ^openblas@0.3.23%gcc@13.2.0~bignuma~consistent_fpcsr+fortran~ilp64+locking+pic+shared build_system=makefile symbol_suffix=none threads=openmp arch=linux-ubuntu22.04-skylake
In this example, netlib-xblas, openblas, netlib-scalapack and intel-oneapi-mkl serves as provider of blas, lapack, scalapack and fftw-api.
There was a problem hiding this comment.
What I mean is:
>>> "foo" or "bar"
'foo'
so the argument above evaluates as the first operand
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
caic99
approved these changes
Apr 9, 2024
alalazo
reviewed
Apr 15, 2024
Comment on lines
63
to
77
| version( | ||
| "2.3.5", | ||
| sha256="81c44be76c820b59a8b95f2b398c772160068a5e323f319e33704ff87dbefc5e", | ||
| deprecated=True, | ||
| ) | ||
| version( | ||
| "2.3.4", | ||
| sha256="16b817311c8055cf055021d895642fbb913585f08ec96f9aff3e0ef20148ec0e", | ||
| deprecated=True, | ||
| ) | ||
| version( | ||
| "2.3.3", | ||
| sha256="a1246c4a6de385826cae15d9a446e862331eb189a592622b1014c7b49504c908", | ||
| deprecated=True, | ||
| ) |
There was a problem hiding this comment.
Let's not add versions that start as deprecated 😉
There was a problem hiding this comment.
OK. Removed by 45ce442.
Comment on lines
+134
to
+136
| variant( | ||
| "tests", default=False, description="Build ABACUS unit tests", when="build_system=cmake" | ||
| ) |
There was a problem hiding this comment.
It's a variable that is set to True when the package is installed asking for build-time tests. You can grep the builtin repo to see how it's used. One example would be hdf5, for instance.
Comment on lines
173
to
176
| depends_on( | ||
| "openblas threads=openmp", | ||
| when="^[virtuals=blas] openblas" or "^[virtuals=lapack] openblas", | ||
| ) |
There was a problem hiding this comment.
What I mean is:
>>> "foo" or "bar"
'foo'
so the argument above evaluates as the first operand
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Package abacus had been created before, but is long outdated. This PR updates new versions of the package, and implements build and run features as in deepmodeling/abacus-develop#3291.
@spackbot fix style