Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

{lib}[GCCcore/11.2.0,foss/2021b] CuPy v11.4.0, pytest v7.2.2 w/ Python 3.9.6 #17526

Merged
70 changes: 70 additions & 0 deletions easybuild/easyconfigs/c/CuPy/CuPy-11.4.0-foss-2021b-CUDA-11.4.1.eb
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
easyblock = 'PythonBundle'

name = 'CuPy'
version = '11.4.0'
versionsuffix = '-CUDA-%(cudaver)s'

homepage = 'https://cupy.dev'
description = "CuPy is an open-source array library accelerated with NVIDIA CUDA."

toolchain = {'name': 'foss', 'version': '2021b'}

builddependencies = [
('pytest', '7.2.2'),
]

dependencies = [
('Python', '3.9.6'),
('SciPy-bundle', '2021.10'),
('CUDA', '11.4.1', '', SYSTEM),
('cuDNN', '8.2.2.26', versionsuffix, SYSTEM),
('NCCL', '2.10.3', versionsuffix),
('cuTENSOR', '1.6.1.5', versionsuffix, SYSTEM),
# Adding cuSPARSELt currently makes the build fail. Keeping it around for later versions to pick up.
# ('cuSPARSELt', '0.3.0.3', versionsuffix, SYSTEM),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this commented? If it's really not needed, can't it just be removed?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was an optional thing, but i had problems getting that to work in my PR. I've forgotten what the reason was, only that i didn't care enough about that optional feature.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It currently crashes/fails to compile some part, I left in in place to be enabled for a newer toolchain

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we put that as a comment? I.e. that it's an optional dep, commented out because it gave issues, but could be re-enabled for newer toolchain?

]

use_pip = True

exts_default_options = {'source_urls': [PYPI_LOWER_SOURCE]}

# A bunch of the tests are failing or are just having problems.
_skip_tests = [
'tests/cupyx_tests/scipy_tests',
'tests/cupyx_tests/distributed_tests',
'tests/cupyx_tests/tools_tests',
'tests/example_tests',
'tests/cupy_tests/testing_tests/test_parameterized.py',
'tests/cupy_tests/fft_tests/test_fft.py',
]
_ignore_list = ' --ignore='.join(_skip_tests)

_deselect_tests = [
'tests/cupy_tests/core_tests/test_carray.py::TestCArray32BitBoundary_param_',
# float16 has too low precision for these tests as they are written
# See https://github.com/easybuilders/easybuild-easyconfigs/pull/17526#issuecomment-1470843170 for details.
'tests/cupy_tests/linalg_tests/test_product.py::TestProduct',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, turns out this syntax isn't a thing in pytest. This line does nothing. Looks like this syntax pytest-dev/pytest#3198 but this was never supported, the issue was closed in favor of a regex of what to include.
Needs to be a different flag -k "not test_tensordot"

Ugh. This sucks. It won't even let us specify the full name, like "TestProduct::test_tensordot", just the last part.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So options are

  1. let this fail on A40, whoever installs it just has to ignore test failures.
  2. patch this test out
  3. add the -k flag, which just really sucks if we ever need to add more than one.

Regardless, this broken ::TestProduct ignore line should be removed, as it does nothing

]
_deselect_list = ' --deselect='.join(_deselect_tests)

exts_list = [
('fastrlock', '0.5', {
'checksums': ['9ae1a31f6e069b5f0f28ba63c594d0c952065de0a375f7b491d21ebaccc5166f'],
}),
('cupy', version, {
# Must run tests here since the required version of pytest is a builddep
# Note! test_cudnn and some others will likely fail on T4 GPUs due to out of memory.
'runtest': 'export CUPY_TEST_GPU_LIMIT=1 CUPY_CACHE_DIR="%%(builddir)s" && '
'pytest --ignore=%s --deselect=%s tests' % (_ignore_list, _deselect_list),
'testinstall': True,
'checksums': ['03d52b2626e02a3a2b46d714c1cd03e702c8fe33915fcca6ed8de5c539964f49'],
}),
]

sanity_check_commands = [
"python -c 'import cupy'",
]

sanity_pip_check = True

moduleclass = 'lib'
80 changes: 80 additions & 0 deletions easybuild/easyconfigs/p/pytest/pytest-7.2.2-GCCcore-11.2.0.eb
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
easyblock = 'PythonBundle'

name = 'pytest'
version = '7.2.2'

homepage = 'https://docs.pytest.org/en/latest/'
description = """The pytest framework makes it easy to write small,
readable tests, and can scale to support complex functional testing for
applications and libraries."""

toolchain = {'name': 'GCCcore', 'version': '11.2.0'}

builddependencies = [
('binutils', '2.37'),
akesandgren marked this conversation as resolved.
Show resolved Hide resolved
]

dependencies = [
('Python', '3.9.6'),
]

use_pip = True

exts_default_options = {'source_urls': [PYPI_LOWER_SOURCE]}

# Note! Some of the file system related tests may fail on shared file systems.
# Notably TestPOSIXLocalPath.test_copy_stat_file, TestPOSIXLocalPath.test_copy_stat_dir
# and test_source_mtime_long_long are known to fail on GPFS
# Build with buildpath and tmpdir set to a local file system to avoid this
# or use --ignore-test-failures
_skip_tests = [
'testing/io/test_terminalwriter.py',
'testing/test_terminal.py',
'testing/test_debugging.py',
'testing/test_config.py',
'testing/test_helpconfig.py',
]
_ignore_tests = ' --ignore='.join(_skip_tests)

exts_list = [
('tomli', '2.0.1', {
'checksums': ['de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f'],
}),
('setuptools-scm', '7.1.0', {
'source_tmpl': 'setuptools_scm-%(version)s.tar.gz',
'checksums': ['6c508345a771aad7d56ebff0e70628bf2b0ec7573762be9960214730de278f27'],
}),
('flit-core', '3.8.0', {
'source_tmpl': 'flit_core-%(version)s.tar.gz',
'checksums': ['b305b30c99526df5e63d6022dd2310a0a941a187bd3884f4c8ef0418df6c39f3'],
}),
('flit-scm', '1.7.0', {
'source_tmpl': 'flit_scm-%(version)s.tar.gz',
'checksums': ['961bd6fb24f31bba75333c234145fff88e6de0a90fc0f7e5e7c79deca69f6bb2'],
}),
('exceptiongroup', '1.1.0', {
'checksums': ['bcb67d800a4497e1b404c2dd44fca47d3b7a5e5433dbab67f96c1a685cdfdf23'],
}),
('hypothesis', '6.54.6', {
'checksums': ['2d5e2d5ccd0efce4e0968a6164f4e4853f808e33f4d91490c975c98beec0c7c3'],
}),
('elementpath', '4.0.1', {
'checksums': ['1162e4c8e5501bd36291b668f4449b8125fea5ef64a26da8d71da31126725aa5'],
}),
('xmlschema', '2.2.2', {
'checksums': ['0caa96668807b4b51c42a0fe2b6610752bc59f069615df3e34dcfffb962973fd'],
}),
(name, version, {
'checksums': ['c99ab0c73aceb050f68929bc93af19ab6db0558791c6a0715723abe9d0ade9d4'],
}),
]

sanity_check_commands = [
"python -c 'import pytest'",
'cd %%(builddir)s/%%(name)s/%%(name)s-%%(version)s && %%(installdir)s/bin/pytest --ignore=%s testing'
% _ignore_tests,
]

sanity_pip_check = True

moduleclass = 'lib'