testing issues

[drcandacemakedamoore]

On my machine there are a couple of failing tests:
FAILED test/parallel_feature_extraction_test.py::test_parallel_feature_extraction - OSError: [Errno 0] AssignProcessToJobObject() failed
FAILED test/parallel_feature_extraction_test.py::test_parallel_dl_preprocessing - OSError: [Errno 0] AssignProcessToJobObject() failed

If these don't fail for you, I can look into it, and with permission given the requirements of the JOSS process, maybe even put in a PR (but I'm not sure if this is allowed)

[alexzwanenburg]

I think this may be an issue with current version of ray instead of mirp. ray requires (but does not install) the chardet package. The error may disappear after installing that package using pip.

[drcandacemakedamoore]

I think this may be an issue with current version of ray instead of mirp. ray requires (but does not install) the chardet package. The error may disappear after installing that package using pip.

I installed chardet, and it put in the latest version (5.2.0), but this does not solve the problem. I see you use the ray package. I have not looked too deeply into it, but it could even be a bug there. Do all your tests pass?

[alexzwanenburg]

At my end (Windows 10 and Windows 11), all tests pass, but did initially encounter a similar error. I will try to reproduce the error in a fresh environment, and figure out which package I installed to resolve it. That package may need to be listed as a dependency.

[alexzwanenburg]

I was not able to reproduce the issue. My environment is as follows. Note that I installed ray using pip.

Name                    Version                   Build  Channel
aiosignal                 1.3.1                    pypi_0    pypi
alabaster                 0.7.16                   pypi_0    pypi
aom                       3.6.0                hd77b12b_0
attrs                     23.2.0                   pypi_0    pypi
babel                     2.14.0                   pypi_0    pypi
blas                      1.0                         mkl
blosc                     1.21.3               h6c2663c_0
bottleneck                1.3.7           py311hd7041d2_0
brotli                    1.0.9                h2bbff1b_7
brotli-bin                1.0.9                h2bbff1b_7
bzip2                     1.0.8                h2bbff1b_5
ca-certificates           2024.3.11            haa95532_0
certifi                   2024.2.2                 pypi_0    pypi
cfitsio                   3.470                h2bbff1b_7
charls                    2.2.0                h6c2663c_0
charset-normalizer        3.3.2                    pypi_0    pypi
click                     8.1.7                    pypi_0    pypi
colorama                  0.4.6                    pypi_0    pypi
contourpy                 1.2.0           py311h59b6b97_0
cycler                    0.11.0             pyhd3eb1b0_0
dav1d                     1.2.1                h2bbff1b_0
docutils                  0.20.1                   pypi_0    pypi
filelock                  3.13.2                   pypi_0    pypi
fonttools                 4.25.0             pyhd3eb1b0_0
freetype                  2.12.1               ha860e81_0
frozenlist                1.4.1                    pypi_0    pypi
giflib                    5.2.1                h8cc25b3_3
icc_rt                    2022.1.0             h6049295_2
icu                       73.1                 h6c2663c_0
idna                      3.6                      pypi_0    pypi
imagecodecs               2023.1.23       py311he6ff3c7_0
imageio                   2.33.1          py311haa95532_0
imagesize                 1.4.1                    pypi_0    pypi
iniconfig                 2.0.0                    pypi_0    pypi
intel-openmp              2023.1.0         h59b6b97_46320
itk                       5.3.0                    pypi_0    pypi
itk-core                  5.3.0                    pypi_0    pypi
itk-filtering             5.3.0                    pypi_0    pypi
itk-io                    5.3.0                    pypi_0    pypi
itk-numerics              5.3.0                    pypi_0    pypi
itk-registration          5.3.0                    pypi_0    pypi
itk-segmentation          5.3.0                    pypi_0    pypi
jinja2                    3.1.3                    pypi_0    pypi
jpeg                      9e                   h2bbff1b_1
jsonschema                4.21.1                   pypi_0    pypi
jsonschema-specifications 2023.12.1                pypi_0    pypi
kiwisolver                1.4.4           py311hd77b12b_0
krb5                      1.20.1               h5b6d351_0
lazy_loader               0.3             py311haa95532_0
lcms2                     2.12                 h83e58a3_0
lerc                      3.0                  hd77b12b_0
libaec                    1.0.4                h33f27b4_1
libavif                   0.11.1               h2bbff1b_0
libbrotlicommon           1.0.9                h2bbff1b_7
libbrotlidec              1.0.9                h2bbff1b_7
libbrotlienc              1.0.9                h2bbff1b_7
libclang                  14.0.6          default_hb5a9fac_1
libclang13                14.0.6          default_h8e68704_1
libdeflate                1.17                 h2bbff1b_1
libffi                    3.4.4                hd77b12b_0
libpng                    1.6.39               h8cc25b3_0
libpq                     12.17                h906ac69_0
libtiff                   4.5.1                hd77b12b_0
libwebp-base              1.3.2                h2bbff1b_0
libzopfli                 1.0.3                ha925a31_0
lz4-c                     1.9.4                h2bbff1b_0
markupsafe                2.1.5                    pypi_0    pypi
matplotlib                3.8.0           py311haa95532_0
matplotlib-base           3.8.0           py311hf62ec03_0
mkl                       2023.1.0         h6b88ed4_46358
mkl-service               2.4.0           py311h2bbff1b_1
mkl_fft                   1.3.8           py311h2bbff1b_0
mkl_random                1.2.4           py311h59b6b97_0
msgpack                   1.0.8                    pypi_0    pypi
munkres                   1.1.4                      py_0
networkx                  3.1             py311haa95532_0
numexpr                   2.8.7           py311h1fcbade_0
numpy                     1.26.4          py311hdab7c0b_0
numpy-base                1.26.4          py311hd01c5d8_0
openjpeg                  2.4.0                h4fc8c34_0
openssl                   3.0.13               h2bbff1b_0
packaging                 24.0                     pypi_0    pypi
pandas                    2.2.1           py311hea22821_0
pillow                    10.2.0          py311h2bbff1b_0
pip                       23.3.1          py311haa95532_0
pluggy                    1.4.0                    pypi_0    pypi
ply                       3.11            py311haa95532_0
protobuf                  5.26.0                   pypi_0    pypi
pydicom                   2.4.4              pyhd8ed1ab_0    conda-forge
pygments                  2.17.2                   pypi_0    pypi
pyparsing                 3.0.9           py311haa95532_0
pyqt                      5.15.10         py311hd77b12b_0
pyqt5-sip                 12.13.0         py311h2bbff1b_0
pytest                    8.1.1                    pypi_0    pypi
python                    3.11.8               he1021f5_0
python-dateutil           2.8.2              pyhd3eb1b0_0
python-tzdata             2023.3             pyhd3eb1b0_0
pytz                      2023.3.post1    py311haa95532_0
pywavelets                1.5.0           py311hd7041d2_0
pyyaml                    6.0.1                    pypi_0    pypi
qt-main                   5.15.2              h19c9488_10
ray                       2.10.0                   pypi_0    pypi
referencing               0.34.0                   pypi_0    pypi
requests                  2.31.0                   pypi_0    pypi
rpds-py                   0.18.0                   pypi_0    pypi
scikit-image              0.22.0          py311hb4ba03d_0
scipy                     1.12.0          py311h9f229c6_0
setuptools                68.2.2          py311haa95532_0
sip                       6.7.12          py311hd77b12b_0
six                       1.16.0             pyhd3eb1b0_1
snappy                    1.1.10               h6c2663c_1
snowballstemmer           2.2.0                    pypi_0    pypi
sphinx                    7.2.6                    pypi_0    pypi
sphinx-rtd-theme          2.0.0                    pypi_0    pypi
sphinxcontrib-applehelp   1.0.8                    pypi_0    pypi
sphinxcontrib-devhelp     1.0.6                    pypi_0    pypi
sphinxcontrib-htmlhelp    2.0.5                    pypi_0    pypi
sphinxcontrib-jquery      4.1                      pypi_0    pypi
sphinxcontrib-jsmath      1.0.1                    pypi_0    pypi
sphinxcontrib-qthelp      1.0.7                    pypi_0    pypi
sphinxcontrib-serializinghtml 1.1.10                   pypi_0    pypi
sqlite                    3.41.2               h2bbff1b_0
tbb                       2021.8.0             h59b6b97_0
tifffile                  2023.4.12       py311haa95532_0
tk                        8.6.12               h2bbff1b_0
tornado                   6.3.3           py311h2bbff1b_0
tzdata                    2024a                h04d1e81_0
urllib3                   2.2.1                    pypi_0    pypi
vc                        14.2                 h21ff451_1
vs2015_runtime            14.27.29016          h5e58377_2
wheel                     0.41.2          py311haa95532_0
xz                        5.4.6                h8cc25b3_0
zfp                       1.0.0                hd77b12b_0
zlib                      1.2.13               h8cc25b3_0
zstd                      1.5.5                hd43e919_0

[alexzwanenburg]

Tests on multiple platforms were not able to reproduce the error, see #70.

[alexzwanenburg]

I cannot reproduce the issue on my Windows installation. The issue is likely with ray:

• [Core] Ray initialization raising "AssignProcessToJobObject() failed" OS Error on Windows ray-project/ray#40867
• AssignProcessToJobObject() failed in ray.init()  ray-project/ray#21994

I have added an try-except block to capture OS errors that prevent ray instances from being to created. A warning is shown instead.