Deprecation warnings #297
Comments
|
"elementwise comparison failed; this will raise an error in the future." is usually because of a type mismatch. For instance: import numpy as np
arr = np.arange(5) # 0, 1, 2, 3, 4 dtype = np.int64
"1" in arr # returns False because types don't match and puts out a deprecation warning
arr2 = np.array(['0', '1', '2', '3', '4'])
np.array_equal(arr, arr2) # returns False and puts out deprecation warningI don't see what would cause this in the ES algorithm though. It only seems to get reported in the tests as well. If you run the ES example it won't report any warnings. " VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray" This one is because of a bug in NiaPy/NiaPy/algorithms/modified/jade.py Line 53 in dc1cd45 Here the x list elements are a mix of scalars and ndarrays, so a matrix cannot be constructed from it, so asarray casts all elements to object type. This is also what causes the other warning about failed comparissons in that test. This test can be skipped since the function isn't used anywhere. |
|
Have a good day everyone, I am happy that the current repository is getting updated. NiaPy is a very useful library for many people. python3 -m pytest -k "_es" -W errorYou could do the same for JADE with the command shown below: python3 -m pytest -k "_jade" -W errorIn this way, each warning would result in an error. |
|
Some additional warnings appeared when building our docx. /builddir/build/BUILD/NiaPy-2.0.0rc13/NiaPy/algorithms/basic/es.py:docstring of NiaPy.algorithms.basic.EvolutionStrategy1p1.initPopulation:9: WARNING: Unexpected indentation. /builddir/build/BUILD/NiaPy-2.0.0rc13/NiaPy/algorithms/basic/es.py:docstring of NiaPy.algorithms.basic.EvolutionStrategy1p1.runIteration:23: WARNING: Unexpected indentation. /builddir/build/BUILD/NiaPy-2.0.0rc13/NiaPy/algorithms/basic/es.py:docstring of NiaPy.algorithms.basic.EvolutionStrategyML.initPopulation:7: WARNING: Enumerated list ends without a blank line; unexpected unindent. /builddir/build/BUILD/NiaPy-2.0.0rc13/NiaPy/algorithms/other/rs.py:docstring of NiaPy.algorithms.other.RandomSearch.setParameters:5: WARNING: Unexpected indentation. /builddir/build/BUILD/NiaPy-2.0.0rc13/NiaPy/benchmarks/sphere.py:docstring of NiaPy.benchmarks.Sphere:1: WARNING: duplicate object description of NiaPy.benchmarks.Sphere, other instance in api/benchmarks, use :noindex: for one of them /builddir/build/BUILD/NiaPy-2.0.0rc13/NiaPy/benchmarks/sphere.py:docstring of NiaPy.benchmarks.Sphere.Name:1: WARNING: duplicate object description of NiaPy.benchmarks.Sphere.Name, other instance in api/benchmarks, use :noindex: for one of them /builddir/build/BUILD/NiaPy-2.0.0rc13/NiaPy/benchmarks/sphere.py:docstring of NiaPy.benchmarks.Sphere.init:1: WARNING: duplicate object description of NiaPy.benchmarks.Sphere.init, other instance in api/benchmarks, use :noindex: for one of them /builddir/build/BUILD/NiaPy-2.0.0rc13/NiaPy/benchmarks/sphere.py:docstring of NiaPy.benchmarks.Sphere.function:1: WARNING: duplicate object description of NiaPy.benchmarks.Sphere.function, other instance in api/benchmarks, use :noindex: for one of them /builddir/build/BUILD/NiaPy-2.0.0rc13/NiaPy/benchmarks/sphere.py:docstring of NiaPy.benchmarks.Sphere.latex_code:1: WARNING: duplicate object description of NiaPy.benchmarks.Sphere.latex_code, other instance in api/benchmarks, use :noindex: for one of them /builddir/build/BUILD/NiaPy-2.0.0rc13/NiaPy/runner.py:docstring of NiaPy.Runner:1: WARNING: duplicate object description of NiaPy.Runner, other instance in api/niapy, use :noindex: for one of them |
|
I got to fix some warnings, which are present in sisco0@00aa68b and sisco0@e5c753f. NiaPy/NiaPy/algorithms/modified/jade.py Lines 17 to 54 in dc1cd45 It seems that some elements are numbers, while others are arrays in the I attach the output of > /home/thebittendonut/dev/NiaPy/NiaPy/algorithms/modified/jade.py(55)CrossRandCurr2Pbest()
-> import pdb; pdb.set_trace()
(Pdb) p x
[-91.17912633708372, -22.72754052518438, 86.4972937536084, array([37.6929134 , 18.0288306 , 66.23389883, 69.48774771, 12.70002755,
78.95772088, 80.91229664, 57.74606603, 46.52453118, 40.05271575]), 98.31891564794938, array([-18.000223 , -34.0996604 , -77.98610639, -76.0043742 ,
-97.53318287, -54.40052513, -20.29343077, -78.85750992,
-13.79769366, -11.57254025]), array([-27.72220261, -50.76700597, 17.60615978, -32.09545235,
-17.55285624, 29.06832172, 6.29216357, 10.45664726,
-44.79441646, -41.08245657]), -3.609677182164205, array([ -45.76234021, -105.20215609, -37.42246389, -88.4825519 ,
-99.20263108, -62.21999831, -74.78654291, -19.64553972,
-55.51563055, -5.54763639]), -40.640832103704106]These should be modified to be a Is not the error related to these lines attached below? NiaPy/NiaPy/algorithms/modified/jade.py Lines 42 to 44 in dc1cd45 In my opinion, it seems that this function should get the population scores and do the After the research, JADE was fixed as stated in sisco0@c31a243 I would continue to fix these warnings, after that the Pull Request would be done. |
|
Currently I am stuck at the ASO tests, as it hangs at the part shown: ❯ python3 -m pytest -k "_aso" -W error --verbose
===================================================================================== test session starts =====================================================================================
platform linux -- Python 3.8.5, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /home/thebittendonut/.local/share/virtualenvs/NiaPy-ZNxshtqu/bin/python3
cachedir: .pytest_cache
rootdir: /home/thebittendonut/dev/NiaPy
collected 966 items / 938 deselected / 28 selected
NiaPy/tests/test_aso.py::AlgorithmTestCase::test_algorithm_get_parameters_fine PASSED [ 3%]
NiaPy/tests/test_aso.py::AlgorithmTestCase::test_algorithm_info_fine PASSED [ 7%]
NiaPy/tests/test_aso.py::AlgorithmTestCase::test_algorithm_run PASSED [ 10%]
NiaPy/tests/test_aso.py::AlgorithmTestCase::test_algorithm_type_parameters PASSED [ 14%]
NiaPy/tests/test_aso.py::ASOTestCase::test_algorithm_get_parameters_fine PASSED [ 17%]
NiaPy/tests/test_aso.py::ASOTestCase::test_algorithm_info_fine PASSED [ 21%]
NiaPy/tests/test_aso.py::ASOTestCase::test_algorithm_run PASSED [ 25%]
NiaPy/tests/test_aso.py::ASOTestCase::test_algorithm_type_parameters PASSED [ 28%]
NiaPy/tests/test_aso.py::ASOTestCase::test_parameter_types PASSED [ 32%]
NiaPy/tests/test_aso.py::ASOElitismTestCase::test_algorithm_get_parameters_fine PASSED [ 35%]
NiaPy/tests/test_aso.py::ASOElitismTestCase::test_algorithm_info_fine PASSED [ 39%]
NiaPy/tests/test_aso.py::ASOElitismTestCase::test_algorithm_run PASSED [ 42%]
NiaPy/tests/test_aso.py::ASOElitismTestCase::test_algorithm_type_parameters PASSED [ 46%]
NiaPy/tests/test_aso.py::ASOElitismTestCase::test_custom_works_fine |
|
@sisco0 Running with -Werror will hang because the algorithm test case runs 2 identical algorithm instances in separate threads, and the exception isn't handled in the thread. It should be handled in the Algorithm class itself, but that feature isn't implemented correctly at the moment. The run method should catch any exception and save it in self.exception I'm assuming, and then you could call Algorithm.bad_run() to check if an error was thrown. EDIT: def run(self, task):
r"""Start the optimization.
Args:
task (Task): Optimization task.
Returns:
Tuple[numpy.ndarray, float]:
1. Best individuals components found in optimization process.
2. Best fitness value found in optimization process.
See Also:
* :func:`NiaPy.algorithms.Algorithm.runTask`
"""
try:
r = self.runTask(task)
return r[0], r[1] * task.optType.value
except (FesException, GenException, TimeException, RefException):
return task.x, task.x_f * task.optType.value
except Exception as e:
self.exception = e
return None, None
``` |
|
Thank you for the brilliant answer @zStupan , as you wisely pointed out the ASO test is not working when using Update: Thanks for the help given, I could finally fix that warning. In order to run all the tests in parallel I used the recommendation at here. Installed the package and used the |
|
@sisco0 Awesome! @GregaVrbancic Could you maybe try adding that pytest extension to the CI, so the test would execute quicker on pull requests? |
|
The complete tests suite seem to be passing successfully without any warning now. Log of parallel tests============================= test session starts ============================== [gw0] linux Python 3.8.5 cwd: /home/thebittendonut/dev/NiaPy [gw1] linux Python 3.8.5 cwd: /home/thebittendonut/dev/NiaPy [gw2] linux Python 3.8.5 cwd: /home/thebittendonut/dev/NiaPy [gw3] linux Python 3.8.5 cwd: /home/thebittendonut/dev/NiaPy [gw4] linux Python 3.8.5 cwd: /home/thebittendonut/dev/NiaPy [gw5] linux Python 3.8.5 cwd: /home/thebittendonut/dev/NiaPy [gw6] linux Python 3.8.5 cwd: /home/thebittendonut/dev/NiaPy [gw7] linux Python 3.8.5 cwd: /home/thebittendonut/dev/NiaPy [gw8] linux Python 3.8.5 cwd: /home/thebittendonut/dev/NiaPy [gw9] linux Python 3.8.5 cwd: /home/thebittendonut/dev/NiaPy [gw10] linux Python 3.8.5 cwd: /home/thebittendonut/dev/NiaPy [gw11] linux Python 3.8.5 cwd: /home/thebittendonut/dev/NiaPy [gw12] linux Python 3.8.5 cwd: /home/thebittendonut/dev/NiaPy [gw13] linux Python 3.8.5 cwd: /home/thebittendonut/dev/NiaPy [gw14] linux Python 3.8.5 cwd: /home/thebittendonut/dev/NiaPy [gw15] linux Python 3.8.5 cwd: /home/thebittendonut/dev/NiaPy [gw1] Python 3.8.5 (default, Jan 27 2021, 15:41:15) -- [GCC 9.3.0] [gw3] Python 3.8.5 (default, Jan 27 2021, 15:41:15) -- [GCC 9.3.0] [gw0] Python 3.8.5 (default, Jan 27 2021, 15:41:15) -- [GCC 9.3.0] [gw2] Python 3.8.5 (default, Jan 27 2021, 15:41:15) -- [GCC 9.3.0] [gw6] Python 3.8.5 (default, Jan 27 2021, 15:41:15) -- [GCC 9.3.0] [gw5] Python 3.8.5 (default, Jan 27 2021, 15:41:15) -- [GCC 9.3.0] [gw4] Python 3.8.5 (default, Jan 27 2021, 15:41:15) -- [GCC 9.3.0] [gw7] Python 3.8.5 (default, Jan 27 2021, 15:41:15) -- [GCC 9.3.0] [gw8] Python 3.8.5 (default, Jan 27 2021, 15:41:15) -- [GCC 9.3.0] [gw9] Python 3.8.5 (default, Jan 27 2021, 15:41:15) -- [GCC 9.3.0] [gw10] Python 3.8.5 (default, Jan 27 2021, 15:41:15) -- [GCC 9.3.0] [gw11] Python 3.8.5 (default, Jan 27 2021, 15:41:15) -- [GCC 9.3.0] [gw12] Python 3.8.5 (default, Jan 27 2021, 15:41:15) -- [GCC 9.3.0] [gw13] Python 3.8.5 (default, Jan 27 2021, 15:41:15) -- [GCC 9.3.0] [gw14] Python 3.8.5 (default, Jan 27 2021, 15:41:15) -- [GCC 9.3.0] [gw15] Python 3.8.5 (default, Jan 27 2021, 15:41:15) -- [GCC 9.3.0] scheduling tests via LoadScheduling NiaPy/tests/test_aba.py::ABATestCase::test_algorithm_type_parameters |
|
I created the Pull Request attached above and updated it in order to pass flake8 linting test. Added to the aforementioned, I would like to thank @zStupan for guiding me through the debugging process. |
|
@sisco0 No problem, always happy to help. The JADE and SADE tests,are skipped because the algorithms aren't implemented yet. |
|
I have been thinking about the parallel CI workflow and I came to a solution I would like to show you, which is present at my forked repository https://github.com/sisco0/NiaPy/tree/faster-ci . If reducing the CI time would be useful in this way, while uploading a coverage report artifact, please inform me to create the corresponding Pull Request @firefly-cpp @GregaVrbancic . These are the timing results for (Linux, Windows) and Python 3.8. |
@sisco0 thanks for contributing. Can you please elaborate a bit more on how the testing is quicker (looking at the first screenshot for Python 2.7 on Ubuntu) since the test it is just split into multiple smaller parts? But when you sum up the consumed time of those smaller parts it exceeds the time consumed in our current workflow (for example: https://github.com/NiaOrg/NiaPy/runs/2226732631). But I do agree, that quicker CI would be awesome. First, we could increase the number of max-parallel to 20. If I am not mistaken this is the maximum value available for our project. I would suggest, we improve the workflow step by step and in the process monitor if everything works as expected. |
|
Additionally, if making any changes to the workflow, please make sure you only update the main.yml. The other definitions are for automatic publishing new releases on pypi and conda and are currently working perfectly. |
|
I understand @GregaVrbancic. Tomorrow I would try to run some tests and hopefully I could return with a solution where the sum of minutes be the same or lower using a parallel approach. |
|
Deprecation warnings were fixed in #299, closing this issue. |
Hi everyone!
During the build of NiaPy rpm package, pytest reported several deprecation warnings.
I am attaching a partial report created by pytest.
=============================== warnings summary ===============================
NiaPy/tests/test_aso.py::ASOElitismTestCase::test_custom_works_fine
NiaPy/tests/test_aso.py::ASOElitismTestCase::test_griewank_works_fine
NiaPy/tests/test_fa.py::FATestCase::test_griewank_works_fine
NiaPy/tests/test_fa.py::FATestCase::test_works_fine
NiaPy/tests/test_jade.py::CrossRandCurr2pbestTestCase::test_function_fine
/usr/lib/python3.9/site-packages/numpy/core/_asarray.py:83: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
return array(a, dtype, copy=False, order=order)
NiaPy/tests/test_es.py: 16660 warnings
/builddir/build/BUILD/NiaPy-2.0.0rc13/NiaPy/algorithms/basic/es.py:329: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.
if e not in c: k += 1
NiaPy/tests/test_jade.py: 200 warnings
/usr/lib/python3.9/site-packages/numpy/core/numeric.py:2378: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.
return bool(asarray(a1 == a2).all())
-- Docs: https://docs.pytest.org/en/stable/warnings.html
The text was updated successfully, but these errors were encountered: