Future proofing: treat most warnings as errors in pytest, and fix existing ones #3380

neutrinoceros · 2021-06-26T09:13:22Z

PR Summary

fix #3379
Though independent, this is complementary with #3376.

An important note is that there are some internal deprecation warnings with no obvious solution yet, so I'm adding them to the ignore-list in conftest.py but they will need to be addressed in future releases. Specifically, I'm talking about fields ("all", "px") and ("gas", "px") (respectively "py", "dpx" and "dpy"), that are used internally in slice selectors, but never with explicit field/particle types. Getting to a stable solution here requires much more thought and design that what I'm solving here so I'll defer it to future works, and in any case it's not as urgent as other deprecation warnings because it's internal so it won't break haphazardly.

TODO

pass CI
document why each ignored warning gets a pass (comments)

conftest.py

yt/visualization/plot_window.py

neutrinoceros · 2021-06-28T14:29:39Z

There remains only one failure in the py38_unittest job but I'm struggling to reproduce it locally. Any help would be greatly appreciated.
edit: done ! ✅
Hopefully this is the last thing that needs solving here, though I'm writing this before I can see the results from the py38 job on Jenkins.

neutrinoceros · 2021-06-28T15:39:38Z

setup.cfg

@@ -47,7 +47,7 @@ install_requires =
    toml>=0.10.2
    tqdm>=3.4.0
    unyt>=2.8.0
-python_requires = >=3.6
+python_requires = >=3.6,<3.12


To anyone wondering: the reasoning here is that the distutils builtin package is slanted for removal in Python 3.12 (which should be released around October 2023), but it is currently used in several places in yt, making it de facto incompatible with Python 3.12
Solving this doesn't seem like a priority but I'll document it as an issue !

matthewturk

I'd like to see one of the changes reworked, but otherwise this looks good to me.

matthewturk · 2021-06-29T17:18:07Z

conftest.py

+        #
+        # >>> warnings from wrong values passed to numpy
+        # these should normally be curated out of the test suite but they are too numerous
+        # to deal with in a reasonable time at the moment.


I think some like this may arise from field detection.

oh, good call ! I'll keep this in mind if I ever feel daring enough to fix these :)

yt/utilities/fortran_utils.py

yt/utilities/tests/test_cosmology.py

yt/visualization/volume_rendering/lens.py

yt/data_objects/construction_data_containers.py

yt/data_objects/index_subobjects/grid_patch.py

neutrinoceros · 2021-06-30T14:29:16Z

pre-commit.ci autofix

…ill make use of the distutils package, targetted for removal in Python 3.12)

…g can be done about it

…nals

Co-authored-by: Corentin Cadiou <contact@cphyc.me>

for more information, see https://pre-commit.ci

neutrinoceros added enhancement Making something better infrastructure Related to CI, versioning, websites, organizational issues, etc yt-4.0 feature targeted for the yt-4.0 release labels Jun 26, 2021

neutrinoceros force-pushed the future_proofing branch 12 times, most recently from 9fd63c8 to 2105651 Compare June 28, 2021 10:40

neutrinoceros mentioned this pull request Jun 28, 2021

Ambiguous field/particle type: special pixel-fields #3381

Closed

neutrinoceros commented Jun 28, 2021

View reviewed changes

conftest.py Outdated Show resolved Hide resolved

neutrinoceros commented Jun 28, 2021

View reviewed changes

conftest.py Outdated Show resolved Hide resolved

neutrinoceros force-pushed the future_proofing branch 3 times, most recently from 45ae7dd to 5b5e4b5 Compare June 28, 2021 13:46

neutrinoceros commented Jun 28, 2021

View reviewed changes

yt/visualization/plot_window.py Outdated Show resolved Hide resolved

neutrinoceros force-pushed the future_proofing branch from f9cbb94 to 58954f3 Compare June 28, 2021 15:30

neutrinoceros commented Jun 28, 2021

View reviewed changes

neutrinoceros mentioned this pull request Jun 28, 2021

MNT: migrate away from distutils #3384

Open

neutrinoceros force-pushed the future_proofing branch from 58954f3 to 0216e48 Compare June 28, 2021 18:13

matthewturk reviewed Jun 29, 2021

View reviewed changes

cphyc approved these changes Jun 30, 2021

View reviewed changes

yt/visualization/volume_rendering/lens.py Show resolved Hide resolved

yt/data_objects/construction_data_containers.py Show resolved Hide resolved

yt/data_objects/index_subobjects/grid_patch.py Outdated Show resolved Hide resolved

neutrinoceros mentioned this pull request Jun 30, 2021

CI: deactivate conda update as a workaround to external failure #3396

Merged

neutrinoceros and others added 23 commits June 30, 2021 20:44

CI: treat most warnings as errors in pytest

32f3048

add a reference to open gh issue

a93c812

add a comment to soon-to-be useless github workflow

18de5df

loosen requirements for astropy and h5py

9a5eec1

add an upper limit to compatible Python versions (at the moment we st…

54c0186

…ill make use of the distutils package, targetted for removal in Python 3.12)

future-proof against Python 3.10 + more_itertools

b764530

fix a deprecation warning (logging.Logger.warn is deprecated)

7c65bdc

fix a fragility in IO operations in a test

d4ef645

drop numpy.mat as the matrix class is pending deprecation

a6990ec

fix a deprecation warning from numpy

9417d16

fix a deprecation warning from unyt

e76a21a

exclude deprecated colormap from matploltib in show_colormaps

34fa305

rewrite a test to check that a deprecation warning is raised where due

7b5ac4a

use a tmp dir as test_data_dir for load tests

c2f13b4

fix a deprecation warning from numpy

3da236c

fix a deprecation warning from numpy

292b139

avoid creating an ndarray from ragged nested sequences

7731aab

change an internal runtime warning into a logger warning since nothin…

d7e1b21

…g can be done about it

simplify logic in read_vector function

e02ad04

bump maximal support mpl version

858b2b9

restore the spirit of np.mat without using deprecated api in VR inter…

f011699

…nals

Update yt/data_objects/index_subobjects/grid_patch.py

b3a14b5

Co-authored-by: Corentin Cadiou <contact@cphyc.me>

[pre-commit.ci] auto fixes from pre-commit.com hooks

7a7eb8e

for more information, see https://pre-commit.ci

neutrinoceros force-pushed the future_proofing branch from 8dc5fd6 to 7a7eb8e Compare June 30, 2021 18:44

matthewturk merged commit 15d22b5 into yt-project:main Jul 1, 2021

neutrinoceros deleted the future_proofing branch July 1, 2021 09:31

This was referenced Jul 1, 2021

Support for Python versions for upcoming 4.0 release #3070

Closed

MAINT: readd firefly_api to full install target #3419

Closed

neutrinoceros mentioned this pull request Aug 31, 2021

MNT: migrate away from packaging.version.parse #3494

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Future proofing: treat most warnings as errors in pytest, and fix existing ones #3380

Future proofing: treat most warnings as errors in pytest, and fix existing ones #3380

neutrinoceros commented Jun 26, 2021 •

edited

neutrinoceros commented Jun 28, 2021 •

edited

neutrinoceros Jun 28, 2021

matthewturk left a comment

matthewturk Jun 29, 2021

neutrinoceros Jun 29, 2021

neutrinoceros commented Jun 30, 2021

Future proofing: treat most warnings as errors in pytest, and fix existing ones #3380

Future proofing: treat most warnings as errors in pytest, and fix existing ones #3380

Conversation

neutrinoceros commented Jun 26, 2021 • edited

PR Summary

PR Summary

TODO

neutrinoceros commented Jun 28, 2021 • edited

neutrinoceros Jun 28, 2021

Choose a reason for hiding this comment

matthewturk left a comment

Choose a reason for hiding this comment

matthewturk Jun 29, 2021

Choose a reason for hiding this comment

neutrinoceros Jun 29, 2021

Choose a reason for hiding this comment

neutrinoceros commented Jun 30, 2021

neutrinoceros commented Jun 26, 2021 •

edited

neutrinoceros commented Jun 28, 2021 •

edited