Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: ruff pt012 violations #16158

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Conversation

paverett
Copy link
Contributor

@paverett paverett commented Mar 5, 2024

Description

This pull request is to resolve all of the PT012 ruff violations in the astropy repository.

  • By checking this box, the PR author has requested that maintainers do NOT use the "Squash and Merge" button. Maintainers should respect this when possible; however, the final decision is at the discretion of the maintainer that merges the PR.

Copy link

github-actions bot commented Mar 5, 2024

Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

  • Do the proposed changes actually accomplish desired goals?
  • Do the proposed changes follow the Astropy coding guidelines?
  • Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
  • Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
  • Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
  • Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
  • Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
  • Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
  • At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

@pllim pllim added this to the v6.1.0 milestone Mar 5, 2024
Copy link
Member

@nstarman nstarman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @paverett for the PR! Some quick comments on cosmology and units.

@@ -1019,8 +1019,6 @@ class HASOde0SubClass(cosmo_cls):
def __init__(self, Ode0):
pass

_COSMOLOGY_CLASSES.pop(HASOde0SubClass.__qualname__, None)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These lines should not be removed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my testing of putting a print statement before and after the subclass init. I don't believe this line is getting called. It will error on the subclass init and never get to this line. And I can't move it out of the context manager or else we get an UnboundLocalError. I'm not sure the best way to keep this line in.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's an interesting issue. It's important to make sure that _COSMOLOGY_CLASSES does not contain HASOde0SubClass after the test. Perhaps for now you can put the line in a try: ... except: UnboundLocalError: ...?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to ensure _COSMOLOGY_CLASSES gets restored after the test then the best way to achieve that is with a yield fixture.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I see how to use a yield fixture here. From reading the documentation it seems like it is used in place of a return. But in this test I dont see a use for a return even from a helper function. The TypeError is raised as soon as the class is declared. We can't return the class definition.

In fact I don't even think the try block will doing anything. The class is declared in the with statement. We can't reference it anymore so the try will always go to the except block. Not sure the best way to proceed on this one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if I'm understanding it right. Everything before the yield happens before the test then when the test completes everything after the yield runs. So I could deep copy _COSMOLOGY_CLASSES before, then after the test runs set _COSMOLOGY_CLASSES back to the deep copy. Like nothing ever happened. However when I tried this broke other tests, raising Keyerrors in other tests like test_w0cdm.py and test_parameters.py and some others. All keyerrors happen while trying to cleanup by popping a test subclass from _COSMOLOGY_CLASSES. My approach must not be correct or I must be missing something.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should post the exact code you tried.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In astropy/cosmology/flrw/tests/conftest.py

import pytest
import copy
from astropy.cosmology import core #imports are where the rest of the imports are, just showing what I imported.

@pytest.fixture
def clean_cosmology_classes():
    original = copy.deepcopy(core._COSMOLOGY_CLASSES)
    yield
    core._COSMOLOGY_CLASSES = original

In astropy/cosmology/flrw/tests/test_base.py

    @pytest.mark.usefixtures("clean_cosmology_classes")
    def test_init_subclass(self, cosmo_cls):
        """Test initializing subclass, mostly that can't have Ode0 in init."""
        super().test_init_subclass(cosmo_cls)

        with pytest.raises(TypeError, match="subclasses of"):

            class HASOde0SubClass(cosmo_cls):
                def __init__(self, Ode0):
                    pass

Test ran pytest astropy/cosmology

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is best to replace the teardown part of your fixture (i.e. core._COSMOLOGY_CLASSES = original) with

core._COSMOLOGY_CLASSES.clear()
core._COSMOLOGY_CLASSES.update(original)

The latter avoids assignments, so it is guaranteed to only modify the items in _COSMOLOGY_CLASSES and not its identity.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha makes sense, thanks for the help!

with pytest.raises(ValueError), warnings.catch_warnings():
# ct, dex also raise warnings - irrelevant here.
warnings.simplefilter("ignore")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't have been moved. If ruff did this, then this should be reported upstream.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PT012 is not among rules Ruff can fix automatically.

Copy link
Contributor Author

@paverett paverett Mar 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved it. Not sure the best way to satisfy PT012 and keep it ignoring warnings. Right now I can think of three options.

  1. Add pytest.mark.filterwarning("ignore") to the function definition.
  2. Split the context managers out and have the with pytest.raises be the inside one.
  3. Add a helper function which contains the lines below the context manager.

Any other options?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like 1 and 4 (add a noqa).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't you just have nested with-blocks?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it complains about the warnings.simplefilter("ignore"). Well more that there are two lines under the raises statement.

Nested with blocks like this?

with pytest.raises(ValueError):
        with warnings.catch_warnings():
            # ct, dex also raise warnings - irrelevant here.
            warnings.simplefilter("ignore")
            u_format.VOUnit().parse(string)

If so I tried that and ruff didn't like it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this work if you change the order of the with-statements?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually this is not testing for an exception from the warnings calls (don't see how they could raise a ValueError, unless perhaps one passes an invalid string to simplefilter()), so I think they might be moved outside the pytest.raises altogether.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as it is inside catch_warnings - otherwise the filter does not get reset.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think this way around

with warnings.catch_warnings():
    # ct, dex also raise warnings - irrelevant here.
    warnings.simplefilter("ignore")
    with pytest.raises(ValueError):
        u_format.VOUnit().parse(string)

it should work correctly and hopefully be accepted by Ruff.

Copy link
Member

@eerovaher eerovaher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is very good to enforce PT012 because apparently we had tests that seem to have been written assuming that if there are multiple lines in a with-block that can raise an error then pytest.raises() checks all of them, when in fact it only checks the first error and the following lines in the block never get executed at all.

I do have some comments about the changes to the code, even though I did not do a thorough review.

Comment on lines 630 to 635
with pytest.raises(TypeError, match=r".* allowed .*"):
AiryDisk2DKernel(2, array=x)
with pytest.raises(TypeError, match=r".* allowed .*"):
Box1DKernel(2, array=x)
with pytest.raises(TypeError, match=r".* allowed .*"):
Box2DKernel(2, array=x)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should clearly be a parametrized test

Copy link
Contributor Author

@paverett paverett Mar 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. Should I move the Ring2DKernel to its own test? It takes 3 parameters when the rest only take 2. Not sure the best way to handle that in a parametrized test.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@WilliamJamieson, do you know why Ring2DKernel needs to have a different number of arguments than everything else?

Copy link
Contributor

@dhomeier dhomeier Mar 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has an inner and an outer radius, so does need two required parameters – could parametrise over a tuple size and then call testkernel(*size, array=x).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would addressing PT012 require editing this file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved

out = capsys.readouterr()[0]
out of the with statement in the test to satisfy PT012. When I ran pytest it uncovered a bug, the variable out was actually equal to pytest <version> and not fitscheck <version>. I added prog to the argument parser of this file so it would return with the program name instead of pytest. Same with the other three scripts.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not obvious to me that boilerplate code is the best solution to a minor problem that arises from the tests calling the main function directly instead of executing the code here as a stand-alone script. We could instead modify the tests so that they call the scripts as separate processes, which would be much more similar to their intended usage.

In any case if addressing PT012 reveals actual bugs then they should be fixed in separate pull requests so that the fixes could be backported.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something like import subprocess then a subprocess.popen("fitscheck --version")? Then check from there? Not sure if there is a module in astropy for calling scripts as separate processes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could instead modify the tests so that they call the scripts as separate processes, which would be much more similar to their intended usage.

I would rather we don't do that. Testing CLIs through function calls is standard practice, and has the benefit of forcing a consistent and testable design.

@paverett, are you able to open a separate PR with just this bug fix ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that's the best option. @eerovaher brings up a good point. The intended usage of a script is to run it on the command line so why should we not test it like its intended usage. It isn't a bug with the script just how we are calling it in the test. If we change the test then there is no problem.

Comment on lines 1299 to 1302
def add_data_and_flush(hdul):
hdul[0].data = np.arange(0, 1000, dtype="int64")
hdul.insert(1, fits.ImageHDU())
hdul.flush()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not obvious introducing a helper function like this is the correct thing to do.

Comment on lines 51 to 52
print(string)
with pytest.raises(ValueError):
print(string)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this test printing anything at all?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not entirely sure, I can remove it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think calling the __str__() or __repr__() methods should trigger the desired error without having to call the entire print() function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright ill try out the repr() method on these tests.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the point here was that in old versions of pytest, one would not see what string actually went wrong. I think the print statement can safely be removed (and I'm pretty sure str() and repr() are tested elsewhere).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, for some tests like the one above it would print just the enumeration of parameters like test_unit_grammar[strings9-unit9], so the actual string would add a bit of convenience, but for single parametrisation even that should not be necessary.

Comment on lines +677 to 679
with open(self.data("test0.fits"), "rb") as handle:
with pytest.raises(ValueError):
with fits.open(handle, mode="ostream") as _:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be better to use a single with-statement for all these context managers.

Copy link
Contributor

@WilliamJamieson WilliamJamieson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The failing tests are due to a revealed bug. These changes should fix that.

g1 = models.Gaussian1D(
[10.2, 10], mean=[3, 3.2], stddev=[0.23, 0.2], n_models=2
)
y1 = g1(self.x1, model_set_axis=False)
_ = fitter(g1, self.x1, y1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
_ = fitter(g1, self.x1, y1)
fitter(g1, self.x1, y1)

There is no use in even invoking the assignment here

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without the assignment, wouldn't this be removed by https://docs.astral.sh/ruff/rules/useless-expression/?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not trigger the issue when I run ruff locally with these changes.

theta=[0, 0],
n_models=2,
)
z = g2(self.x.flatten(), self.y.flatten())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
z = g2(self.x.flatten(), self.y.flatten())
x = (self.x.flatten(), self.x.flatten())
y = (self.y.flatten(), self.y.flatten())
z = g2(x, y)

There is a bug in this test which is revealed by these changes. This should fix that bug. The message value will have to change.

@@ -235,17 +235,17 @@ def test_nonlinear_lsqt_Nset_2d(self, fitter):
r"Input argument .* does not have the correct dimensions in .* for a model"
r" set with .*"
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
)
MESSAGE = r"Non-linear fitters can only fit one data set at a time"

Replace the whole message here.

theta=[0, 0],
n_models=2,
)
z = g2(self.x.flatten(), self.y.flatten())
_ = fitter(g2, self.x, self.y, z)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
_ = fitter(g2, self.x, self.y, z)
fitter(g2, x, y, z)

@WilliamJamieson
Copy link
Contributor

It is very good to enforce PT012 because apparently we had tests that seem to have been written assuming that if there are multiple lines in a with-block that can raise an error then pytest.raises() checks all of them, when in fact it only checks the first error and the following lines in the block never get executed at all.

I agree. This type of issue also points to us considering including the test code within the code coverage as the test files should theoretically have 100% coverage if we include all our various CI configurations. I think a coverage measurement may have also pointed out these areas.

Copy link
Member

@taldcroft taldcroft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine for io.ascii and time.

Comment on lines 644 to 651
if len(opt) == 1:
a = opt[0]
with pytest.raises(TypeError, match=r".* allowed .*"):
kernel(a, array=x)
else:
a, b = opt
with pytest.raises(TypeError, match=r".* allowed .*"):
kernel(a, b, array=x)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this work?

Suggested change
if len(opt) == 1:
a = opt[0]
with pytest.raises(TypeError, match=r".* allowed .*"):
kernel(a, array=x)
else:
a, b = opt
with pytest.raises(TypeError, match=r".* allowed .*"):
kernel(a, b, array=x)
with pytest.raises(TypeError, match=r".* allowed .*"):
kernel(*opt, array=x)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's much better, thanks!

Comment on lines 793 to 795
# while a good idea, this approach did not work; it actually writes to disk
# arr = np.memmap('file.np', mode='w+', shape=(512, 512, 512), dtype=complex)
# this just allocates the memory but never touches it; it's better:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks to me like these comments should be moved too.

@@ -45,7 +45,20 @@
AiryDisk2DKernel,
Ring2DKernel,
]

KERNEL_TYPES_WITH_ARGS = [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is parametrization for a single test then why is it written out as a module-level constant?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will remove the constant and use it in the parameterize statement.

Latitude(lon)

lon = Longitude(10, "deg")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is lon being defined again?

Comment on lines +63 to +68
assert (
exc_info.type is ValueError
and "format must be 'ascii.latex'" in exc_info.value.args[0]
or exc_info.type is IORegistryError
and "No writer defined for format" in exc_info.value.args[0]
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks very suspicious. Surely we know which error the code raises?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can raise either error. Removing one or the other from the with statment produces failing tests. Maybe there is a better way to combine them in the with raises statement but I haven't been able to come up with one.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be the case that the test needs to be redesigned, but that task is beyond the scope of this pull request.

@@ -1019,8 +1019,6 @@ class HASOde0SubClass(cosmo_cls):
def __init__(self, Ode0):
pass

_COSMOLOGY_CLASSES.pop(HASOde0SubClass.__qualname__, None)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to ensure _COSMOLOGY_CLASSES gets restored after the test then the best way to achieve that is with a yield fixture.

@@ -48,8 +48,8 @@ def test_unit_grammar(strings, unit):
"string", ["sin( /pixel /s)", "mag(mag)", "dB(dB(mW))", "dex()"]
)
def test_unit_grammar_fail(string):
string.__repr__()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we discussed to using the repr method instead of print, here: #16158 (comment)

If it is not necessary I can remove it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is no longer necessary

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, remove it.

with pytest.raises(ValueError), warnings.catch_warnings():
# ct, dex also raise warnings - irrelevant here.
warnings.simplefilter("ignore")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't you just have nested with-blocks?

@@ -2243,7 +2243,6 @@ def test_slogdet(self):
# TODO: Could be supported if we had a natural logarithm unit.
with pytest.raises(TypeError):
logdet = np.linalg.slogdet(self.q)
assert hasattr(logdet, "unit")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this removal change the test?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't see how hasattr could ever raise TypeError on the first parameter, so it shouldn't – but slodget is returning a tuple, so that assert seems out of scope anyway.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And if the np.linalg.slogdet() call raises an error then logdet isn't defined anyways. So the deletion of the line is indeed correct.

Comment on lines +1739 to 1742
report_length = 1024
real_length = 1023
with pytest.raises(urllib.error.ContentTooShortError):
report_length = 1024
real_length = 1023
download_file(TESTURL, cache=False)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do report_length and real_length even do anything?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, removing them breaks the test. Seems like they are used up above in MockURL class.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is very surprising that just setting local variables like that actually has an effect on the tests, but fixing this would go beyond the scope of this pull request.

Comment on lines 797 to 799
def init_wcs_and_alter(hdr):
w = wcs.WCS(hdr, _do_set=False)
w.all_pix2world([[536.0, 894.0]], 0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the helper function needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No just had to flip flop the with statements.

@eerovaher
Copy link
Member

I can see that in many cases the code I commented on is a rewrite of already existing code, I'm just not convinced that the existing code is always worth rewriting.

Comment on lines 869 to 870
text = "A\tB\tC\nN\tS\tN\n4\tb\ta" # C column contains non-numeric data
with pytest.raises(ValueError) as e:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
text = "A\tB\tC\nN\tS\tN\n4\tb\ta" # C column contains non-numeric data
with pytest.raises(ValueError) as e:
# C column contains non-numeric data
with pytest.raises(ValueError) as e:
read_rdb("A\tB\tC\nN\tS\tN\n4\tb\ta")

Agreed; temporary variables are unnecessary here, and so is forcing the line split with trailing comma – the comment could better be placed on a separate line (be it inside or outside the context, dunno how picky PT012 is there).

Comment on lines 646 to 650
with (
pytest.raises(ValueError),
open(self.data("test0.fits")) as handle,
fits.open(handle) as _,
):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So is the nested with construct disallowed by PT012 (cf. also https://github.com/astropy/astropy/pull/16158/files#r1530684773)?
I was not familiar with that form of using with; while it is kind of neat, it does not make the nested nature very clear and breaks symmetry with the reading operations that are not raising.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is allowed to nest once with one statement inside the inner statement. Once a second nesting statement is added it complains.

Allowed

with pytest.raises(ValueError), open(self.data("test0.fits")) as handle:
    with fits.open(handle) as _:
        pass

Not Allowed

with pytest.raises(ValueError):
    with open(self.data("test0.fits")) as handle:
        with fits.open(handle) as _:
            pass

As for these changes, @eerovaher thought they might be better with a single with statement. I thought it made sense and made the change. #16158 (comment)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My suggestion might not have been a good one in this case. When I had a quick look at the tests I saw a bunch of nested with-statements and thought that it might be good to rewrite them as a single multi-line with-statement (which is new syntax in Python 3.10). However, the tests here are checking that some specific with-statements themselves are raising errors and having nested with-statements makes that clearer. So now I would recommend rewriting the nested with-statements if and only if the error being checked does not come from one of them.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there seem to be some open issues on Ruff handling those cases right, and the flake8 rule only lists the one-level nesting, not making clear whether this should apply recursively. Whatever flake8 says, the rationale for not missing an exception from the intended statement (here the fits.open(handle)) does not apply even for 3 or more context levels, because that statement simply could not be reached and thus not tested if any of the outer withs already failed.

@@ -622,22 +635,11 @@ def test_kernel2d_initialization(self):
with pytest.raises(TypeError):
Kernel2D()

def test_array_keyword_not_allowed(self):
@pytest.mark.parametrize(["kernel", "opt"], KERNEL_TYPES_WITH_ARGS)
def test_array_keyword_not_allowed(self, kernel, opt):
"""
Regression test for issue #10439
"""
x = np.ones([10, 10])
with pytest.raises(TypeError, match=r".* allowed .*"):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
with pytest.raises(TypeError, match=r".* allowed .*"):
with pytest.raises(TypeError, match=r"Array argument not allowed for kernel.*"):

seems GitHub is not keeping up with current code version at all today; but if this is still valid, might as well put a fuller message here for clarity.

Copy link
Contributor

@mhvk mhvk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some inline comments (somehow seem to have started a review, so some are a bit old)

Comment on lines 51 to 52
print(string)
with pytest.raises(ValueError):
print(string)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the point here was that in old versions of pytest, one would not see what string actually went wrong. I think the print statement can safely be removed (and I'm pretty sure str() and repr() are tested elsewhere).

@@ -48,8 +48,8 @@ def test_unit_grammar(strings, unit):
"string", ["sin( /pixel /s)", "mag(mag)", "dB(dB(mW))", "dex()"]
)
def test_unit_grammar_fail(string):
string.__repr__()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is no longer necessary

with pytest.raises(ValueError), warnings.catch_warnings():
# ct, dex also raise warnings - irrelevant here.
warnings.simplefilter("ignore")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as it is inside catch_warnings - otherwise the filter does not get reset.

@astrofrog astrofrog modified the milestones: v6.1.0, v7.0.0 Apr 4, 2024
@eerovaher
Copy link
Member

Different astropy sub-packages have different maintainers, so if a pull request edits many sub-packages then it needs approvals from many people. If there are too many people involved then it can easily happen that everybody thinks that someone else is reviewing, in which case no one does. It looks to me like that's exactly what is happening here.

It would be good to get the updates merged because this pull request is fixing many mistakes in tests, but the best way to achieve that might be to open separate pull requests for the separate sub-packages. Then it would be clear whose responsibility it is to review each such pull request.

@pllim
Copy link
Member

pllim commented Apr 15, 2024

Looks like @WilliamJamieson requested changes so maybe he can at least re-review?

@dhomeier
Copy link
Contributor

Different astropy sub-packages have different maintainers, so if a pull request edits many sub-packages then it needs approvals from many people. If there are too many people involved then it can easily happen that everybody thinks that someone else is reviewing, in which case no one does. It looks to me like that's exactly what is happening here.

Not sure if they expect that the reviews are already done by others, as the review scope should be within the respective sub-packages, but some of their maintainers simply don't have that much time (and I know of some who have already ceded code ownership because they felt overwhelmed by the Ruff PR reviews). Here we seem to only have explicit approval for io.ascii and time, since finding the right test structure is non-trivial in many cases, but for those sub-packages where maintainers have already responded and suggested changes, I think it's worth somehow cherry-picking the commits here and try to get them merged; moving the remaining sub-packages' changes to separate PRs.

Copy link
Contributor

@nden nden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

modeling and wcs look good

Copy link
Member

@nstarman nstarman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cosmology also looks good. Thanks @paverett !

Copy link
Member

@eerovaher eerovaher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

coordinates needs to be updated.

Longitude(lat)

lat = Latitude(10, "deg")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line needlessly duplicates line 842.

Suggested change
lat = Latitude(10, "deg")

@dhomeier dhomeier mentioned this pull request Apr 19, 2024
32 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Review in progress
Development

Successfully merging this pull request may close these issues.

None yet