Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Raise warnings as exceptions #317

Merged
merged 18 commits into from
Jun 1, 2022
Merged

Conversation

bsipocz
Copy link
Member

@bsipocz bsipocz commented Apr 28, 2022

This is to make the testing more robust in pyvo itself.

First commit just fixes the heading in setup.cfg which uncovers a lot of issues on its own already, will enable the warning raising once these are all fixed.

to close #307

The fix to the config file also fixes #306 (though most doctests will also need remote-data enabled to run)

@bsipocz
Copy link
Member Author

bsipocz commented Apr 28, 2022

OK, so for the most recent commits locally all tests and docs build pass, without remote-data turned on.

@bsipocz bsipocz modified the milestones: v1.4, v1.3.1 Apr 28, 2022
@codecov
Copy link

codecov bot commented Apr 28, 2022

Codecov Report

Merging #317 (90e07dc) into main (5830167) will increase coverage by 1.22%.
The diff coverage is 100.00%.

❗ Current head 90e07dc differs from pull request most recent head cbe19a9. Consider uploading reports for the commit cbe19a9 to get more accurate results

@@            Coverage Diff             @@
##             main     #317      +/-   ##
==========================================
+ Coverage   75.46%   76.68%   +1.22%     
==========================================
  Files          44       44              
  Lines        5123     5121       -2     
==========================================
+ Hits         3866     3927      +61     
+ Misses       1257     1194      -63     
Impacted Files Coverage Δ
pyvo/dal/adhoc.py 66.58% <ø> (ø)
pyvo/dal/mimetype.py 82.92% <ø> (+12.19%) ⬆️
pyvo/dal/scs.py 68.57% <ø> (ø)
pyvo/dal/sia.py 65.44% <ø> (ø)
pyvo/dal/sia2.py 77.08% <ø> (ø)
pyvo/dal/sla.py 64.54% <ø> (ø)
pyvo/dal/ssa.py 56.00% <ø> (ø)
pyvo/dal/tap.py 73.37% <ø> (ø)
pyvo/io/vosi/endpoint.py 87.23% <100.00%> (+0.33%) ⬆️
pyvo/dal/exceptions.py 76.38% <0.00%> (-0.64%) ⬇️
... and 8 more

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@bsipocz bsipocz force-pushed the tests_raise_warnings branch 2 times, most recently from fe447df to bc2e70a Compare April 28, 2022 23:17
@bsipocz bsipocz marked this pull request as ready for review April 28, 2022 23:17
@bsipocz
Copy link
Member Author

bsipocz commented Apr 28, 2022

The doctests in the narrative docs will require enabling remote-data testing in CI, I would rather do that separately as a follow-up, but if you prefer, I can add the changes to this one.

@bsipocz bsipocz force-pushed the tests_raise_warnings branch 3 times, most recently from 9ab2c84 to b134048 Compare April 29, 2022 04:28

>>> url = row.getdataurl()
>>> fileobj = row.getdataset()
>>> obj = row.getdataobj()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This last one is failing for me. Could point to a bug.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opened #324 for it, and commented out the line for now.

pyvo/dal/tests/test_datalink.py Outdated Show resolved Hide resolved
pyvo/dal/tests/test_tap.py Show resolved Hide resolved
tox.ini Show resolved Hide resolved
@msdemlei
Copy link
Contributor

msdemlei commented May 10, 2022 via email

Copy link
Contributor

@andamian andamian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bsipocz - does this PR raise exceptions or it's just preparing for it? I assume that tests need to run in pedantic mode, correct?

conftest.py Outdated
# Note that we don't need to change the environment variables back or remove
# them after testing, because they are only changed for the duration of the
# Python process, and this configuration only matters if running pytest
# directly, not from e.g. an IPython session.

from astropy.utils.iers import conf as iers_conf

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with iers and I don't know why this is necessary. Can it be documented if someone asks?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iers is a huge pile of pain.
This is a workaround to make it avoid trying to download the table when running the tests locally (it would error out with remote data stuff).

So, yes, I can dig up some explanation about this, I'm pretty sure I inherited the workaround from other places.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be enough to move the comment from the line below to before the import?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need a long explanation. Something that will help us remember what is all about and possibly trace it, so we, or other maintainers after us can tell when it's safe to remove or change it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is my last comment. I could merge it afterwards unless you want Tom's feedback as well.

assert len(service.get_job_list(phases=['EXECUTING'], last=3,
after=datetime.datetime.now())) == 6

with pytest.warns(UnknownElementWarning):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should not happen. I'm currently working to find a fix and might remove it in the future.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #331. It's a race now... :-)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the fix. I'll remove this workaround and rebase this PR once that other one is merged.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't have to do that. If this is ready we can merge it first and I'll deal with this in the other PR. That's why I said it's a race... :-)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With #332 now merged, could you please remove this exception? Thanks

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeap, I already did -1min of your comment :) (as I was just chatting with Tom about this)

@bsipocz
Copy link
Member Author

bsipocz commented May 27, 2022

does this PR raise exceptions or it's just preparing for it? I assume that tests need to run in pedantic mode, correct?

No, we're raising all exceptions, except the ones explicitly listed in setup.cfg. The aim is to have only the bare minimum of them in there, and handle all the rest in the code or tests.

@bsipocz
Copy link
Member Author

bsipocz commented May 27, 2022

@andamian - The one big item missing in this PR is to add a remote-data CI job (#330). That would be a big PR as it will need to fix all doctests as right now most of them are skipped as the examples are reaching out to remote services.

@andamian
Copy link
Contributor

@andamian - The one big item missing in this PR is to add a remote-data CI job (#330). That would be a big PR as it will need to fix all doctests as right now most of them are skipped as the examples are reaching out to remote services.

Agree 100%

Copy link
Contributor

@tomdonaldson tomdonaldson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing all this @bsipocz!

@tomdonaldson tomdonaldson merged commit 0702f29 into astropy:main Jun 1, 2022
@bsipocz bsipocz modified the milestones: v1.3.1, v1.4 Sep 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve test suite: raise all warnings as exceptions Enable doctesting the narrative documentation
4 participants