Run doctests only when using pinned requirements for most recent Python version #2468
Labels
documentation infrastructure
Plasma Lv0 | Novice
Issues that do not require knowledge of physics
priority: low
Issues & PRs of low urgency and importance that may be deferred until later
Python Lv3 | Proficient
Issues that require proficiency in Python
refactoring ♻️
Improving an implementation without adding new functionality
Description of improvement
Instead of running doctests every time we run tests, we should consider switching to running doctests only on the pinned requirements in
requirements.txt
that get updated via an automated PR every week.Motivation
Sometimes upstream packages like NumPy and Astropy change how they print out things like
ndarray
andQuantity
objects. If we ran doctests only on a single updated set of package versions, we wouldn't have to do any version-dependent skipping of doctests.Since we currently run code coverage on an older version of the tests, a consequence of this change would be that lines of code covered only by doctests would no longer show up as being covered by tests. This would help us avoid situations where certain lines of code are only run during doctests and not in our standard test suite, or a function is only tested in doctests.
Implementation strategy
The change would be in
tox.ini
and/or.github/workflows/tests.yml
. Might be easiest to try getting ChatGPT to do this one for us. It would probably involve changing the flags provided topytest
.Additional context
See also #135, #229, #653, #2148, astropy/astropy#14553, astropy/astropy#15095, astropy/astropy#15312
The text was updated successfully, but these errors were encountered: