Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: use nightly wheels #353

Merged
merged 8 commits into from
Sep 14, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 27 additions & 43 deletions .github/workflows/cibuild.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,33 +11,21 @@ on:
- cron: "0 3 * * *"

jobs:
egginfo:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.8
uses: actions/setup-python@v2
with:
python-version: 3.8
- name: Install tox
run: python -m pip install --upgrade tox
- name: egg-info
run: tox -e egg_info
tests:
needs: egginfo
runs-on: ubuntu-latest
name: ${{ matrix.python-version }}
name: ${{ matrix.python-version }} with all dependencies
strategy:
fail-fast: true
matrix:
python-version: ["3.8", "3.9", "3.10"]

steps:
- name: Checkout code
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install tox
Expand All @@ -47,23 +35,24 @@ jobs:
# remove "." from matrix.python-version to get the defaults tox python versions
tox_python_target=$(echo py${{matrix.python-version}} | sed 's/\.//')
echo "Tox python target: " $tox_python_target
tox -e $tox_python_target-test
tox -e $tox_python_target-test-alldeps


mac_windows:
runs-on: ${{ matrix.os }}
name: ${{ matrix.os }} with mandatory dependencies
needs: tests
strategy:
fail-fast: true
matrix:
os: [macos-latest, windows-latest]
steps:
- name: Checkout code
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: 3.9
- name: Install tox
Expand All @@ -72,60 +61,55 @@ jobs:
run: tox -e py39-test


style_coverage:
stylecheck:
runs-on: ubuntu-latest
needs: tests
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Set up Python 3.8
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: 3.8
- name: Setup Graphviz
uses: ts-graphviz/setup-graphviz@v1
- name: Install tox
run: python -m pip install --upgrade tox
- name: Check codestyle
run: tox -e codestyle
- name: Check coverage
run: tox -e cov
- name: Upload coverage to codecov
uses: codecov/codecov-action@v2
with:
file: ./coverage.xml

devastropy:

devdeps:
# TODO: this job should be allowed to fail except that the functionality
# is not available in actions yet.
runs-on: ubuntu-latest
needs: tests
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Set up Python 3.10
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: "3.10"
- name: Install tox
run: python -m pip install --upgrade tox
- name: Run tests against devastropy
run: tox -e py310-test-devastropy
- name: Run tests against dev dependencies
run: tox -e py310-test-devdeps-alldeps-cov

- name: Upload coverage to codecov
uses: codecov/codecov-action@v3
with:
file: ./coverage.xml
verbose: true



oldestdeps:
runs-on: ubuntu-latest
needs: tests
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Set up Python 3.8
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: 3.8
- name: Install tox
run: python -m pip install --upgrade tox
- name: Run tests against oldest dependencies
run: tox -e py38-test-oldestdeps

#publish:
#needs: coverage
#TODO - should be trigger by tagging associated with creating a new release in
# github. The tag should be compared to the version in setup.cfg to ensure
# consistency.
1 change: 0 additions & 1 deletion README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,6 @@ The following packages are required for PyVO:
The following packages are optional dependencies and are required for the
full functionality:

* mimeparse
* pillow

For running the tests, and building the documentation, the following
Expand Down
1 change: 0 additions & 1 deletion pyvo/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,6 @@
from ._astropy_init import *
# ----------------------------------------------------------------------------

# For egg_info test builds to pass, put package imports here.
from . import registry
from .dal import ssa, sia, sla, scs, tap
from . import auth
Expand Down
7 changes: 6 additions & 1 deletion pyvo/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,13 @@ def pytest_configure(config):

# Customize the following lines to add/remove entries from the list of
# packages for which version numbers are displayed when running the tests.
PYTEST_HEADER_MODULES['Astropy'] = 'astropy' # noqa
PYTEST_HEADER_MODULES['requests'] = 'requests' # noqa

PYTEST_HEADER_MODULES.pop('Pandas', None)
PYTEST_HEADER_MODULES['scikit-image'] = 'skimage'
PYTEST_HEADER_MODULES.pop('h5py', None)
PYTEST_HEADER_MODULES.pop('Scipy', None)
PYTEST_HEADER_MODULES.pop('Matplotlib', None)

from . import __version__
TESTED_VERSIONS['pyvo'] = __version__
7 changes: 7 additions & 0 deletions pyvo/dal/tests/test_mimetype.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,12 @@
get_pkg_data_contents = partial(
get_pkg_data_contents, package=__package__, encoding='binary')

try:
from PIL import Image
HAS_PILLOW = True
except ImportError:
HAS_PILLOW = False


@pytest.fixture()
def mime(mocker):
Expand All @@ -35,6 +41,7 @@ def callback(request, context):


@pytest.mark.usefixtures('mime')
@pytest.mark.skipif('not HAS_PILLOW')
def test_mime_object_maker():

assert 'Text content' == mime_object_maker(mime_url+'mime-text',
Expand Down
1 change: 0 additions & 1 deletion setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,6 @@ python_requires = >=3.8

[options.extras_require]
all =
mimeparse
pillow
test =
pytest-astropy
Expand Down
50 changes: 13 additions & 37 deletions tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -3,65 +3,41 @@
# as oldestdeps and devastropy might not support the full python range
# listed here
envlist =
py{38,39,310}-test{,-oldestdeps, -devastropy}
egg_info
cov
py{38,39,310}-test{,-alldeps,-oldestdeps,-devdeps}{,-cov}
linkcheck
codestyle
build_docs

requires =
setuptools >= 30.3.0
pip >= 19.3.1
tox-pypi-filter >= 0.12

[testenv]
pypi_filter = https://raw.githubusercontent.com/astropy/ci-helpers/main/pip_pinnings.txt

extras =
test
all

commands =
pip freeze
pytest --pyargs pyvo
test: test
alldeps: all

# tox environments are constructed with so-called 'factors' (or terms)
# separated by hyphens, e.g. test-devdeps-cov. Lines below starting with factor:
# will only take effect if that factor is included in the environment name. To
# see a list of example environments that can be run, along with a description,
# run:
#
# tox -l -v
#
description =
run tests
oldestdeps: with astropy 4.1.*
devastropy: with astropy latest
oldestdeps: with oldest supported dependencies
devdeps: with development version of dependencies
cov: determine the code coverage

deps =
devastropy: git+https://github.com/astropy/astropy.git#egg=astropy
cov: coverage

oldestdeps: astropy==4.1
# We set a suitably old numpy along with an old astropy, no need to pick up
# deprecations and errors due to their unmatching versions
oldestdeps: numpy==1.16

[testenv:egg_info]
description = ensure egg_info works without dependencies
commands =
pip freeze
python setup.py egg_info
devdeps: pip install -U --pre --only-binary :all: -i https://pypi.anaconda.org/scipy-wheels-nightly/simple numpy
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stuart proposed another way to do this over at astropy/astropy#13614

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, interesting. That was more like the first approach we had, and then we switched over to having these individual pip installs. I think Stuart's version is cleaner in the case when you have multiple packages. However, here we only use the scipy-wheels-nightly for numpy, and something else for astropy, and don't install other packages. So I would say we should keep it as a one-liner like this rather than adding extras.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's very little to choose between them, so use your favourite, but sunpy uses multiple index URLs with the same pattern https://github.com/sunpy/sunpy/blob/main/tox.ini#L34

devdeps: pip install -U --pre -i https://pypi.anaconda.org/astropy/simple astropy

[testenv:cov]
description = determine the code coverage
deps:
# had to pin this
coverage==4.5.4

commands =
pip freeze
pytest --pyargs pyvo --cov pyvo --cov-config={toxinidir}/setup.cfg
coverage xml -o {toxinidir}/coverage.xml
!cov: pytest --pyargs pyvo
cov: pytest --pyargs pyvo --cov pyvo --cov-config={toxinidir}/setup.cfg
cov: coverage xml -o {toxinidir}/coverage.xml


[testenv:docs]
Expand Down