Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run Continuous Integration tests on Github Actions #475

Merged
merged 41 commits into from
Jul 10, 2020
Merged
Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
4c0fd39
Run Continuous Integration tests on Github Actions
weiji14 May 19, 2020
31340ee
Install make earlier
weiji14 May 19, 2020
9a4e7bc
Set Linux/MacOS environment variables properly
weiji14 May 20, 2020
51472dc
Install pytest and other dev dependencies
weiji14 May 20, 2020
9c2e393
Don't fail-fast and cancel all jobs
weiji14 May 20, 2020
2220702
Update setup-miniconda Github Action from v1 to v1.3.1
weiji14 May 20, 2020
6936b3c
Don't set any environment variables for all OSes
weiji14 May 21, 2020
6906426
Cache GMT tutorial files in ${{ github.workspace }}/.gmt on 20200605
weiji14 Jun 5, 2020
d2ce09b
Fix typo python_version to python-version
weiji14 Jun 7, 2020
e407081
Fix python_version to python-version
seisman Jun 9, 2020
841f937
Disable tests to enable caches
seisman Jun 10, 2020
9fd73f0
Re-enable tests to check caches
seisman Jun 10, 2020
1a23b55
Change github action name
seisman Jun 10, 2020
0ba3d7e
Simplify job name
seisman Jun 10, 2020
c744db9
The versioneer pacakge need tags to determine the PyGMT version
seisman Jun 10, 2020
3a06afc
Change runner.os to matrix.os
seisman Jun 10, 2020
05c225f
Improve the docs
seisman Jun 10, 2020
8bd8a66
Add a cron job for daily tests
seisman Jun 10, 2020
d4408c0
Merge branch 'master' into gh-actions-ci
seisman Jun 29, 2020
c12cfff
Change earth_relief_60m to earth_relief_01d
seisman Jun 29, 2020
9179bc2
Let conda install packages listed in requirements.txt
seisman Jun 29, 2020
8662be0
Single conda command to install all packages
seisman Jun 29, 2020
a806efa
Add github.ref to the cache key
seisman Jun 29, 2020
895e871
Disable the cache temporarily to cache the GMT files
seisman Jun 29, 2020
e0c79e6
Enable tests again
seisman Jun 29, 2020
4d271c0
Set fetch-depth to 0
seisman Jun 29, 2020
0c1ba71
Create an environment using environment.yml
seisman Jun 29, 2020
80c6508
Still slow using environment.yml
seisman Jun 29, 2020
bf2126f
Cache conda packages
seisman Jun 30, 2020
9c97722
Disable tests to cache conda packages
seisman Jun 30, 2020
37f2a53
Update cache keys
seisman Jun 30, 2020
8ff30d9
Enable tests
seisman Jun 30, 2020
1b947d3
Run Github Actions CI tests on GMT 6.1.0
weiji14 Jul 7, 2020
cd7d7da
Merge branch 'master' into gh-actions-ci
seisman Jul 9, 2020
098e03d
Revert "Run Github Actions CI tests on GMT 6.1.0"
weiji14 Jul 7, 2020
b4c7492
Disable Azure Pipeline CI on Windows and macOS, but keep Style Checks on
weiji14 Jul 9, 2020
b6445d0
Do not cache the conda packages
seisman Jul 9, 2020
b731f0c
Add the status badge to README
seisman Jul 10, 2020
5b65455
Test Python 3.6-3.8
seisman Jul 10, 2020
36fa209
Let the github actions run in all PRs
seisman Jul 10, 2020
a5af2dc
Use PYTEST_EXTRA in Makefile
seisman Jul 10, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .azure-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ jobs:
########################################################################################
- job:
displayName: 'Mac'
condition: False

pool:
vmImage: 'macOS-10.14'
Expand Down Expand Up @@ -171,6 +172,7 @@ jobs:
########################################################################################
- job:
displayName: 'Windows'
condition: False

pool:
vmImage: 'vs2017-win2016'
Expand Down
99 changes: 99 additions & 0 deletions .github/workflows/ci_tests.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
seisman marked this conversation as resolved.
Show resolved Hide resolved
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions

name: Tests

on:
push:
branches: [ master ]
pull_request:
weiji14 marked this conversation as resolved.
Show resolved Hide resolved
branches: [ master ]
seisman marked this conversation as resolved.
Show resolved Hide resolved
# Schedule daily tests
schedule:
- cron: '0 0 * * *'

jobs:
test:
name: ${{ matrix.os }} - Python ${{ matrix.python-version }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
python-version: [3.7]
seisman marked this conversation as resolved.
Show resolved Hide resolved
os: [ubuntu-latest, macOS-latest, windows-latest]

steps:
# Checkout current git repository
- name: Checkout
uses: actions/checkout@v2.3.1
with:
# fecth all history so that versioneer works
fetch-depth: 0

# Setup Miniconda
- name: Setup Miniconda
uses: goanpeca/setup-miniconda@v1.6.0
with:
python-version: ${{ matrix.python-version }}
channels: conda-forge

# Install GMT and other required dependencies from conda-forge
- name: Install GMT and required dependencies
shell: bash -l {0}
run: |
requirements_file=full-conda-requirements.txt
cat requirements.txt requirements-dev.txt > $requirements_file
cat << EOF >> $requirements_file
gmt=6.0.0
make
EOF
conda install --yes --file $requirements_file
Comment on lines +32 to +49
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can speed up the dependency install using caching too, see https://github.com/goanpeca/setup-miniconda#caching. I've used it for one of my projects, the relevant commit is at weiji14/deepicedrain@391632b.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I saw that. But I'm not sure how the caches are updated. For example, we may cache numpy v1.18.0 now. When new numpy versions are released, conda will download and install the latest version (e.g., numpy v1.19.0). What will happen to the caches? Do the caches contains the tarballs of numpy v1.18.0, v1.19.0 or both? Do we have to manually update the cache key?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cache would hash the environment.yml/requirement.txt files, and unless the hash changes, the cache will stay at v1.18.0 (unless we manually update the cache key). Maybe we should start thinking about a minimum supported dependency policy, as I suggested at #340 (comment).


# Show installed pkg information for postmortem diagnostic
- name: List installed packages
shell: bash -l {0}
run: conda list

# Cache the ${HOME}/.gmt directory, for docs and testing
- name: Cache GMT directory
uses: actions/cache@v2
id: cache
with:
path: |
~/.gmt/cache
~/.gmt/server
key: cache-gmt-${{ github.ref }}-${{ runner.os }}-20200609
restore-keys: cache-gmt-refs/heads/master-

# Workaround for the timeouts of 'gmt which' on Linux and Windows
- name: Download remote data using wget (Linux & Windows)
shell: bash -l {0}
run: |
if [ "$RUNNER_OS" == "Windows" ]; then choco install wget; fi # install wget on Windows
mkdir ~/.gmt ~/.gmt/cache ~/.gmt/server
wget --no-check-certificate https://oceania.generic-mapping-tools.org/gmt_hash_server.txt -P ~/.gmt/server/
for data in earth_relief_01d.grd earth_relief_30m.grd earth_relief_10m.grd; do
wget --no-check-certificate https://oceania.generic-mapping-tools.org/${data} -P ~/.gmt/server/
done
Comment on lines +70 to +76
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to change these lines when bumping to 6.1. In 6.1, the earth relief data are stored in ~/.gmt/server/earth/earth_relief/ directory.

for data in ridge.txt Table_5_11.txt tut_bathy.nc tut_quakes.ngdc tut_ship.xyz usgs_quakes_22.txt; do
wget --no-check-certificate https://oceania.generic-mapping-tools.org/cache/${data} -P ~/.gmt/cache/
done
if: steps.cache.outputs.cache-hit != 'true' && runner.os != 'macOS'

# Download remote files, if not already cached
- name: Download remote data (macOS)
shell: bash -l {0}
run: gmt which -Gu @earth_relief_01d @earth_relief_30m @earth_relief_10m @ridge.txt @Table_5_11.txt @tut_bathy.nc @tut_quakes.ngdc @tut_ship.xyz @usgs_quakes_22.txt
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to change gmt which -Ga in GMT 6.1.

if: steps.cache.outputs.cache-hit != 'true' && runner.os == 'macOS'

# Install the package that we want to test
- name: Install the package
shell: bash -l {0}
run: |
python setup.py sdist --formats=zip
pip install dist/*

# Run the tests
- name: Test with pytest
shell: bash -l {0}
run: make test PYTEST_EXTRA="-r P"
seisman marked this conversation as resolved.
Show resolved Hide resolved