Github Actions CI Setup #2634
Closed
Github Actions CI Setup #2634
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Removed cyordereddict recipe that we no longer depend on Removed strict bounds rewriting in setup.py. Now setup.py reads from requirements.in, which is not strict in its bounds Need to update case for pip-compile's output Filter out comments from lockfile, now that we have "via"
danisim
force-pushed
the
gh-actions-ci
branch
2 times, most recently
from
c33f7aa
to
8c69e05
Compare
danisim
force-pushed
the
gh-actions-ci
branch
from
8c69e05
to
ade2044
Compare
added 14 commits
January 27, 2020 15:35
ssanderson
reviewed
Jan 29, 2020
| - name: Pip cache | ||
| uses: actions/cache@v1 | ||
| with: | ||
| path: ~/.cache/.pip/pip_np$NUMPY_VERSION |
There was a problem hiding this comment.
Given that we're using a key, I don't think we should need to encode the numpy_version into the pip cache directory anymore.
ssanderson
reviewed
Jan 29, 2020
Comment on lines
+60
to
+72
| TALIB_VERSION=$(cat ./etc/requirements.txt | grep "ta-lib" | sed "s/ta-lib==\([^ ]*\) *.*/\1/") | ||
| CERTIFI_VERSION=$(cat ./etc/requirements.txt | grep "certifi" | sed "s/certifi==\([^ ]*\) *.*/\1/") | ||
|
|
||
| if [[ "$RUNNER_OS" == "Linux" ]]; then | ||
| sudo apt-get install python3-venv | ||
| fi | ||
| python3 -m venv testenv | ||
| source ./testenv/bin/activate | ||
| conda install -c anaconda libgfortran=3.0 | ||
| pip install pip cython numpy pandas scipy ta-lib certifi -c etc/requirements.txt --cache-dir="$HOME/.cache/.pip/pip_np$NUMPY_VERSION" | ||
| etc/dev-install --cache-dir="$HOME/.cache/.pip/pip_np$NUMPY_VERSION" | ||
| pip freeze | sort | ||
| echo "::set-env name=PATH::$PATH" |
There was a problem hiding this comment.
This seems like more complexity than I would have expected/hoped would be necessary. A couple quick suggestions:
- Given that we're using the
setup-pythonaction to make a fresh python for each of these builds, I don't think it should be necessary to create a separate virtualenv for the build. - I don't think we should need to use conda for anything here anymore. In particular, installing
anacondabrings in a lot of stuff we don't want (anaconda is python + all the packages that anaconda the company curates, c.f. https://stackoverflow.com/questions/30034840/what-are-the-differences-between-conda-and-anaconda). - I don't think we want to be pre-installing all the packages you currently have listed in the top-level
pip install. In particular, we probably don't want to be installingscipyandpandasat the same time as we installnumpy, since the reason we pre-install numpy and Cython is so that those packages are available at build time for scipy and pandas (and other packages that depend on them at build time). I'd expect that the right sequence of installs here should look something like:pip install pip==<pinned pip version> setuptools=<pinned setuptools version>,pip install -r etc/requirements_build.in -c etc/requirements.txt,pip install -e .[all] -c etc/requirements.txt.
There was a problem hiding this comment.
@ssanderson Sorry you caught this PR in a wip state but appreciate the feedback!
I set up a virtualenv to get around this error that I was running into when I tried to pip install directly to the actions host.
ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/cython.py'
Consider using the `--user` option or check the permissions.
I tried to strip away the conda installs and as you warned me last week, pip doesn't handle the dependencies as well and I was running into a bunch of dependency constraint errors.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Setup for GH actions to replace Travis/Appveyor config.
Note: