New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update various Galaxy requirements. #4884

Merged
merged 1 commit into from Oct 27, 2017

Conversation

Projects
None yet
3 participants
@jmchilton
Member

jmchilton commented Oct 27, 2017

At least requests and Babel are required to get the latest "locked" version of the dev requirements compatible with our current locked versions in requirements.txt.

Removed repoze.lru from an earlier variant of this since the wheel didn't work - good catch @nsoranzo.

Update various Galaxy requirements.
At least requests and Babel are required to get the latest "locked" version of the dev requirements compatible with our current locked versions in requirements.txt.

Removed repoze.lru from an earlier variant of this since the wheel didn't work - good catch @nsoranzo.

@galaxybot galaxybot added the triage label Oct 27, 2017

@galaxybot galaxybot added this to the 18.01 milestone Oct 27, 2017

@nsoranzo

This comment has been minimized.

Member

nsoranzo commented Oct 27, 2017

While inspecting the Travis logs, I've found that Galaxy downloads the following packages from PyPI instead of https://wheels.galaxyproject.org/ :

asn1crypto-0.23.0-py2.py3-none-any.whl
bcrypt-3.1.4-cp27-cp27mu-manylinux1_x86_64.whl
boltons-17.1.0-py2.py3-none-any.whl
boto3-1.4.7-py2.py3-none-any.whl
botocore-1.7.36-py2.py3-none-any.whl
cffi-1.11.2-cp27-cp27mu-manylinux1_x86_64.whl
cryptography-2.1.2-cp27-cp27mu-manylinux1_x86_64.whl
docopt-0.6.2.tar.gz
enum34-1.1.6-py2-none-any.whl
h5py-2.7.1-cp27-cp27mu-manylinux1_x86_64.whl
jmespath-0.9.3-py2.py3-none-any.whl
PasteScript-2.0.2-py2.py3-none-any.whl
pulsar_galaxy_lib-0.8.0-py2-none-any.whl
pyasn1-0.3.7-py2.py3-none-any.whl
pycparser-2.18.tar.gz
pykwalify-1.6.0-py2.py3-none-any.whl
PyNaCl-1.1.2-cp27-cp27mu-manylinux1_x86_64.whl
python_dateutil-2.5.3-py2.py3-none-any.whl
python-genomespaceclient-0.1.8.tar.gz
python_swiftclient-3.4.0-py2.py3-none-any.whl
s3transfer-0.1.11-py2.py3-none-any.whl
testfixtures-4.10.0-py2.py3-none-any.whl

I've also looked in https://wheels.galaxyproject.org/packages/ and they're indeed not there. Most of them don't have a recipe on starforge, but docopt, python-dateutil and testfixtures should definetly be there.

@jmchilton

This comment has been minimized.

Member

jmchilton commented Oct 27, 2017

@nsoranzo Those are existing... "problems"? To whatever extend they are actually problematic. Maybe we should just keep a running list on starforge - maybe expand galaxyproject/starforge#173 which includes some of those already? Against dev pykwalify doesn't seem to come from wheels.galaxyproject.org either (e.g. https://travis-ci.org/galaxyproject/galaxy/jobs/293786833) and these other ones aren't related to this PR. pulsar_galaxy_lib-0.8.0-py2-none-any.whl was added by Mr. Starforge himself and it does not come from there.

@nsoranzo

This comment has been minimized.

Member

nsoranzo commented Oct 27, 2017

After some investigation, it seems the plan is to not build on Starforge if wheels are already available, see #4674 . Therefore the list can be shortened to the ones that needs to be compiled:

docopt-0.6.2.tar.gz
pycparser-2.18.tar.gz
python-genomespaceclient-0.1.8.tar.gz

which I have added to galaxyproject/starforge#173 as you suggested.

Those are existing... "problems"? To whatever extend they are actually problematic.

The problem is that they may disappear from PyPI. So we need to have a process to make sure they are stored on depot, as mentioned in the PR linked above.

Against dev pykwalify doesn't seem to come from wheels.galaxyproject.org either

Yes, but you said you copied the files in galaxyproject/starforge#172 (comment) , seems fishy.

@jmchilton

This comment has been minimized.

Member

jmchilton commented Oct 27, 2017

Yes, but you said you copied the files in galaxyproject/starforge#172 (comment) , seems fishy.

Well... it turns out I copied a lot of the files just not all the files. Fixed pykwalify.

Collecting pykwalify==1.6.0 (from -r requirements.txt (line 40))
  Downloading https://wheels.galaxyproject.org/packages/pykwalify-1.6.0-py2.py3-none-any.whl

At worst though it wasn't a regression.

@jmchilton

This comment has been minimized.

Member

jmchilton commented Oct 27, 2017

I'll rerun the unit tests - those are the ones that would break with pykwalify changes I think.

@nsoranzo nsoranzo merged commit 235a590 into galaxyproject:dev Oct 27, 2017

6 checks passed

api test Build finished. 306 tests run, 5 skipped, 0 failed.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
framework test Build finished. 162 tests run, 0 skipped, 0 failed.
Details
integration test Build finished. 57 tests run, 0 skipped, 0 failed.
Details
lgtm analysis: JavaScript No alert changes
Details
toolshed test Build finished. 577 tests run, 0 skipped, 0 failed.
Details
@jmchilton

This comment has been minimized.

Member

jmchilton commented Oct 27, 2017

Thanks for the merge @nsoranzo and thanks for digging into this so much. I'm learning a lot about Galaxy dependencies 😅.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment