New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Planemo fetches old database #61

Closed
erasche opened this Issue Feb 6, 2015 · 10 comments

Comments

Projects
None yet
3 participants
@erasche
Copy link
Member

erasche commented Feb 6, 2015

relevant line

Instead of fetching a specific revision that will slowly age out, how about just symlinking it (or providing a 'latest')?

@dannon

This comment has been minimized.

Copy link
Member

dannon commented Feb 6, 2015

I actually tried to implement this in the galaxy_downloads repo but botched it, let me review what went wrong.

@dannon

This comment has been minimized.

@erasche

This comment has been minimized.

Copy link
Member Author

erasche commented Feb 6, 2015

Ah, so not as simple it could be. Someone just needs to publish an ftp directory of them... (or for the code to understand github's view of a symlink)

@dannon

This comment has been minimized.

Copy link
Member

dannon commented Feb 6, 2015

Minor progress figuring this out (maybe -- you can get the contents anyway):
curl --header "Content-Type: application/vnd.github.VERSION.raw" https://api.github.com/repos/jmchilton/galaxy-downloads/contents/latest.sqlite

@erasche

This comment has been minimized.

Copy link
Member Author

erasche commented Feb 6, 2015

Huh. I would've figured that a request to https://raw.githubusercontent.com/dannon/galaxy-downloads/master/latest.sqlite with that header would've responded correctly, but it looks like it's JUST the API that responds that way. That's bloody annoying.

Okay, my vote is for changing

DATABASE_TEMPLATE_URL = DOWNLOADS_URL + "db_gx_rev_0120.sqlite"

from

DATABASE_TEMPLATE_URL = DOWNLOADS_URL + "db_gx_rev_0120.sqlite"

to

import urllib
LATEST_URL = "https://raw.githubusercontent.com/jmchilton/galaxy-downloads/master/latest.sqlite"
DATABASE_TEMPLATE_URL = DOWNLOADS_URL + urllib.urlopen(LATEST_URL).read()

It's too easy...even if it isn't as clean as some solution which would redirect properly.

@dannon

This comment has been minimized.

Copy link
Member

dannon commented Feb 6, 2015

Annoying that github won't just serve the file, but I think this solution is fine. I can mirror the change in galaxy's test framework, since it has the same issue.

jmchilton added a commit that referenced this issue Feb 8, 2015

Enhancements to database downloading.
 - Don't download twice for tests.
 - Download the latest using ideas from @erasche and @dannon.

Closes issue #61.

@jmchilton jmchilton closed this Feb 8, 2015

@jmchilton

This comment has been minimized.

Copy link
Member

jmchilton commented Feb 10, 2015

I may roll this back or tweak it somehow - I don't like that planemo is now incompatible with the latest stable. I feel like too old is better than too new.

Wonder if it is possible to just force Galaxy to use the newer database - in many cases it would probably just work.

@erasche

This comment has been minimized.

Copy link
Member Author

erasche commented Feb 10, 2015

I noticed that too in my testing. I agree, that's unpleasant behaviour.

maybe it would make sense to:

  • offer a "latest" db
  • offer an "old" db
  • default to one but provide a flag for the other

Or something like that.

@jmchilton

This comment has been minimized.

Copy link
Member

jmchilton commented Feb 10, 2015

I think I want to move this functionality into Galaxy's run_tests.sh - that way API and casper tests could potentially leverage it and run_tests.sh can know what version it is on.

@erasche

This comment has been minimized.

Copy link
Member Author

erasche commented Feb 10, 2015

Sure, that'd make sense too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment