Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve gammapy download #1763

Merged
merged 4 commits into from Sep 10, 2018
Merged

Improve gammapy download #1763

merged 4 commits into from Sep 10, 2018

Conversation

@Bultako
Copy link
Member

@Bultako Bultako commented Sep 5, 2018

This PR extends functionalities of gammapy download - initially added in #1369 as a CLI to address issues of PIG 4 - Setup for tutorial notebooks and data in #1419.

This PR provides gammapy download with new options:

  • specify gammapy version release or commit hash in Github repo
  • downloading a specific file or all content from a specific folder
  • recursive/no-recursive flag to avoid content download recursively
$ gammapy download 
Usage: gammapy download [OPTIONS] COMMAND [ARGS]...

  Download datasets and notebooks.

  Download from the 'gammapy-extra' Github repository the content of
  'datasets' or 'notebooks' folders. The files are copied into a folder
  created at the current working directory.

  Examples
  --------

  $ gammapy download notebooks
  $ gammapy download datasets
  $ gammapy download --file=first_steps.ipynb notebooks
  $ gammapy download --dest=localfolder --fold=catalogs/fermi --no-recursive datasets
  $ gammapy download --hash=master notebooks

Options:
  --dest TEXT                   Folder where the files will be copied.
  --file TEXT                   Specific file to download.
  --fold TEXT                   Specific folder to download.
  --release TEXT                Release or commit hash in Github repo.
  --recursive / --no-recursive  Deactivate recursive scan of a folder.
  -h, --help                    Show this message and exit.

Commands:
  datasets   Download datasets
  notebooks  Download notebooks
@cdeil cdeil added the feature label Sep 5, 2018
@cdeil cdeil added this to To Do in DOCUMENTATION via automation Sep 5, 2018
@cdeil cdeil added this to the 0.9 milestone Sep 5, 2018
Copy link
Member

@cdeil cdeil left a comment

@Bultako - Thanks!

Do you think it would be be possible / advantageous to add tests that execute this code for gammapy download?

Generally we're trying to reduce / remove tests that hit the network because of annoying timeout failures and generally they are slow, but having non-trivial code without automated tests can also be problematic.

Tests can be added later for this, merge whenever you like.

@Bultako Bultako force-pushed the Bultako:gp-download branch from 71e6c8e to 8947786 Sep 5, 2018
@Bultako
Copy link
Member Author

@Bultako Bultako commented Sep 5, 2018

@cdeil
Yes, it would be good to have some tests. At least for small files or datasets that we think there will always be available. Now that we can download specific files, it's possible.
👍

@Bultako Bultako force-pushed the Bultako:gp-download branch 5 times, most recently from da4137a to 897df7e Sep 6, 2018
@Bultako Bultako force-pushed the Bultako:gp-download branch from 897df7e to 12a73ee Sep 10, 2018
@Bultako Bultako merged commit 1ecadc7 into gammapy:master Sep 10, 2018
3 of 4 checks passed
3 of 4 checks passed
continuous-integration/travis-ci/pr The Travis CI build could not complete due to an error
Details
Codacy/PR Quality Review Up to standards. A positive pull request.
Details
Scrutinizer Analysis: 448 new issues, 4 updated code elements – Tests: passed
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
DOCUMENTATION automation moved this from To Do to Done Sep 10, 2018
@Bultako Bultako deleted the Bultako:gp-download branch Sep 10, 2018
@cdeil cdeil changed the title Improvement of gammapy download Improve gammapy download Nov 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
DOCUMENTATION
  
Done
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants