Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make local resource storage locations configurable #5315

Merged
merged 7 commits into from May 29, 2018

Conversation

@AlexDBlack
Copy link
Contributor

commented May 24, 2018

Fixes: #5102
Fixes: #5333

  • DL4J: configurable local storage location for zoo models and datasets (MNIST etc)
    • Option 1: configure via system property - org.deeplearning4j.resources.directory
    • Option 2: configure via DL4JResources.setBaseDirectory
  • DataVec: configurable local storage for resources/datasets
  • Iris moved online (no longer bundled in JAR)
  • Moved MNIST/EMNIST to Azure
  • Fixed some tests so they do not interfere with resources (i.e., don't delete the local files - use a different folder!)
  • Added Downloader class to ND4J common
  • Zoo non-model resources (label text/JSON files) moved online - #5333

Base URLs are also configurable in case we even need mirrors or need to set up a different download location with the same folder structure.

Note that: default location for resources has changed...

  • DL4J model zoo models are now under (user.home)/.deeplearning4j/models/(model name)
  • DL4J datasets are now under (user.home)/.deeplearning/data/(dataset name)
    • This includes things like MNIST/EMNIST etc, which were previously under (user.home)/MNIST etc
  • DataVec datasets are now under (user.home)/.datavec/data/(dataset name)

@AlexDBlack AlexDBlack force-pushed the ab_5102_downloads branch from 196a108 to 0f9fea3 May 25, 2018

@AlexDBlack AlexDBlack requested a review from crockpotveggies May 25, 2018

@AlexDBlack AlexDBlack force-pushed the ab_5102_downloads branch from 0f9fea3 to d09cecd May 26, 2018

@AlexDBlack AlexDBlack merged commit d426b3b into master May 29, 2018

1 of 2 checks passed

codeclimate 17 issues to fix
Details
continuous-integration/jenkins/pr-merge This commit looks good
Details

@AlexDBlack AlexDBlack deleted the ab_5102_downloads branch May 29, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.