Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hyper-parameter Tuning Support -- Optuna #62

Merged
merged 21 commits into from
Jul 28, 2021
Merged

Hyper-parameter Tuning Support -- Optuna #62

merged 21 commits into from
Jul 28, 2021

Conversation

ncilfone
Copy link
Contributor

Partially addressing #60 by integrating Optuna support. Basic framework for accessing tuner backend. Should be able to extend to other backends as long as they have a similar 'define-and-run style interface' similar to Optuna where parameters can be freely used within the code and not managed internally (this might not be the case for Ax).

Also adds black/isort linting.

Missing unit tests.

@ncilfone ncilfone added the enhancement New feature or request label Jul 12, 2021
@coveralls
Copy link

coveralls commented Jul 12, 2021

Pull Request Test Coverage Report for Build 1024042568

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 647 of 929 (69.64%) changed or added relevant lines in 25 files are covered.
  • 4 unchanged lines in 2 files lost coverage.
  • Overall coverage decreased (-18.4%) to 77.012%

Changes Missing Coverage Covered Lines Changed/Added Lines %
spock/addons/s3/configs.py 6 7 85.71%
spock/handlers.py 28 30 93.33%
spock/addons/tune/init.py 0 3 0.0%
spock/backend/saver.py 84 87 96.55%
spock/addons/s3/utils.py 22 26 84.62%
spock/backend/payload.py 122 132 92.42%
spock/addons/tune/tuner.py 0 16 0.0%
spock/addons/tune/builder.py 0 22 0.0%
spock/addons/tune/interface.py 0 28 0.0%
spock/addons/tune/config.py 0 31 0.0%
Files with Coverage Reduction New Missed Lines %
spock/utils.py 1 97.5%
spock/builder.py 3 76.71%
Totals Coverage Status
Change from base Build 875925631: -18.4%
Covered Lines: 995
Relevant Lines: 1292

💛 - Coveralls

…gs (from cmd line or as input into ConfigArgBuilder) are passed thus falling back on all defaults or definitions from the command line. fix-up of all cmdline usage pattern. there were certain edge cases that were not getting caught correctly if it wasn't overriding an existing payload from a yaml file. #61
@ncilfone
Copy link
Contributor Author

Also covers #61 as I was too lazy to cut it on a new branch since this already had massive backend changes for hyper-parameter support.

@ncilfone ncilfone marked this pull request as ready for review July 21, 2021 17:28
@ncilfone ncilfone merged commit ea01717 into master Jul 28, 2021
@ncilfone ncilfone deleted the tune branch August 2, 2021 13:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove config as a required argument Support for Common Hyper-parameter Tuning Libs
2 participants