Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optuna integration & example script (2) #46

Merged
merged 81 commits into from
Nov 10, 2020
Merged

Optuna integration & example script (2) #46

merged 81 commits into from
Nov 10, 2020

Conversation

keisuke-nakata
Copy link
Member

This is the same PR as #21 (The original PR was broken due to private-public migration issue).

Example script for tuning hyper parameters by Optuna.

I modified experiments/train_agent_*.py to return statistics (and raise optuna.TrialPruned exception) which is needed for calculating a score of runs (trials) / pruning trials.

@keisuke-nakata
copy from examples/gym/train_dqn_gym.py
7689f6f
@keisuke-nakata
minor fix doc
9ddb854
@keisuke-nakata
return statistics from training
af9b7ec
@keisuke-nakata
update script
5b36dd2
@keisuke-nakata
follow black style
87add3a
@keisuke-nakata
evaluator may be called before agent.get_statistics().
f91128a
@keisuke-nakata
fix cpu tests (fast)
c82b826
@keisuke-nakata
black
c261d26
@keisuke-nakata
more tolerance...
adc8664
@keisuke-nakata
fix lint again...
bbd829e
@keisuke-nakata
wait small amount of time in order to avoid missing a queue item
2154ff3
@keisuke-nakata
Merge commit 'bebb7c83d95de70216f9be58e07c60c9bea0daf8' into optuna-i…
39571d2 
…ntegration2

# Conflicts:
#	pfrl/experiments/evaluator.py
@keisuke-nakata
reconsider hyper parameters
f3140d6
@keisuke-nakata
refine hyper param
2aa4ba1
@keisuke-nakata
remove redundant setups
83440fb
@keisuke-nakata
black
4672081
@keisuke-nakata
black
8a6f502
@keisuke-nakata
fix train_agent_async to pass tests
6abba17
@keisuke-nakata
adam_eps should be derived from loguniform
974e4ec
@keisuke-nakata
add optuna requirements
e2c53cc
@keisuke-nakata
add readme
fe74291
@keisuke-nakata
add optuna to setup.py
c4f232a
@keisuke-nakata
ignore missing imports for optuna
905ed89
@keisuke-nakata
c=int(loguniform)
50744f8
@keisuke-nakata
add example test
165eb5b
@keisuke-nakata
add missing hyphen
bf840d6
@keisuke-nakata
fix lint
12ce1d0
@keisuke-nakata
fix suggesting decay_step works with small args.steps
bd34dc2
@keisuke-nakata
add --env args for CI and supported spaces checks
c9f602d
@keisuke-nakata
rename file
62a0f3a
@pfn-ci-bot
Copy link

Successfully created a job for commit 2190fc8:

@keisuke-nakata
Copy link
Member Author

@muupan I fixed PR:

  • Make Optuna dependency optional.
  • Collects/returns evaluation stats only.
  • Unsupport collecting eval stats for train_agent_async.
    • Currently the collected eval stats are only used for optuna-integration script (optuna_dqn_obs1d.py).
    • Optuna-integration script also requires evaluation_hooks to raise TrialPruned exception for notifying optuna to prune the trial.
    • It's a bit difficult to re-raise an exception from subprocess (async-mode).

@keisuke-nakata
Merge commit '9be4726d327b7ce32d9008c40119c98c93febad5' into optuna-i…
a5f7007 
…ntegration2

# Conflicts:
#	pfrl/experiments/__init__.py
#	setup.cfg
@pfn-ci-bot
Copy link

Successfully created a job for commit a5f7007:

@pfn-ci-bot
Copy link

Successfully created a job for commit 5223b4d:

muupan
muupan requested changes Nov 9, 2020
View reviewed changes
examples/optuna/README.md Outdated Show resolved Hide resolved
examples/optuna/README.md Outdated Show resolved Hide resolved
examples/optuna/optuna_dqn_obs1d.py Outdated Show resolved Hide resolved
examples/optuna/optuna_dqn_obs1d.py Outdated Show resolved Hide resolved
examples/optuna/README.md Outdated Show resolved Hide resolved
examples/optuna/optuna_dqn_obs1d.py Outdated Show resolved Hide resolved
examples/optuna/optuna_dqn_obs1d.py Outdated Show resolved Hide resolved
pfrl/experiments/train_agent_async.py Outdated Show resolved Hide resolved
pfrl/experiments/train_agent_batch.py Show resolved Hide resolved
examples/optuna/README.md Outdated Show resolved Hide resolved
keisuke-nakata and others added 9 commits November 10, 2020 13:40
@keisuke-nakata @muupan
Update examples/optuna/README.md
a8bb8c8 
Co-authored-by: Yasuhiro Fujita <muupan@gmail.com>
@keisuke-nakata @muupan
Update examples/optuna/README.md
cf476ba 
Co-authored-by: Yasuhiro Fujita <muupan@gmail.com>
@keisuke-nakata @muupan
Update examples/optuna/optuna_dqn_obs1d.py
c0dd76b 
Co-authored-by: Yasuhiro Fujita <muupan@gmail.com>
@keisuke-nakata @muupan
Update examples/optuna/README.md
bdc4010 
Co-authored-by: Yasuhiro Fujita <muupan@gmail.com>
@keisuke-nakata @muupan
Update examples/optuna/optuna_dqn_obs1d.py
6d3af59 
Co-authored-by: Yasuhiro Fujita <muupan@gmail.com>
@keisuke-nakata @muupan
Update examples/optuna/optuna_dqn_obs1d.py
ecddbf4 
Co-authored-by: Yasuhiro Fujita <muupan@gmail.com>
@keisuke-nakata @muupan
Update examples/optuna/optuna_dqn_obs1d.py
fb6d110 
Co-authored-by: Yasuhiro Fujita <muupan@gmail.com>
@keisuke-nakata @muupan
Update examples/optuna/README.md
a790802 
Co-authored-by: Yasuhiro Fujita <muupan@gmail.com>
@keisuke-nakata
revert train_agent_async.py
91e585e
@pfn-ci-bot
Copy link

Successfully created a job for commit 91e585e:

@keisuke-nakata
Copy link
Member Author

@muupan Sorry, I have added small changes since your last review:

@pfn-ci-bot
Copy link

Successfully created a job for commit 317589b:

@muupan
Copy link
Member

muupan commented Nov 10, 2020

Those changes look ok.

@muupan muupan merged commit b9d2ee4 into pfnet:master Nov 10, 2020
@keisuke-nakata keisuke-nakata deleted the optuna-integration2 branch November 10, 2020 06:08
@muupan muupan added the enhancement New feature or request label Dec 11, 2020
@muupan muupan added this to the v0.2.0 milestone Dec 14, 2020
@muupan muupan added the example Modification or addition of examples label Dec 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@muupan muupan muupan approved these changes

Assignees
No one assigned
Labels
enhancement New feature or request example Modification or addition of examples
Projects
None yet
Milestone
v0.2.0
Development

Successfully merging this pull request may close these issues.
3 participants
@keisuke-nakata @pfn-ci-bot @muupan