Fix issue #100 #102
Fix issue #100 #102
Conversation
| @@ -65,7 +65,7 @@ | |||
| choices=["halving", "median", "none"], | |||
| ) | |||
| parser.add_argument("--n-startup-trials", help="Number of trials before using optuna sampler", type=int, default=10) | |||
| parser.add_argument("--n-evaluations", help="Number of evaluations for hyperparameter optimization", type=int, default=20) | |||
| parser.add_argument("--n-evaluations", help="Number of timesteps at which the environment is evaluated during for a single hyperparameter combination", type=int, default=20) | |||
There was a problem hiding this comment.
that's not what it does...
For hyperparameter optimization, you give each trial a maximum budget of n_timesteps and to decide whether to prune or not, you evaluate each trial every n_timesteps // args.n_evaluations
btw, could you merge your two doc PR together and update the changelog?
There was a problem hiding this comment.
"you give each trial a maximum budget of n_timesteps and to decide whether to prune or not"
After reading through the code either I'm misunderstanding you, I'm going crazy or thats wrong. n_timesteps how by timesteps an individual training run for set of hyperparameters runs for and that's used for reference in pruning. e.g. model.learn(self.n_timesteps, callback=eval_callback)
However, my new sentence did have some grammar errors, this is what it probably should've said:
"Maximum number of training timesteps for each set of hyperparameters during optimization"
Is that better?
There was a problem hiding this comment.
"Maximum number of training timesteps for each set of hyperparameters during optimization"
This is what n_timesteps is when doing hyperparameter optimization.
And using the TrialEvalCallback, we can stop each trial early by returning False according to the pruner:
rl-baselines3-zoo/utils/callbacks.py
Line 49 in f6b3ff7
which resutls in a trial doing less than
n_timesteps.
And to decide whether to prune or not, you have regular evaluations every n_timesteps // args.n_evaluations, so a maximum of args.n_evaluations in total.
If you want, there is a self contained example in the optuna repo or here: https://github.com/araffin/rl-handson-rlvs21/blob/main/optuna/sb3_simple.py
Close #100