How to set number of epochs trained instead of total_timestep

[maxmatical]

In the example codes, the call to train the agent is as follows:

model.learn(total_timesteps=10000)

Is there a way to specify the number of epochs a model is trained, instead of by timesteps?

[araffin]

Hello,

How do you define an epoch? You maybe mean episodes?

[maxmatical]

I think it depends on the definition, but I mean an epoch in the traditional DL sense, a forward/backward pass through the entirety of the data

[araffin]

I mean an epoch in the traditional DL sense, a forward/backward pass through the entirety of the data

Well, we are in a RL setting, "the entirety of the data" does not really makes sense here, no?

[maxmatical]

Epochs are defined in openai spinups

ppo(env_fn=env_fn, ac_kwargs=ac_kwargs, steps_per_epoch=5000, epochs=250, logger_kwargs=logger_kwargs)

Which would be the equivalent of nupdates in baselines

So I'm just curious if there's an equivalent variable in baselines. If I want to set the number of episodes, is there an option to do so, or is it enough to set the number of steps?

[araffin]

Epochs are defined in openai spinups

I see.

If you can want an equivalent of epochs, with one process:

for PPO2 for instance, epochs= total_timesteps // n_steps (in the sense of spinning up) which is as you pointed out nupdates.
However, in the the multiprocessing case, epochs= total_timesteps // (n_steps * n_envs).

For other algorithms, like SAC that updates every steps, that does not really makes sense...

or is it enough to set the number of steps?

There is no option, but you can easily convert your number of epochs into a number of steps (cf above), but that depends on the algorithm used.

[maxmatical]

I see. Thanks for your answers.

[blurLake]

How about DDPG? I saw in openai spinups

spinup.ddpg_tf1(..., steps_per_epoch=4000, epochs=100, ...)

Is there a similar setting in stable_baselines?