[BUG] How to continue training from a save checkpoint

[zoctipus]

Describe the bug
I wanted resume the training from a saved checkpoint

The green curve is previously trained model
The red curve is restart of the training after loading 450M steps check point of green curve by the following curve.

As you can see the red curve looks like it didn't load the weight at all as it pattern matches a curve that trained from scratch

below code is how I loaded the model

sac.build_with_env(env)
sac.load_model(os.path.join(MODEL_DIR,'policy.pt'))
sac.fit_online(env, buffer, n_steps=5000000, n_steps_per_epoch=10000,random_steps = 5000, \
	logdir=OUT_DIR,eval_env=env, tensorboard_dir=OUT_DIR+'/logs',save_interval=10, utd=UTD)

I wanted to ask if the above code is how resume training works in d3rlpy, and if not, how to resume training the modeling from a checkpoint with the epoch, learning rate, entropy...etc that left from the checkpoint. Thank you!

[takuseno]

Yeah, your code seems right. Probably, you should drop random_steps option since your policy is pretrained.

[takuseno]

I believe that you should be able to get the expected performance by dropping random_steps. Let me close this issue. Feel free to reopen this if there is any further discussion.