-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Question] TD3_DEFAULT_CONFIG #55
Comments
I assume the regularization noise is always defined by the user for training. A typical configuration can be found in the following example ( skrl/docs/source/examples/gym/td3_gym_pendulum.py Lines 87 to 88 in b760dc7
In this example, the parameters are the same as those used by the TD3 authors to evaluate the algorithm in the paper |
Thanks for your answering @Toni-SM |
Many thanks for reporting this issue. A warning message has been added to the development version ( skrl/skrl/agents/torch/td3/td3.py Lines 160 to 161 in bca2173
skrl/skrl/agents/torch/td3/td3.py Lines 357 to 361 in bca2173
|
Hi @Toni-SM
While utilizing TD3 agent on the pendulum-v1 env with the default config, I checked that "smooth_regularization_noise" is None in default config.
however, that cfg make this below error.
skrl/agents/torch/td3/td3.py in _update(self, timestep, timesteps)
393 # target policy smoothing
394 next_actions, _, _ = self.target_policy.act({"states": sampled_next_states, **rnn_policy}, role="target_policy")
--> 395 noises = torch.clamp(self._smooth_regularization_noise.sample(next_actions.shape),
396 min=-self._smooth_regularization_clip,
397 max=self._smooth_regularization_clip)
AttributeError: 'NoneType' object has no attribute 'sample'
I add the line
cfg_td3["smooth_regularization_noise"] = GaussianNoise(mean=0, std=1)
to avoid that error and it works.Could you check this error and kindly let me know what would be a suitable default value for smooth_regularization_noise in TD3?
The text was updated successfully, but these errors were encountered: