New TD3 hyperparameters really improve the performance?

[zuoxingdong]

Could you confirm that the new hyperparameters for TD3 (i.e. network size from [400, 300] to [256, 256], batch size from 100 to 256, learning rate from 1e-3 to 3e-4) really improve the performance?

In my experiment, it does not demonstrate a consistent improvement.

[sfujim]

The new hyper-parameters are necessary for TD3 to learn on Humanoid, but there shouldn't be any big changes on the other environments.

[zuoxingdong]

@sfujim thanks for your reply!

Did you mean that for other environments (e.g. HalfCheetah, Hopper etc.), we should use the original set of hyperparameters and use the new one only for Humanoid?

[sfujim]

If you're only interested in maximizing performance then probably. The original hyper-parameters were also not well-optimized as we originally wanted to stay close to DDPG for a fair comparison. If you are comparing to other methods which don't use per-environment hyper-parameters then in my opinion using only one set of hyper-parameters is more fair but it's up to you & your use-case. Best of luck!