[Question] Why average actor's gradient in DDPG but not in SAC ?

[Gaiejj]

Required prerequisites

• I have read the documentation https://omnisafe.readthedocs.io.
• I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
• Consider asking first in a Discussion.

Questions

We utilize distributed.avg_grads to average the gradients of the actor and critic network in DDPG, but not in SAC's actor network. In fact, due to the current lack of support for off-policy algorithms in the A3C parallel paradigm (preliminary checks are present in the omnisafe/algorithms/algo_wrapper.py file, and off-policy algorithms will not initiate training in the A3C parallel paradigm), in practice, gradient averaging is not performed during the training process of off-policy algorithms. A future pull request will removes redundant code, enhances readability, and does not affect the performance of the algorithm.

[Gaiejj]

Pull request #253 solve it.

[zmsn-2077]

Already approve PR #253.