I have some questions about the adjustment of experiment parameters.

[chillybird]

I ran the default experiment "ant-v2, 2x4" and used the default parameters to get the results in the first picture. Later, I modified the parameters (n_rollout_threads: 24, num_mini_batch: 4, ppo_epoch: 40) and got the results in the second picture.

I have also made other modifications to the experimental parameters, but I have not achieved the performance shown in the article in the experiment of "ant-v2, 2x4" provided by the code.
So I want to ask if there are some rules or skills in parameter adjustment of HAPPO / HATRPO algorithm.

[chillybird]

I also experimented with the pybullet.minitaur environment(the return of episode with good results is about 6.5). The first picture is the result of using HAPPO, and the latter is the result of using MAPPO.

The red curve in the first picture represents entropy and the blue one represents reward

[cyanrain7]

I also experimented with the pybullet.minitaur environment(the return of episode with good results is about 6.5). The first picture is the result of using HAPPO, and the latter is the result of using MAPPO. The red curve in the first picture represents entropy and the blue one represents reward

Well, if you want to get better performance on Mujoco by using happo, you can adjust the learning rate of actor as 5e-4 (--lr 5e-4) and modify the num_mini_batch as 1 (--num_mini_batch 1 ), the performance of happo is underestimated by our origin experiment, hope this can help you! (the other parameter not need to modify)

[chillybird]

I also experimented with the pybullet.minitaur environment(the return of episode with good results is about 6.5). The first picture is the result of using HAPPO, and the latter is the result of using MAPPO. The red curve in the first picture represents entropy and the blue one represents reward

Well, if you want to get better performance on Mujoco by using happo, you can adjust the learning rate of actor as 5e-4 (--lr 5e-4) and modify the num_mini_batch as 1 (--num_mini_batch 1 ), the performance of happo is underestimated by our origin experiment, hope this can help you! (the other parameter not need to modify)

Thank you for your reply. I will use these parameters for the next experiment.