Entropy scheduling

[zxzzz0]

• I have marked all applicable categories:
  • exception-raising bug
  • RL algorithm bug
  • system worker bug
  • system utils bug
  • code design/refactor
  • documentation request
  • new feature request
• I have visited the readme and doc
• I have searched through the issue tracker and pr tracker
• I have mentioned version numbers, operating system and environment, where applicable: (Not applicable)

In reinforcement learning there is well known explore-exploit dilemma. In league training it's crucial that we can have a better entropy coefficient scheduling because of the following reasons:
(1) If entropy of policy drops too fast to zero, it might get stuck in local optimum and failed to explore more states.
(2) If entropy of policy drops too slow, it might fail to select the right action at pivotal moments and the training is very slow.

One solution to address above problem is to have a good scheduling. Assume there are some validation measurements that we can use like win rate, we only decrease entropy coefficient when the win rate is on plateau.

It's similar to learning rate scheduler ReduceLROnPlateau in PyTorch link

May us know if there could be some documentations about how entropy scheduling can be supported?

[PaParaZz1]

This problem can be abstracted into a general problem——how to add customized scheduling module in the whole training pipeline, and we will provide some examples in one week.

[PaParaZz1]

This issue has been solved in #38