model.predict called before model.train?

[shanemcandrewai]

Hi,

From the code, I see trainer.learn() calls self.exceute_episode() which in turn, calls self.mcts.run when then calls model.predict(state). Is it intentional to predict the policy and value before training the network?

[JoshVarty]

Yes, it's intentional to use the police/value networks before they've actually been trained.

Initially they are randomly generated (and probably don't make good predictions). However, we use these random police/value networks to play a bunch of games and we keep track of who won/lost in each game. We use these games as a temporary dataset and train each random policy/value network to be slightly better.

We repeat this process many times. Although we started with a random policy/value network, over time we have improved it such that it's no longer random and actually starts to accurately predict moves and values.