Let lc0 switch to another NN during the game (PR 573 rebased to current master)

[hans-ekbrand]

This a a rewrite of PR 573 against the current master.

Let lc0 switch to another NN during the game. The intended use case is to combine a stronger general NN with and an NN which is better at endgames.

This patch exports five new UCI-options SecondWeightsFile, SwitchAt, SWCpuct, SWFpuValue, SWPolicyTemperature

SecondWeightsFile defines a path to secondary weights file which substitute the primary weights file when SwitchAtpieces remain on the board. SWCpuct, SWFpuValue, SWPolicyTemperature defines Cpuct, Fpu and PolicySoftmaxTemp to be used with the secondary NN.

This PR has been tested with Sergio Vieri's 128x10 net against itself + Medium Ender, but the PR itself is NN neutral.

To verify that the NN is switched, use the LogFile UCI directive and look for the string Will switch to second NN. For transparency, CPuct, FPU and PolicyTemperature is logged at each move, to clarify when the settings for the second NN is in effect.

[jhorthos]

Excellent Hans - thanks. Will probably give it a try even with an inferior end net just to test it out. dkappe has one ender net that is almost as good as T60 and I should have better ones eventually. Oh I see - you already tested with Medium Ender.

One question - do the secondary cpuct values apply to both root and tree cpuct?

[Naphthalin]

@borg323 what is the current status of your hybrid backend, and does it allow switching NNs based on specifyable conditions?

[borg323]

No, but it isn't very hard to do - but it will only work with conditions that can be inferred from the NN input.