Make sure all features of lczero.exe are implemented in lc0

[mooskagh]

• Time management (aka smart pruning)
• FPU reduction
  ...

[mooskagh]

So for now there are the following differences between lczero and lc0:

• (done!) FPU reduction not implemented

• (done!) Time management (aka smart pruning) is not implemented

• (done!) There is "virtual loss bug" which is not ported

• No CPU backend

• No openCL backend (or other way to support AMD)

• gzipped weights file is not supported (possibly move that to client.exe side)

• training data is not gzipped (possibly move that to client.exe side)

• (fixed!) lc0 plays weaker. 1200 nodes lc0 is roughly equal to 800 nodes lczero. Reasons unknown. Possible hypothesis:

  • due to "virtual loss bug"
  • Networks return slightly different results on openCL and TF weights, and it was trained on openCL weights, so it performs slightly better.
  • due to smart pruning
  • There is indeed a bug, which is hard to notice by debugging random moves.

[mooskagh]

For the weaker play, the plan is to:

• have "virtual loss bug" fixed in lczero to sync up mcts algorithm
• port openCL backend from lczero to lz0 to sync up NN computation
• after run evaluation of the same position on lczero and lc0 and debug differences

[jjoshua2]

And now there is option in lczero for 6 piece syzygy WDL and DTZ too. WDL shouldn't be too hard to port, but DTZ root probing relies on SF similarities. @Tilps?

[Tilps]

The existing tbprobe.cpp logic for both depends on having a material key and are written specifically to use stockfish's Position class API. But should be easy enough to port to some other arbitrary position class.
root_probe (whether using dtz or wdl) depends on having an 'active' status to stop search from using the root moves that root_probe determines are inferior - which requires the 'smart pruning' to be added. Once that exists it should be pretty straight forward.

[killerducky]

I would add smart pruning to the candidates for why it's 1200nodes to equal 800nodes. It gained quite a lot of Elo for LZGo. Quickest way to test is just disable smart pruning in lczero. If someone doesn't do it first I can try this maybe by Wednesday night.

[mooskagh]

When I disable both "smart pruning" and "virtual loss bug", lc0 and lczero produce nearly identical results. https://docs.google.com/spreadsheets/d/1jltiG1l8T0U9tOaIplDnragi60kMGPyCu2tYmRwNXMA/edit#gid=0

I'm going to port both into lc0 then.

[mooskagh]

I've implemented "virtual loss bug" and smart pruning, and from quick tests it looks promising. Will wait for more thorough tests to complete.

[mooskagh]

• Fpu reduction implemented and enabled (to 0.2)
• Virtual loss bug disabled
• Last input plane is all ones now like in lczero
• CPuct adjusted to 1.2

[mooskagh]

This issue is covered by smaller issues, so closing this umbrella bug.