Update q to cp conversion formula #1193
Merged
+5
−2
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Based on the current TCEC games, the Q to centipawn conversion formula seems to be too pessimistic at high Q. lc0 cp evaluations are often much lower than the other engines.
I downloaded the TCEC SUFI evaluations and compared to the lc0 cp evaluations to SF evaluations. Below are the results:
The evaluation for Q < 0 comes from only one game and can be ignored. The current formula is the line labeled with centipawn. The first too optimistic conversion function is centipawn_2018 line. I also plotted the conversion function from #841 that used the old equation but fitted to more games. The #841 equation seems to fit very well to the median SF evaluation. I can get slightly better fit with more complex functions, but this one has advantage of being easily invertible which is nice for calculating the Q from centipawns. This PR reverts the conversion function to one from #841.
The average error to median SF eval on this dataset is 0.79 pawns with the current function, 0.28 with this function and 1.85 with the centipawn_2018.
The script for downloading evaluations and plotting the graphs can be found at: https://gist.github.com/Ttl/9926f70800b3fbd7314c150108d4ba61