Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Monte Carlo Tree Search #12

Open
PascalCremer opened this issue Jul 17, 2019 · 2 comments
Open

Implement Monte Carlo Tree Search #12

PascalCremer opened this issue Jul 17, 2019 · 2 comments
Assignees
Labels
enhancement New feature or request

Comments

@PascalCremer
Copy link
Contributor

PascalCremer commented Jul 17, 2019

One can implement MCTS based on the value model to improve model performance. This is not intended to be used during data generation due to the high cost of computation.

@PascalCremer PascalCremer added the enhancement New feature or request label Jul 17, 2019
@PascalCremer PascalCremer self-assigned this Jul 19, 2019
@cleeff
Copy link
Collaborator

cleeff commented Jul 19, 2019

I implemented a first MCTS version which can already me used in interactive mode. I thought about implementing this as a model wrapper but the interface doesn't quite match. Maybe we can find a way to merge the two interfaces s.t. MCTS can be used in comparison with models without MCTS without too much extra code.

c_puct Parameter certainly needs to be tuned. Also currently the prior policy is set to 1 uniformly. Softmax of the move evaluation might make more sense and should certainly be tried.

@cleeff cleeff assigned cleeff and unassigned PascalCremer Jul 19, 2019
@cleeff
Copy link
Collaborator

cleeff commented Jul 19, 2019

I just added a sigmoid(model_output) factor to U. Otherwise the first move will be entirely random. Now it looks much better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants