Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

学習率を0.02と小さくしました #18

Open
yssaya opened this issue Oct 28, 2019 · 4 comments

Comments

@yssaya
Copy link
Collaborator

@yssaya yssaya commented Oct 28, 2019

学習率を今までの
learning rate = 0.001, mini_batch = 64 (mini_batch = 64, iter_size = 1)
から
learning rate = 0.02, mini_batch = 4096 (mini_batch = 128, iter_size = 32)

に変更しました。
次のw775.txtから新しい学習率になります。
見かけの学習率は 0.001 -> 0.02 と大きくなっていますが、ミニバッチが 64 -> 4096 と大きくなっているため
実際の学習率は下がっています。

iter_size = 32、というのは内部でgradientsを32回足し合わせ、仮想的にミニバッチ4096を実現させる
Caffe独自の機能です。

AlphaZeroの論文では
learning rate = 0.2, mini_batch = 4096
から
learning rate = 0.02, mini_batch = 4096
に340万棋譜の時点で変更しています。
AobaZeroは420万棋譜なので、やや遅い変更になります。

論文ではこの後、
learning rate = 0.002 1000万棋譜
learning rate = 0.0002 1700万棋譜
と学習率を下げています。学習は2400万棋譜で終わっています。

2017_2018_alphazero_elo_w668

@sbbdms

This comment has been minimized.

Copy link

@sbbdms sbbdms commented Oct 28, 2019

Could you please release a separate download link for the latest network?
Except in midnight, the network connection to the remote server is too poor from here, recently it becomes even poorer. (300kb/min, or even <100kb/min)
It would be much convenient if there's a separate link, which can be downloaded via a VPN.

@yssaya

This comment has been minimized.

Copy link
Collaborator Author

@yssaya yssaya commented Oct 28, 2019

I have uploaded it as a test. The web server has a low bandwidth, so it may stop if a normal page becomes difficult to see after uploading.

Direct link. I will add a link on http://www.yss-aya.com/aobazero/index_e.html.
http://yss-aya.com/aobazero/w000000000774.txt.xz

@yssaya

This comment has been minimized.

Copy link
Collaborator Author

@yssaya yssaya commented Oct 28, 2019

I have added a link. Latest weight can be downloaded from "Latest weight".

@sbbdms

This comment has been minimized.

Copy link

@sbbdms sbbdms commented Oct 30, 2019

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.