Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Will anyone use KataGo to attend the World AI GO Competition on 27, Sep in Fuzhou, China? #320

Closed
HackYardo opened this issue Sep 24, 2020 · 19 comments

Comments

@HackYardo
Copy link

The English introduce of this competition on 2019 is here: leela-zero/leela-zero#2278
The Chinese introduce of this competition on 2020 is here: https://sports.sina.com.cn/go/2020-09-08/doc-iivhvpwy5522546.shtml
The email address of this competition is here: fuzhouworldaigo@sina.com
The deadline to enroll this competition is Sep 25 00:00 (UTC+8), there is only 10 minutes left.
I know KataGo is powerful, and I hope it will attend this competition, but I don't know if someone have enrolled.

@OmnipotentEntity
Copy link
Contributor

There have been some people in China who are collaborating with lightvector, but the language barrier has been a problem, and he requested assistance with this a while ago via discord. I do not know if anyone stepped up to help. But 10 minutes is probably a little short notice ;D

@lightvector
Copy link
Owner

lightvector commented Sep 27, 2020

Yep, as you've probably noticed in announcements from elsewhere, yes someone has arranged to run it in the tournament.

KataGo's participation is mostly for fun. I'm not 100% confident in exactly what configuration and hardware the person or group who will operate it intend to use, and although I trust that they're interested in KataGo doing well and configuring it appropriately, the communication has not been the easiest due to the language difference.

For the record, stating my guess prior to the tournament taking place, sort of in response to some of the messages I've received privately and through other channels from people hyped for KataGo to be in this tournament, some of which have seemed bizarrely-to-me highly-secretive and cutthroat about beating others and winning:

I think that KataGo is unlikely to place near the top (but perhaps it will, it would be a pleasant surprise). Unlike many competitors, KataGo is open source - its code, its data, and its techniques. KataGo (I hope) has pushed the state of the art in its field of research - and doing that well and sharing freely with the community has always been the goal. I doubt it would be hard for a dedicated group of developers, particularly ones working full-time, to replicate its methods, and if any of them happened to know any further improvements and new techniques not yet published so far, it would not be hard for them to improve yet further. Undoubtedly, there are many further algorithmic and training improvements to be found, and I'd find it shocking if among many smart researchers it weren't the case that more things have been found. And of course there are undoubtedly well-funded research groups and labs out there with much larger amounts of computation for training and testing available to them, that KataGo's resources would be no match for.

Tournaments like this are primarily for entertainment. They don't involve the hardware controls or the hundreds to thousands of games you'd run when actually trying to scientifically compare different programs and training methods and architectures, instead the focus is on commentary and engaging and bringing people together in a shared activity.

So let's just enjoy the entertainment. Enjoy spectating the games, best of luck to everyone of course, and have fun!

@HackYardo
Copy link
Author

There have been some people in China who are collaborating with lightvector, but the language barrier has been a problem, and he requested assistance with this a while ago via discord. I do not know if anyone stepped up to help. But 10 minutes is probably a little short notice ;D

Thanks for your reply, my meaning is a consultation, not a notice. I'm happy to see the information about that KataGo may attend this competition. And in the condition of the same time and hardware, KataGo is stronger than Golaxy and FineArt based on the tests made by some amateurs and professors in China.

@HackYardo
Copy link
Author

@lightvector
Thanks for your reply. This tournament is just for fun because it doesn't involve the control of hardware. And KataGo's goal is pushing researches and sharing. However, I believe it will do well in this tournament!

@HackYardo
Copy link
Author

HackYardo commented Sep 27, 2020

Did you ever contact to the AGA(American Go Association), it is one co-organizer of this tournament? They may give help to the problem of language difference.

@HackYardo HackYardo reopened this Sep 27, 2020
@l1t1
Copy link

l1t1 commented Sep 27, 2020

@mega-optimus
Copy link

There seems to be some incidents on the first day, don't know what was the cause.

@hwj-111
Copy link

hwj-111 commented Sep 28, 2020

I just feel that the result of this kind of tournaments without any specs on hardware/energy consumption is less meaningful. The bots win most likely due to they have more computing power.

It will be cool if there are some AI Go competitions that has an up-limit on power consumption (say, <1000W)

@l1t1
Copy link

l1t1 commented Sep 29, 2020

player win lost
LeelaZero 4 2
KataGo 4 2

@lightvector
Copy link
Owner

lightvector commented Sep 29, 2020

player win lost
LeelaZero 4 2
KataGo 4 2

Some of these games include unfair wins and losses due to timeouts or failure to connect due to unstable network or internet at the competition site, for both Leela Zero and KataGo. Many other participants have also experienced timeouts or other issues - I think somewhere around a quarter to a third of the games in the tournament as a whole in the first two rounds were decided by these issues, rather than by the actual skill of play of the bots, and some games in later rounds too. Possibly the lags have also influenced games where it wasn't quite bad enough to cause a timeout. And there have been other discussions that make you sort of question what's going on about other details too.

Even after going in expecting (as I mentioned in my initial post above) that the results in tournaments like this aren't worth trusting too much because of such a small sample size and lack of controls, definitely feeling a disappointed in this tournament due to the mess of issues above, regardless of what the outcome from here is.

@mega-optimus
Copy link

image
current leader board

@HackYardo
Copy link
Author

@mega-optimus
Because loss in round1 and round2, 'KataGo' has lower points than '石子旋风', so that the latter promotes to top8 instead of the former.
Preliminary ranking calculation methods are compared in turn: 1. The points of each team, the higher the points are ranked first; 2. The sum of the opponents of each team, and the higher the total of the points are ranked first;

@lightvector
Some said this tournament's rule is new:

rule=Chinese, komi=6.5, button=used

Players need to fit the new rule by switcing AI during playing, and as a result, timeout is everywhere.
Another issue is that KataGo is open, everyone can use it for the goals of themselves.
So as you mentioned, let's just have fun!

@l1t1
Copy link

l1t1 commented Sep 30, 2020

@alreadydone
Copy link

Interview of lightvector by Yike published: https://mp.weixin.qq.com/s/XWMFvQYB8a5hBcpIHmz5DQ

@HackYardo
Copy link
Author

@alreadydone
Interview of lightvector by Branton Demoss at the end of the article: https://brantondemoss.com/writing/kata/

@l1t1
Copy link

l1t1 commented Nov 30, 2020

semi final matches began

@l1t1
Copy link

l1t1 commented Dec 1, 2020

someone says two of the top four are using katago https://m.newsmth.net/article/Weiqi/648486

@l1t1
Copy link

l1t1 commented Dec 1, 2020

@HackYardo
Copy link
Author

HackYardo commented Dec 1, 2020

someone says two of the top four are using katago https://m.newsmth.net/article/Weiqi/648486

mmexport1606694908092

In their own statements, four in eight are using KataGo, they are ChaoRan, YiLe, izis and Tian.
@l1t1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants