Curious about katago elo

[jaewoo1120]

Based on katago Elo, what do you expect the score for fineart and Golaxy to be?
It might be awkward because it was written in Google Translate

[Friday9i]

Based on the selfplay Elo, you cannot compare! Why: because we don't know Fineart selfplay Elo 2) Even if we knew, their process to compute selfplay Elo is for sure not identical 3) Even if identical, there's little reason that 2 equivalently strong engine get comparable selfplay Elo...
The only way to know is to organize matches with 100s of games and well define and comparable hardware

[jaewoo1120]

Thank you for your answer. I know that Katago and other artificial intelligence are not comparable using Katago elo scores.
Wouldn't you be able to predict in some degree some things like blue spot match rate?

[jaewoo1120]

In particular, golaxy can be played directly on sites such as 19x19.com. Specifications are also described. I think we can get a lot of information from the site.

[Lefannew]

You couldn't compare kataGo with likes of Golaxy,Fineart,and MyrtleGo.
Since they are not in the same scaling system,for instance,you know LeelaZero's elo is 16k,however the strength is no where close to kataGo.Elo is nothing but a scaling system for a particular project.
On the other hand,you talked about Golaxy's website,I don't know if you know the Golaxy devs already publicly stated that they don't give users the best network. Plus you wouldn't know hardware usage and network details based on Golaxy's website.Therefore kataGo's elo can only compare with itself,not any other bots.

[OmnipotentEntity]

Also Golaxy seems like it might actually use open source nets, including katago's, in their games (but not in analysis), in order to remove blind spots, which confounds direct comparison and analysis.

[jaewoo1120]

Thank you to everyone who answered!! :)
I understand that the original question, "Using Elo, Catago and other artificial intelligence cannot be compared."

I don't know much about AI, so I didn't quite understand Omnipotent Entity.
Does it mean that there can be an open-source nets like KATAGO in the GOLAXY, which eliminates the confusion of the GOLAXY?
That is different from the original question, but I would appreciate it if you could explain in what way it works to put open source inside.

It might be awkward because it was written in Google Translate

[Lefannew]

Based on what I found from an interview question
A guy named “calmgo" asked team Golaxy : Can I evaluate the player's strength through AI coincidence rate or other games sgfs analysis methods?

Golaxy: In fact, this method is not scientific. Take Golaxy against LeelaZero games in the semi-finals. Some enthusiasts used katago analysis to evaluate both LeelaZero and Golaxy. Leela's coincidence rate was more than 70％, while Golaxy is only over 50％, which is unconvincing. Golaxy is trying to use other methods to evaluate the strength of Go players.

Also lightvector said: "coincidence rates are a terrible metric to pay attention to for most purposes. They say very little about strength."

Basically "coincidence rate" is nonsense from where he and team Golaxy stand.