Rate command #81
Rate command #81
Conversation
…e tests. no noticeable speedup
…raditional ranges
… and thus running ratings
| @@ -0,0 +1,22 @@ | |||
|
|
|||
|
|
|||
| def rating_to_traditional_range(rating_f): | |||
There was a problem hiding this comment.
Does "traditional range" actually exist? My impression of AGA raw ratings was that all positive ratings were "rounded up" so that [0, 1] = 1dan, whereas negative ratings were rounded down so that [-1, 0] = 1kyu.
There was a problem hiding this comment.
the 'kyu-dan boundary' is between [-1,1] -- you can see how they're adjusted in the current regime here: https://github.com/usgo/AGA-Ratings-Program/blob/master/collection.cpp#L356.
The math on that one is a little different; IIRC, it maybe enforces a lower bound of -30, but there is no upper bound enforced in the math. The ratings that expand up past 7.0 are only anchored by the fact that there's a lot of priors, IIUC
There was a problem hiding this comment.
(i'm pretty happy about the fact that, with ZERO priors, zero anchors, zero thumbs-on-the-scale, we end up with a really close match on a couple dozen different players i looked up. Would love to turn that into a quantifiable thing, but i'm really happy so far)
| @@ -36,44 +34,45 @@ def sanitized_games(games): | |||
| elif not (g.result.startswith('W') or g.result.startswith('B')): | |||
| print('unknown result: ', g) | |||
| pass | |||
| elif g.date_played is None or g.date_played.timestamp() == 0.0: | |||
| print('No date played: ', g) | |||
| pass | |||
There was a problem hiding this comment.
I was confused by pass, since the intent is to skip this game. I'd use continue everywhere just for consistency, even though code-wise it executes identically as pass in this case because the final action is in an else branch.
| @@ -100,8 +99,10 @@ def rate_all(t_from=None, t_to=None, iters=200, lam=.22): | |||
| for k,v in rating_prior.items(): | |||
There was a problem hiding this comment.
Is this sensitive to outliers? If you got one unusually bad or good player, would everyone else end up with squished ratings? Maybe it'd be more robust to use the mean and std deviation and rescale that way.
There was a problem hiding this comment.
Also, this seems like the kind of renormalization that should happen after the algorithm is finished, but I can also see why it would help stabilize the iterations within the loop.
There was a problem hiding this comment.
re: outliers it's certainly possible. I didn't test any crazy outliers on the bottom side -- on the top side, it seems to do 'the right thing' on the AGA's set, which has some undefeated players on the top like Rui Naiwei, Myungwan Kim, etc.
re: renormalization: I'm not sure. I added this when i was still having some non-convergence issues, but it might not still be needed.
| elif rating < 7: | ||
| return "%dd" % int(rating) | ||
| else: | ||
| return "7d" |
There was a problem hiding this comment.
AlphaGo can only be 7d AGA :D
There was a problem hiding this comment.
yeah this might need to be something else later ;)
There was a problem hiding this comment.
I thought the AGA ratings went up to 9D?
There was a problem hiding this comment.
"ratings" go up to 11ish. Ranks only go to 7d.
|
Is this PR ready to be merged? |
|
hmm, i think so. Let me give it a quick pass |
Further refactors & cleanup from your previous comments. Runs all good...
Next steps are refactoring to save a little state on repeated runs -- would be useful for bootstrapping/testing as we try and find the proper values of these parameters.