Improving the "Smart" algorithm

[mikekasprzak]

Looking at the metrics for this game:

https://ldjam.com/events/ludum-dare/40/tax-evasion-hero

Karma for Feedback given: 6
Ratings given: 112.625
Ratings received: 35.1429
Classic Balance: 183.3397 *
Smart Balance: 20.2661 *

The Karma score is low, but they've given a lot of ratings since last week (>100). I was going to suggest that there was a design bug, but I forgot that Karma given is from comments on other games. So that 6 is likely justified.

That said, to keep things more dynamic, it might be worth adjusting the limits over time.

For example, there's a hard limit of 50 points you can earn from ratings in the algorithm. What might be better is instead, lower that limit to 30, and every day raise the limit by 2. So by the end of the first week the limit will be ~44, and by the end of the event ~72.

Those figures aren't necessarily perfect, but the goal here is to lower the effect of abuse right-out-of-the-gate. i.e. Rating 50 games in the first two hours. Right as the event ends we are the busiest, and people are most interested in playing+rating immediately after the event. Spreading out the "love" if you will

This has the added benefit of adding something to look forward to every day, and a reason to come back and check daily (hopefully being motivating enough to play more games while you're here).

Pacing out the long tail of extra scores has the additional benefit of giving actual time for the Feedback scores to accumulate points. Feedback Karma requires actual other members to acknowledge a comment to see benefit, and it takes time for real people to find and see stuff.

[local-minimum]

35 is a bit low, and I guess you should be able to get rated even if you never comment. But if they had commented in any decent form, I bet they would have reached 100 in total thanks to play back people like me.

Other solution is to inform those with low karma but many rated of this feedback as strategy.

Other solution is to have multiple ranking systems live at once and have every fifth position be from alternative method.

The dynamic calculation idea feeds into the larger question of which sorting filter to have as default.

[mikekasprzak]

Other solution is to inform those with low karma but many rated of this feedback as strategy.

Oh interesting. Offer this as a tip while voting.

[zwrawr]

#1648

[local-minimum]

From #1648:

Rather than have to change it, I wouldn't mind making the "smart" filter smart enough to include game
near the cut-off just to push them over. It is called "smart" after all. ;)

There's a problem with that there will always be games needing to be pushed over the threshold so one way to solve this in the general sense would be to have the smart filter smart enough to make errors.

// f1 - f3 are scaling functions. f1 and f2 could be logarithmic functions. f3 needs to be a special function that has two local minima (at 0 and 20). Dynamics over time can be controlled by the log base of f1 and f2.
a = f1(karma)
b = f2(votesGiven)
c = f3(votesRecieved)

baseScore = (a + b) / c

// Add an error, e.g. from normal distribution scaled by magnitude of the baseScore
score = baseScore + e(baseScore)

My thought is that randomness will make it more fair because it means everyone that deserves a chance to be played will get one without being stuck around on page two for ever. f1 and f2 would also increase the incentive to keep playing and rating for those doing a lot of work since it removes the hard cap and makes it soft.