Contextual example to highlight a problem any new App Mining governance should solve

[cuevasm]

As we have seen, the period we've allowed for App Miners to adjust to new rules (i.e. not making changes in a scoring period) has also opened the door for folks to exploit the lag in decision-making and stay ahead of any changes. It's also handcuffed App Reviewers from quickly bettering the end scoring result.

A clear example is in Awario where we see a new site or two rise up each month that clearly have a wonky Reach score and/or obviously isn't a News/Blog site, but is counted as such. Unfortunately, what has happened is that since we can't adapt to these as they come up (there is an unclear decision-making process that takes at least 1 month or more), people game these holes for Reach. I'm not talking fringe cases or grey areas, I'm talking obvious stuff here.

An example this month is Upwork, @wilsonbright was kind and noble enough to report it himself and say I could remove it. However, the current rules would preclude me from making this change without a month decision period because we can't make in period changes (and the last time I tried to push one through quickly, enough people benefiting fell back on this rule and we had to wait, letting the scores suffer in the meantime.)

In summary, I think the decision-making process should be faster, of course. Specifically, I think you could accomplish this by better empowering App Reviewers or whatever plays a similar function, to enforce the spirit of their review vs. being held to super-specific policies. This wouldn't only push out bad actors easier, but would reward good ones more as well. This month, I had to tell poor @joberding that her mentions didn't count because she added a hashtag to the front of them. Now, are those clearly mentions of her apps? Yes. Can I make that change to her benefit in the scoring period? Sadly no. I've also had to disallow Black Hole vs BlackHole for @Walterion01. These things seem silly, but under the current rules, there is no way to make good faith exceptions (and I can see how it could turn into a slippery slope too).

So it works both ways and I think a stronger score is generally more important than giving notice about changes, even if it gives the occasional bit of whiplash in a scoring period. By moving more quickly on changes to the scores, you can get toward an overall better ranking more quickly too, which I think benefits everyone more in the long-run.

[njordhov]

For context, these are examples of sites @cuevasm says current rules would preclude him from removing:

• 285K Reach from posting job announcements on Upwork (Mumble)
• 445K Reach from a post in a LinkedIn channel with 181 followers (dMail)

Related issues:
#221 Remove more self-posting sites with wacky Reach numbers from Awario scores
#208 Remove more self-publishing platforms from Awario News/Blog reach
#199 Calculation of Awario social scores can easily be gamed
#194 Better structured proposal process
#190 Remove sponsored articles from News/Blog reach for Awario
#189 Remove Meetup.com from Reach in News/Blog for Awario
#171 Remove medium reach from blended awareness score

[cuevasm]

Yes, when I've tried to move more quickly to remove clearly odd sites, I get pushback from folks because we have the standing rule that we can't make changes during a scoring period. Even for ones that seem obvious, I will get backchannel messages or enough people saying to wait on the tickets - and I can't deny that's correct, we all agreed not to make changes in period. This is exacerbated by the unclear ratification process when we do agree.