core(scoring): loosen metric thresholds

[patrickhulce]

As we just discussed, the new scoring thresholds based on 50/75 were making the requirements for 99/100 too unachievable. The root issue was that our PODR was not considered with our policy. The new policy instead...

Median === 75th percentile
PODR = 95th percentile

This makes a 100 roughly equal to the 98th percentile and keeps the green/orange boundary of 75 to the ~87th percentile which is pretty close to our old policy. Checkout the new scoring calculator sheet https://docs.google.com/spreadsheets/d/1Cxzhy5ecqJCucdf1M0iOzM8mIxNc7mmx107o5nj38Eo/edit#gid=0 to play around with the values.

[brendankenny]

LGTM

does this affect the decisions that went into #5083?

[patrickhulce]

does this affect the decisions that went into #5083?

for better or for worse, the score thresholds/curves of the metrics themselves were not considered at all as part of weighting committee rulings, so this doesn't affect it :)

[kaycebasques]

@patrickhulce what does PODR mean?

[patrickhulce]

@kaycebasques it stands for Point Of Diminishing Returns

It's roughly a score of ~97 and is the point on the log normal probability curve where the derivative reaches its peak. Basically, going farther past that point will yield minimal increases to the score.

[connorjclark]

It's roughly a score of ~97

FYI for future code archaeologists: PODR is not necessarily ~97, it varies based on what the median is set as. Everything else @patrickhulce said is true though.

[patrickhulce]

Yes, future code archaeologists please do not interpret "roughly a score of ~97" to mean that it in any way is trying to target a score of 97. It just so happened that for many of our curves the PODR is 99-95, so roughly ~97 felt appropriate to say. Everything else @connorjclark said is true 😉