ServerResponseTime should use lognormal CDF and not threshold value

[koraa]

ServerResponseTime currently uses a piecewise mapping from raw value (response time) to score:

scoreFromRaw = (responseTime) => Number(responseTime < TOO_SLOW_THRESHOLD_MS)

This sort of threshold value is undesirable as arbitrarily small changes in response time may lead to a jump in the score (and thus drastically increased variance). It is hard to see why a response time of 500 ms is much worse than a response time of 499ms.

Even if an empirical distribution cannot be determined for ServerResponseTime, you could eyeball a relatively good approximation (e.g. p10=500, median=520) which would yield a pretty good score at 500, a perfect score around 480 and an abysmal score at 550 already.

[patrickhulce]

Thanks for filing @koraa! The reason for the threshold here is that due to the way DevTools throttling operates, any values below the threshold are not actually observable to Lighthouse. Even sites that returned in 5ms will appears as though they are taking at least 562.5ms. We agree a smooth curve from a much lower value up to 500 would be far more preferable but there simply wouldn't be any interesting signal in the differences in those scores (in addition to failing a large chunk of fast servers).

[koraa]

Thank you! I didn't know that!

Would it be possible to augment dev tools somehow?

[patrickhulce]

What sort of augmentation are you thinking?

If the root ask is to fix the throttling issues in Chromium, I don't think that's likely. from what I know it's a very large multi-quarter effort that no one here is equipped to tackle in the forseeable future. Getting a smoother server response time curve isn't exactly high on the list of reasons to justify it either ;)

[koraa]

OK, sure that makes sense. Should I close the issue or leave it for $sometime.

[patrickhulce]

We can close for now. If it ever becomes feasible or anyone becomes aware of changes to throttling that make this possible again we'll happily reopen :)