Skip to content

Commit

Permalink
Add comments
Browse files Browse the repository at this point in the history
  • Loading branch information
ehmicky committed Dec 19, 2021
1 parent ff0d0fb commit 0047e84
Showing 1 changed file with 30 additions and 0 deletions.
30 changes: 30 additions & 0 deletions src/stats/variance.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,45 @@ export const getVarianceStats = function (
return { variance, stdev, rstdev }
}

// All measures are sometimes identical due to:
// - Low time resolution
// - Custom units
// - Boolean units
// However, this leads to `variance` being 0, which:
// - Ends the benchmark too early
// - Leads to poorer `diffPrecise`
// We fix this by using the `variance` of the measures as if the next measure
// was not identical:
// - Because if the `variance` is still low enough for benchmark to end, then
// it is not worth computing that extra measure, i.e. continuing the
// benchmark
// - Also, if the following measures are not identical, the difference of
// `variance` will be minimal, ensuring a smooth transition
// The hypothetical, non-identical next measure is computed to be a small
// percentage away for the identical measure
// - We use a relative percentage instead of an absolute value like `1` because
// - It is unitless, giving the same results regardless of the `mean`
// magnitude
// - It works well with floats
// Outliers are excluded from this logic, i.e. they might not match the
// identical measures
// - Therefore, we detect this situation with `min === max`
const getIdenticalVariance = function ({ minIndex, maxIndex, mean }) {
const sumDeviation = getIdenticalSumDeviation(mean)
return computeVariance(sumDeviation, minIndex, maxIndex + 1)
}

// This is like `getSumDeviation()` with an array of repeated measures plus
// an additional, slightly different one.
// Since repeated measures have `0` deviations, we can simplify the formula.
const getIdenticalSumDeviation = function (mean) {
return (mean * IDENTICAL_VARIANCE_SHIFT) ** 2
}

// Relative percentage of shift between the repeated measure and the additional
// one.
// A higher value makes identical measures end too early.
// A lower value makes benchmarks slower.
const IDENTICAL_VARIANCE_SHIFT = 1e-2

// Retrieve variance of an array of floats (cannot be NaN/Infinity).
Expand Down

0 comments on commit 0047e84

Please sign in to comment.