The runtime section explains the metrics available for benchmarking the runtime performance.
This is the default option for benchmarking runtime. If no argument is passed to the -rt
flag from the cli, then the average runtime of n iterations would be represented on the frontend. It can however also be set explicitly by passing mean
argument to the -rt
flag from the cli
This option can be selected by passing minimum
argument to the -rt
flag from the cli. This will choose the lowest runtime of n iterations to be represented on the frontend.
runtime = min [x1, ...., xn]
This option can be selected by passing maximum
argument to the -rt
flag from the cli. This will choose the highest runtime of n iterations to be represented on the frontend.
runtime = max [x1, ...., xn]
This option can be selected by passing first
argument to the -rt
flag from the cli. This will choose the first runtime of n iterations to be represented on the frontend.
runtime = x1
This option can be selected by passing median
argument to the -rt
flag from the cli. This will calculate the median runtime of n iterations.
where runtimes = [x1 ≤ x2 ≤ ... ≤ xn] .
This option can be selected by passing harmonic
argument to the -rt
flag from the cli. This will calculate the harmonic mean of n iterations and display it on the frontend.
Note
Mathematically none of the values should be zero as taking an inverse is an intermediary step when calculating the harmonic mean. However, the harmonic_mean
funtion from the statistics
library will handle the zero values in the input without throwing a ZeroDivisionError
.
This option can be selected by passing trim
argument to the -rt
flag from the cli. This will calculate the trimmed mean of n iterations and display it on the frontend. The trimmed mean is calculated using the following steps.
- Order the runtime results [x1 ≤ x2 ≤ ... ≤ xn]
- Remove 20% of the lowest and highest runtimes.
- Calculate the arithmetic mean of remaining runtimes using Equation
$\ref{1}$ .