add a function to compute new metrics from tuning results

[simonpcouch]

A first step for infrastructure for fairness metrics! This function is closely analogous to collect_metrics(), but allows for computing new metrics after tuning without needing to re-fit and re-predict from resampled models.🏄

# load needed packages:
library(tune)
library(parsnip)
library(rsample)
library(yardstick)

# evaluate a linear regression against resamples.
# note that we pass `save_pred = TRUE`:
res <-
  fit_resamples(
    linear_reg(),
    mpg ~ cyl + hp,
    bootstraps(mtcars, 5),
    control = control_grid(save_pred = TRUE)
  )

# to return the metrics supplied to `fit_resamples()`:
collect_metrics(res)
#> # A tibble: 2 × 6
#>   .metric .estimator  mean     n std_err .config             
#>   <chr>   <chr>      <dbl> <int>   <dbl> <chr>               
#> 1 rmse    standard   3.62      5  0.172  Preprocessor1_Model1
#> 2 rsq     standard   0.746     5  0.0129 Preprocessor1_Model1

# to compute new metrics:
compute_metrics(res, metric_set(mae))
#> # A tibble: 1 × 6
#>   .metric .estimator  mean     n std_err .config             
#>   <chr>   <chr>      <dbl> <int>   <dbl> <chr>               
#> 1 mae     standard    2.89     5   0.151 Preprocessor1_Model1

# if `metrics` is the same as that passed to `fit_resamples()`,
# then `collect_metrics()` and `compute_metrics()` give the same
# output, though `compute_metrics()` is quite a bit slower:
all.equal(
  collect_metrics(res),
  compute_metrics(res, metric_set(rmse, rsq))
)
#> [1] TRUE

^{Created on 2023-04-05 with reprex v2.0.2}

There is likely some additional work to do for accommodating iterative tuning, so marking as a draft for now!

[DavisVaughan]

Lots of minor comments, but nothing super blocking so I will approve too

[topepo]

Looks great!

[github-actions]

This pull request has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.