add collect_*() function for extracted objects

[simonpcouch]

Closes #409.🦃 I opted to call this helper collect_extracts() rather than the suggested name in the issue for consistency with other collect_*() function names, but am open to discussion here.

library(tidymodels)

res_fit <- 
  fit_resamples(
    linear_reg(),
    mpg ~ .,
    bootstraps(mtcars, 5),
    control = control_resamples(extract = extract_fit_engine)
  )

res_nothing <- 
  fit_resamples(
    linear_reg(),
    mpg ~ .,
    bootstraps(mtcars, 5)
  )

res_error <- 
  fit_resamples(
    linear_reg(),
    mpg ~ .,
    bootstraps(mtcars, 5),
    control = control_resamples(extract = function(x) {stop("no!")})
  )
#> x Bootstrap1: preprocessor 1/1, model 1/1 (extracts): Error in extractor(object): no!
#> x Bootstrap2: preprocessor 1/1, model 1/1 (extracts): Error in extractor(object): no!
#> x Bootstrap3: preprocessor 1/1, model 1/1 (extracts): Error in extractor(object): no!
#> x Bootstrap4: preprocessor 1/1, model 1/1 (extracts): Error in extractor(object): no!
#> x Bootstrap5: preprocessor 1/1, model 1/1 (extracts): Error in extractor(object): no!

collect_extracts(res_fit)
#> # A tibble: 5 × 3
#>   id         .extracts .config             
#>   <chr>      <list>    <chr>               
#> 1 Bootstrap1 <lm>      Preprocessor1_Model1
#> 2 Bootstrap2 <lm>      Preprocessor1_Model1
#> 3 Bootstrap3 <lm>      Preprocessor1_Model1
#> 4 Bootstrap4 <lm>      Preprocessor1_Model1
#> 5 Bootstrap5 <lm>      Preprocessor1_Model1

collect_extracts(res_nothing)
#> Error in `collect_extracts()`:
#> ! Failed to collect extracted objects.
#> ℹ Please supply a control object (`?tune::control_grid()`) with a non-`NULL`
#>   `extract` argument during resample fitting.

collect_extracts(res_error)
#> # A tibble: 5 × 3
#>   id         .extracts      .config             
#>   <chr>      <list>         <chr>               
#> 1 Bootstrap1 <try-errr [1]> Preprocessor1_Model1
#> 2 Bootstrap2 <try-errr [1]> Preprocessor1_Model1
#> 3 Bootstrap3 <try-errr [1]> Preprocessor1_Model1
#> 4 Bootstrap4 <try-errr [1]> Preprocessor1_Model1
#> 5 Bootstrap5 <try-errr [1]> Preprocessor1_Model1

^{Created on 2022-11-17 with reprex v2.0.2}

Interactively in the IDE, the error message with collect_extracts(res_nothing) formats control object as a hyperlink to those docs, rather than printing the link.

[hfrick]

I like the name!

I think it would be helpful if the error said that there was nothing to collect, rather than that collection failed in general (there could be so many reasons for this). This is also in the spirit of what collect_predictions() does.

collect_notes() returns location while collect_extracts() return .config. They look similar; are they the same information just displayed differently?

Do we still want a collect_glmnet_coefficients()? If so, then let's open a new issue for it so that this idea doesn't get lost when closing #409

[simonpcouch]

I think it would be helpful if the error said that there was nothing to collect, rather than that collection failed in general (there could be so many reasons for this). This is also in the spirit of what collect_predictions() does.

Agreed! Addressed in 265e311. :)

collect_notes() returns location while collect_extracts() return .config. They look similar; are they the same information just displayed differently?

Similar information, though my understanding is that locations are put together for interactive debugging—they only occur when things go wrong, and also include information on which part of the tune code path (i.e. within-Preproc/Model/Repeat/Iter) the issue occurred in—whereas .configs are constructed for straightforward joining.

Do we still want a collect_glmnet_coefficients()? If so, then let's open a new issue for it so that this idea doesn't get lost when closing #409

You got it! #582

[hfrick]

🚢 Nice!

[github-actions]

This pull request has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.