Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to tune penalty for glmnet with non-default family #738

Closed
cb12991 opened this issue May 31, 2022 · 9 comments · Fixed by #890
Closed

Unable to tune penalty for glmnet with non-default family #738

cb12991 opened this issue May 31, 2022 · 9 comments · Fixed by #890
Labels
bug an unexpected problem or unintended behavior

Comments

@cb12991
Copy link

cb12991 commented May 31, 2022

The problem

I'm unable to tune the penalty hyperparameter for a glmnet model specification with non-default family engine argument. I've traced the error down to the multi_predict generic not having a method for generalized model fits (class _glmnetfit).

I have also tried passing specific path_values as an engine argument (which I think will need to be done to correctly compare penalty values) but that didn't resolve the underlying issue with multi_predict.

Reproducible example

library(tidyverse)
library(tidymodels)

recipe <- recipe(hp ~ ., data = mtcars)
lasso <- linear_reg(
  mixture = 1,
  penalty = !!tune()
) %>% 
  set_engine(
    engine = 'glmnet',
    family = gaussian(link = 'log')
  )
wflow <- workflow(recipe, lasso)
grid <- grid_max_entropy(penalty(), size = 10)
folds <- vfold_cv(mtcars)

res <- tune_grid(
  object = wflow,
  resamples = folds,
  grid = grid
)
#> x Fold01: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> x Fold02: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> x Fold03: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> x Fold04: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> x Fold05: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> x Fold06: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> x Fold07: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> x Fold08: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> x Fold09: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> x Fold10: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> Warning: All models failed. See the `.notes` column.

res %>% collect_notes %>% distinct(note) %>% reduce(c) %>% cli::cli_ul()
#> • Error in `parsnip::multi_predict()`: ! No `multi_predict` method exists for
#> objects with classes '_glmnetfit', 'model_fit'

Created on 2022-05-31 by the reprex package (v2.0.1)
@EmilHvitfeldt
Copy link
Member

EmilHvitfeldt commented May 31, 2022 •
edited
Loading

Hello @cb12991! I'm not able to reproduce the error using CRAN versions or dev versions of the core tidymodels packages, Can you include the session information in your reprex using sessioninfo::session_info()?

CRAN reprex

library(tidyverse)
library(tidymodels)

recipe <- recipe(hp ~ ., data = mtcars)
lasso <- linear_reg(
  mixture = 1,
  penalty = !!tune()
) %>% 
  set_engine(
    engine = 'glmnet'
  )
wflow <- workflow(recipe, lasso)
grid <- grid_max_entropy(penalty(), size = 10)
folds <- vfold_cv(mtcars)

res <- tune_grid(
  object = wflow,
  resamples = folds,
  grid = grid
)

res
#> # Tuning results
#> # 10-fold cross-validation 
#> # A tibble: 10 × 4
#>    splits         id     .metrics          .notes          
#>    <list>         <chr>  <list>            <list>          
#>  1 <split [28/4]> Fold01 <tibble [20 × 5]> <tibble [0 × 3]>
#>  2 <split [28/4]> Fold02 <tibble [20 × 5]> <tibble [0 × 3]>
#>  3 <split [29/3]> Fold03 <tibble [20 × 5]> <tibble [0 × 3]>
#>  4 <split [29/3]> Fold04 <tibble [20 × 5]> <tibble [0 × 3]>
#>  5 <split [29/3]> Fold05 <tibble [20 × 5]> <tibble [0 × 3]>
#>  6 <split [29/3]> Fold06 <tibble [20 × 5]> <tibble [0 × 3]>
#>  7 <split [29/3]> Fold07 <tibble [20 × 5]> <tibble [0 × 3]>
#>  8 <split [29/3]> Fold08 <tibble [20 × 5]> <tibble [0 × 3]>
#>  9 <split [29/3]> Fold09 <tibble [20 × 5]> <tibble [0 × 3]>
#> 10 <split [29/3]> Fold10 <tibble [20 × 5]> <tibble [0 × 3]>

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.2.0 (2022-04-22)
#>  os       macOS Monterey 12.2.1
#>  system   aarch64, darwin20
#>  ui       X11
#>  language (EN)
#>  collate  en_US.UTF-8
#>  ctype    en_US.UTF-8
#>  tz       America/Los_Angeles
#>  date     2022-05-31
#>  pandoc   2.17.1.1 @ /Applications/RStudio.app/Contents/MacOS/quarto/bin/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package      * version    date (UTC) lib source
#>  assertthat     0.2.1      2019-03-21 [1] CRAN (R 4.2.0)
#>  backports      1.4.1      2021-12-13 [1] CRAN (R 4.2.0)
#>  broom        * 0.8.0      2022-04-13 [1] CRAN (R 4.2.0)
#>  cellranger     1.1.0      2016-07-27 [1] CRAN (R 4.2.0)
#>  class          7.3-20     2022-01-16 [1] CRAN (R 4.2.0)
#>  cli            3.3.0      2022-04-25 [1] CRAN (R 4.2.0)
#>  codetools      0.2-18     2020-11-04 [1] CRAN (R 4.2.0)
#>  colorspace     2.0-3      2022-02-21 [1] CRAN (R 4.2.0)
#>  crayon         1.5.1      2022-03-26 [1] CRAN (R 4.2.0)
#>  DBI            1.1.2      2021-12-20 [1] CRAN (R 4.2.0)
#>  dbplyr         2.1.1      2021-04-06 [1] CRAN (R 4.2.0)
#>  dials        * 0.1.1      2022-04-06 [1] CRAN (R 4.2.0)
#>  DiceDesign     1.9        2021-02-13 [1] CRAN (R 4.2.0)
#>  digest         0.6.29     2021-12-01 [1] CRAN (R 4.2.0)
#>  dplyr        * 1.0.9      2022-04-28 [1] CRAN (R 4.2.0)
#>  ellipsis       0.3.2      2021-04-29 [1] CRAN (R 4.2.0)
#>  evaluate       0.15       2022-02-18 [1] CRAN (R 4.2.0)
#>  fansi          1.0.3      2022-03-24 [1] CRAN (R 4.2.0)
#>  fastmap        1.1.0      2021-01-25 [1] CRAN (R 4.2.0)
#>  forcats      * 0.5.1      2021-01-27 [1] CRAN (R 4.2.0)
#>  foreach        1.5.2      2022-02-02 [1] CRAN (R 4.2.0)
#>  fs             1.5.2      2021-12-08 [1] CRAN (R 4.2.0)
#>  furrr          0.3.0      2022-05-04 [1] CRAN (R 4.2.0)
#>  future         1.25.0     2022-04-24 [1] CRAN (R 4.2.0)
#>  future.apply   1.9.0      2022-04-25 [1] CRAN (R 4.2.0)
#>  generics       0.1.2      2022-01-31 [1] CRAN (R 4.2.0)
#>  ggplot2      * 3.3.6      2022-05-03 [1] CRAN (R 4.2.0)
#>  glmnet       * 4.1-4      2022-04-15 [1] CRAN (R 4.2.0)
#>  globals        0.15.0     2022-05-09 [1] CRAN (R 4.2.0)
#>  glue           1.6.2      2022-02-24 [1] CRAN (R 4.2.0)
#>  gower          1.0.0      2022-02-03 [1] CRAN (R 4.2.0)
#>  GPfit          1.0-8      2019-02-08 [1] CRAN (R 4.2.0)
#>  gtable         0.3.0      2019-03-25 [1] CRAN (R 4.2.0)
#>  hardhat        0.2.0      2022-01-24 [1] CRAN (R 4.2.0)
#>  haven          2.5.0      2022-04-15 [1] CRAN (R 4.2.0)
#>  highr          0.9        2021-04-16 [1] CRAN (R 4.2.0)
#>  hms            1.1.1      2021-09-26 [1] CRAN (R 4.2.0)
#>  htmltools      0.5.2      2021-08-25 [1] CRAN (R 4.2.0)
#>  httr           1.4.3      2022-05-04 [1] CRAN (R 4.2.0)
#>  infer        * 1.0.0      2021-08-13 [1] CRAN (R 4.2.0)
#>  ipred          0.9-12     2021-09-15 [1] CRAN (R 4.2.0)
#>  iterators      1.0.14     2022-02-05 [1] CRAN (R 4.2.0)
#>  jsonlite       1.8.0      2022-02-22 [1] CRAN (R 4.2.0)
#>  knitr          1.39       2022-04-26 [1] CRAN (R 4.2.0)
#>  lattice        0.20-45    2021-09-22 [1] CRAN (R 4.2.0)
#>  lava           1.6.10     2021-09-02 [1] CRAN (R 4.2.0)
#>  lhs            1.1.5      2022-03-22 [1] CRAN (R 4.2.0)
#>  lifecycle      1.0.1      2021-09-24 [1] CRAN (R 4.2.0)
#>  listenv        0.8.0      2019-12-05 [1] CRAN (R 4.2.0)
#>  lubridate      1.8.0      2021-10-07 [1] CRAN (R 4.2.0)
#>  magrittr       2.0.3      2022-03-30 [1] CRAN (R 4.2.0)
#>  MASS           7.3-56     2022-03-23 [1] CRAN (R 4.2.0)
#>  Matrix       * 1.4-1      2022-03-23 [1] CRAN (R 4.2.0)
#>  modeldata    * 0.1.1      2021-07-14 [1] CRAN (R 4.2.0)
#>  modelr         0.1.8      2020-05-19 [1] CRAN (R 4.2.0)
#>  munsell        0.5.0      2018-06-12 [1] CRAN (R 4.2.0)
#>  nnet           7.3-17     2022-01-16 [1] CRAN (R 4.2.0)
#>  parallelly     1.31.1     2022-04-22 [1] CRAN (R 4.2.0)
#>  parsnip      * 0.2.1      2022-03-17 [1] CRAN (R 4.2.0)
#>  pillar         1.7.0      2022-02-01 [1] CRAN (R 4.2.0)
#>  pkgconfig      2.0.3      2019-09-22 [1] CRAN (R 4.2.0)
#>  plyr           1.8.7      2022-03-24 [1] CRAN (R 4.2.0)
#>  pROC           1.18.0     2021-09-03 [1] CRAN (R 4.2.0)
#>  prodlim        2019.11.13 2019-11-17 [1] CRAN (R 4.2.0)
#>  purrr        * 0.3.4      2020-04-17 [1] CRAN (R 4.2.0)
#>  R.cache        0.15.0     2021-04-30 [1] CRAN (R 4.2.0)
#>  R.methodsS3    1.8.1      2020-08-26 [1] CRAN (R 4.2.0)
#>  R.oo           1.24.0     2020-08-26 [1] CRAN (R 4.2.0)
#>  R.utils        2.11.0     2021-09-26 [1] CRAN (R 4.2.0)
#>  R6             2.5.1      2021-08-19 [1] CRAN (R 4.2.0)
#>  Rcpp           1.0.8.3    2022-03-17 [1] CRAN (R 4.2.0)
#>  readr        * 2.1.2      2022-01-30 [1] CRAN (R 4.2.0)
#>  readxl         1.4.0      2022-03-28 [1] CRAN (R 4.2.0)
#>  recipes      * 0.2.0      2022-02-18 [1] CRAN (R 4.2.0)
#>  reprex         2.0.1      2021-08-05 [1] CRAN (R 4.2.0)
#>  rlang          1.0.2      2022-03-04 [1] CRAN (R 4.2.0)
#>  rmarkdown      2.14       2022-04-25 [1] CRAN (R 4.2.0)
#>  rpart          4.1.16     2022-01-24 [1] CRAN (R 4.2.0)
#>  rsample      * 0.1.1      2021-11-08 [1] CRAN (R 4.2.0)
#>  rstudioapi     0.13       2020-11-12 [1] CRAN (R 4.2.0)
#>  rvest          1.0.2      2021-10-16 [1] CRAN (R 4.2.0)
#>  scales       * 1.2.0      2022-04-13 [1] CRAN (R 4.2.0)
#>  sessioninfo    1.2.2      2021-12-06 [1] CRAN (R 4.2.0)
#>  shape          1.4.6      2021-05-19 [1] CRAN (R 4.2.0)
#>  stringi        1.7.6      2021-11-29 [1] CRAN (R 4.2.0)
#>  stringr      * 1.4.0      2019-02-10 [1] CRAN (R 4.2.0)
#>  styler         1.7.0      2022-03-13 [1] CRAN (R 4.2.0)
#>  survival       3.3-1      2022-03-03 [1] CRAN (R 4.2.0)
#>  tibble       * 3.1.7      2022-05-03 [1] CRAN (R 4.2.0)
#>  tidymodels   * 0.2.0      2022-03-19 [1] CRAN (R 4.2.0)
#>  tidyr        * 1.2.0      2022-02-01 [1] CRAN (R 4.2.0)
#>  tidyselect     1.1.2      2022-02-21 [1] CRAN (R 4.2.0)
#>  tidyverse    * 1.3.1      2021-04-15 [1] CRAN (R 4.2.0)
#>  timeDate       3043.102   2018-02-21 [1] CRAN (R 4.2.0)
#>  tune         * 0.2.0      2022-03-19 [1] CRAN (R 4.2.0)
#>  tzdb           0.3.0      2022-03-28 [1] CRAN (R 4.2.0)
#>  utf8           1.2.2      2021-07-24 [1] CRAN (R 4.2.0)
#>  vctrs          0.4.1      2022-04-13 [1] CRAN (R 4.2.0)
#>  withr          2.5.0      2022-03-03 [1] CRAN (R 4.2.0)
#>  workflows    * 0.2.6      2022-03-18 [1] CRAN (R 4.2.0)
#>  workflowsets * 0.2.1      2022-03-15 [1] CRAN (R 4.2.0)
#>  xfun           0.31       2022-05-10 [1] CRAN (R 4.2.0)
#>  xml2           1.3.3      2021-11-30 [1] CRAN (R 4.2.0)
#>  yaml           2.3.5      2022-02-21 [1] CRAN (R 4.2.0)
#>  yardstick    * 0.0.9      2021-11-22 [1] CRAN (R 4.2.0)
#> 
#>  [1] /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

Dev regrex

library(tidyverse)
library(tidymodels)

recipe <- recipe(hp ~ ., data = mtcars)
lasso <- linear_reg(
  mixture = 1,
  penalty = !!tune()
) %>% 
  set_engine(
    engine = 'glmnet'
  )
wflow <- workflow(recipe, lasso)
grid <- grid_max_entropy(penalty(), size = 10)
folds <- vfold_cv(mtcars)

res <- tune_grid(
  object = wflow,
  resamples = folds,
  grid = grid
)

res
#> # Tuning results
#> # 10-fold cross-validation 
#> # A tibble: 10 × 4
#>    splits         id     .metrics          .notes          
#>    <list>         <chr>  <list>            <list>          
#>  1 <split [28/4]> Fold01 <tibble [20 × 5]> <tibble [0 × 3]>
#>  2 <split [28/4]> Fold02 <tibble [20 × 5]> <tibble [0 × 3]>
#>  3 <split [29/3]> Fold03 <tibble [20 × 5]> <tibble [0 × 3]>
#>  4 <split [29/3]> Fold04 <tibble [20 × 5]> <tibble [0 × 3]>
#>  5 <split [29/3]> Fold05 <tibble [20 × 5]> <tibble [0 × 3]>
#>  6 <split [29/3]> Fold06 <tibble [20 × 5]> <tibble [0 × 3]>
#>  7 <split [29/3]> Fold07 <tibble [20 × 5]> <tibble [0 × 3]>
#>  8 <split [29/3]> Fold08 <tibble [20 × 5]> <tibble [0 × 3]>
#>  9 <split [29/3]> Fold09 <tibble [20 × 5]> <tibble [0 × 3]>
#> 10 <split [29/3]> Fold10 <tibble [20 × 5]> <tibble [0 × 3]>

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.2.0 (2022-04-22)
#>  os       macOS Monterey 12.2.1
#>  system   aarch64, darwin20
#>  ui       X11
#>  language (EN)
#>  collate  en_US.UTF-8
#>  ctype    en_US.UTF-8
#>  tz       America/Los_Angeles
#>  date     2022-05-31
#>  pandoc   2.17.1.1 @ /Applications/RStudio.app/Contents/MacOS/quarto/bin/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package      * version    date (UTC) lib source
#>  assertthat     0.2.1      2019-03-21 [1] CRAN (R 4.2.0)
#>  backports      1.4.1      2021-12-13 [1] CRAN (R 4.2.0)
#>  broom        * 0.8.0      2022-04-13 [1] CRAN (R 4.2.0)
#>  cellranger     1.1.0      2016-07-27 [1] CRAN (R 4.2.0)
#>  class          7.3-20     2022-01-16 [1] CRAN (R 4.2.0)
#>  cli            3.3.0      2022-04-25 [1] CRAN (R 4.2.0)
#>  codetools      0.2-18     2020-11-04 [1] CRAN (R 4.2.0)
#>  colorspace     2.0-3      2022-02-21 [1] CRAN (R 4.2.0)
#>  crayon         1.5.1      2022-03-26 [1] CRAN (R 4.2.0)
#>  DBI            1.1.2      2021-12-20 [1] CRAN (R 4.2.0)
#>  dbplyr         2.1.1      2021-04-06 [1] CRAN (R 4.2.0)
#>  dials        * 0.1.1      2022-04-06 [1] CRAN (R 4.2.0)
#>  DiceDesign     1.9        2021-02-13 [1] CRAN (R 4.2.0)
#>  digest         0.6.29     2021-12-01 [1] CRAN (R 4.2.0)
#>  dplyr        * 1.0.9      2022-04-28 [1] CRAN (R 4.2.0)
#>  ellipsis       0.3.2      2021-04-29 [1] CRAN (R 4.2.0)
#>  evaluate       0.15       2022-02-18 [1] CRAN (R 4.2.0)
#>  fansi          1.0.3      2022-03-24 [1] CRAN (R 4.2.0)
#>  fastmap        1.1.0      2021-01-25 [1] CRAN (R 4.2.0)
#>  forcats      * 0.5.1      2021-01-27 [1] CRAN (R 4.2.0)
#>  foreach        1.5.2      2022-02-02 [1] CRAN (R 4.2.0)
#>  fs             1.5.2      2021-12-08 [1] CRAN (R 4.2.0)
#>  furrr          0.3.0      2022-05-04 [1] CRAN (R 4.2.0)
#>  future         1.25.0     2022-04-24 [1] CRAN (R 4.2.0)
#>  future.apply   1.9.0      2022-04-25 [1] CRAN (R 4.2.0)
#>  generics       0.1.2      2022-01-31 [1] CRAN (R 4.2.0)
#>  ggplot2      * 3.3.6      2022-05-03 [1] CRAN (R 4.2.0)
#>  glmnet       * 4.1-4      2022-04-15 [1] CRAN (R 4.2.0)
#>  globals        0.15.0     2022-05-09 [1] CRAN (R 4.2.0)
#>  glue           1.6.2      2022-02-24 [1] CRAN (R 4.2.0)
#>  gower          1.0.0      2022-02-03 [1] CRAN (R 4.2.0)
#>  GPfit          1.0-8      2019-02-08 [1] CRAN (R 4.2.0)
#>  gtable         0.3.0      2019-03-25 [1] CRAN (R 4.2.0)
#>  hardhat        0.2.0.9000 2022-05-31 [1] Github (tidymodels/hardhat@2074ccb)
#>  haven          2.5.0      2022-04-15 [1] CRAN (R 4.2.0)
#>  highr          0.9        2021-04-16 [1] CRAN (R 4.2.0)
#>  hms            1.1.1      2021-09-26 [1] CRAN (R 4.2.0)
#>  htmltools      0.5.2      2021-08-25 [1] CRAN (R 4.2.0)
#>  httr           1.4.3      2022-05-04 [1] CRAN (R 4.2.0)
#>  infer        * 1.0.0      2021-08-13 [1] CRAN (R 4.2.0)
#>  ipred          0.9-12     2021-09-15 [1] CRAN (R 4.2.0)
#>  iterators      1.0.14     2022-02-05 [1] CRAN (R 4.2.0)
#>  jsonlite       1.8.0      2022-02-22 [1] CRAN (R 4.2.0)
#>  knitr          1.39       2022-04-26 [1] CRAN (R 4.2.0)
#>  lattice        0.20-45    2021-09-22 [1] CRAN (R 4.2.0)
#>  lava           1.6.10     2021-09-02 [1] CRAN (R 4.2.0)
#>  lhs            1.1.5      2022-03-22 [1] CRAN (R 4.2.0)
#>  lifecycle      1.0.1      2021-09-24 [1] CRAN (R 4.2.0)
#>  listenv        0.8.0      2019-12-05 [1] CRAN (R 4.2.0)
#>  lubridate      1.8.0      2021-10-07 [1] CRAN (R 4.2.0)
#>  magrittr       2.0.3      2022-03-30 [1] CRAN (R 4.2.0)
#>  MASS           7.3-56     2022-03-23 [1] CRAN (R 4.2.0)
#>  Matrix       * 1.4-1      2022-03-23 [1] CRAN (R 4.2.0)
#>  modeldata    * 0.1.1      2021-07-14 [1] CRAN (R 4.2.0)
#>  modelr         0.1.8      2020-05-19 [1] CRAN (R 4.2.0)
#>  munsell        0.5.0      2018-06-12 [1] CRAN (R 4.2.0)
#>  nnet           7.3-17     2022-01-16 [1] CRAN (R 4.2.0)
#>  parallelly     1.31.1     2022-04-22 [1] CRAN (R 4.2.0)
#>  parsnip      * 0.2.1.9002 2022-05-31 [1] Github (tidymodels/parsnip@f2f24a0)
#>  pillar         1.7.0      2022-02-01 [1] CRAN (R 4.2.0)
#>  pkgconfig      2.0.3      2019-09-22 [1] CRAN (R 4.2.0)
#>  prodlim        2019.11.13 2019-11-17 [1] CRAN (R 4.2.0)
#>  purrr        * 0.3.4      2020-04-17 [1] CRAN (R 4.2.0)
#>  R.cache        0.15.0     2021-04-30 [1] CRAN (R 4.2.0)
#>  R.methodsS3    1.8.1      2020-08-26 [1] CRAN (R 4.2.0)
#>  R.oo           1.24.0     2020-08-26 [1] CRAN (R 4.2.0)
#>  R.utils        2.11.0     2021-09-26 [1] CRAN (R 4.2.0)
#>  R6             2.5.1      2021-08-19 [1] CRAN (R 4.2.0)
#>  Rcpp           1.0.8.3    2022-03-17 [1] CRAN (R 4.2.0)
#>  readr        * 2.1.2      2022-01-30 [1] CRAN (R 4.2.0)
#>  readxl         1.4.0      2022-03-28 [1] CRAN (R 4.2.0)
#>  recipes      * 0.2.0      2022-02-18 [1] CRAN (R 4.2.0)
#>  reprex         2.0.1      2021-08-05 [1] CRAN (R 4.2.0)
#>  rlang          1.0.2      2022-03-04 [1] CRAN (R 4.2.0)
#>  rmarkdown      2.14       2022-04-25 [1] CRAN (R 4.2.0)
#>  rpart          4.1.16     2022-01-24 [1] CRAN (R 4.2.0)
#>  rsample      * 0.1.1      2021-11-08 [1] CRAN (R 4.2.0)
#>  rstudioapi     0.13       2020-11-12 [1] CRAN (R 4.2.0)
#>  rvest          1.0.2      2021-10-16 [1] CRAN (R 4.2.0)
#>  scales       * 1.2.0      2022-04-13 [1] CRAN (R 4.2.0)
#>  sessioninfo    1.2.2      2021-12-06 [1] CRAN (R 4.2.0)
#>  shape          1.4.6      2021-05-19 [1] CRAN (R 4.2.0)
#>  stringi        1.7.6      2021-11-29 [1] CRAN (R 4.2.0)
#>  stringr      * 1.4.0      2019-02-10 [1] CRAN (R 4.2.0)
#>  styler         1.7.0      2022-03-13 [1] CRAN (R 4.2.0)
#>  survival       3.3-1      2022-03-03 [1] CRAN (R 4.2.0)
#>  tibble       * 3.1.7      2022-05-03 [1] CRAN (R 4.2.0)
#>  tidymodels   * 0.2.0      2022-03-19 [1] CRAN (R 4.2.0)
#>  tidyr        * 1.2.0      2022-02-01 [1] CRAN (R 4.2.0)
#>  tidyselect     1.1.2      2022-02-21 [1] CRAN (R 4.2.0)
#>  tidyverse    * 1.3.1      2021-04-15 [1] CRAN (R 4.2.0)
#>  timeDate       3043.102   2018-02-21 [1] CRAN (R 4.2.0)
#>  tune         * 0.2.0.9002 2022-05-31 [1] Github (tidymodels/tune@08d6ae3)
#>  tzdb           0.3.0      2022-03-28 [1] CRAN (R 4.2.0)
#>  utf8           1.2.2      2021-07-24 [1] CRAN (R 4.2.0)
#>  vctrs          0.4.1      2022-04-13 [1] CRAN (R 4.2.0)
#>  withr          2.5.0      2022-03-03 [1] CRAN (R 4.2.0)
#>  workflows    * 0.2.6.9001 2022-05-31 [1] Github (tidymodels/workflows@9a9e231)
#>  workflowsets * 0.2.1      2022-03-15 [1] CRAN (R 4.2.0)
#>  xfun           0.31       2022-05-10 [1] CRAN (R 4.2.0)
#>  xml2           1.3.3      2021-11-30 [1] CRAN (R 4.2.0)
#>  yaml           2.3.5      2022-02-21 [1] CRAN (R 4.2.0)
#>  yardstick    * 0.0.9.9000 2022-05-31 [1] Github (tidymodels/yardstick@e56b69f)
#> 
#>  [1] /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

Created on 2022-05-31 by the reprex package (v2.0.1)

@cb12991
Copy link
Author

cb12991 commented May 31, 2022

Sure, but it doesn't look like you changed the family argument from the default "gaussian". Using the default does not generate the error for me either.

Updated Reprex

library(tidyverse)
library(tidymodels)

grid <- grid_max_entropy(penalty(), size = 10)
folds <- vfold_cv(mtcars)
recipe <- recipe(hp ~ ., data = mtcars)

lasso <- linear_reg(
  mixture = 1,
  penalty = !!tune()
) %>% 
  set_engine(
    engine = 'glmnet'
  )
wflow <- workflow(recipe, lasso)

res <- tune_grid(
  object = wflow,
  resamples = folds,
  grid = grid
)

res %>% collect_notes %>% distinct(note) %>% reduce(c) %>% cli::cli_ul()

lasso_loglink <- linear_reg(
  mixture = 1,
  penalty = !!tune()
) %>% 
  set_engine(
    engine = 'glmnet',
    family = gaussian(link = 'log')
  )
wflow_loglink <- wflow %>% update_model(lasso_loglink)

res_loglink <- tune_grid(
  object = wflow_loglink,
  resamples = folds,
  grid = grid
)
#> x Fold01: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> x Fold02: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> x Fold03: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> x Fold04: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> x Fold05: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> x Fold06: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> x Fold07: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> x Fold08: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> x Fold09: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> x Fold10: preprocessor 1/1, model 1/1 (predictions): Error in `parsnip::multi_pred...
#> Warning: All models failed. See the `.notes` column.

res_loglink %>% collect_notes %>% distinct(note) %>% reduce(c) %>% cli::cli_ul()
#>   • Error in `parsnip::multi_predict()`: ! No `multi_predict` method exists for
#>   objects with classes '_glmnetfit', 'model_fit'

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.2.0 (2022-04-22 ucrt)
#>  os       Windows 10 x64 (build 19042)
#>  system   x86_64, mingw32
#>  ui       RTerm
#>  language (EN)
#>  collate  English_United States.utf8
#>  ctype    English_United States.utf8
#>  tz       America/Chicago
#>  date     2022-05-31
#>  pandoc   2.17.1.1 @ C:/Users/grc3b/RStudio/bin/quarto/bin/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package      * version    date (UTC) lib source
#>  assertthat     0.2.1      2019-03-21 [1] CRAN (R 4.2.0)
#>  backports      1.4.1      2021-12-13 [1] CRAN (R 4.2.0)
#>  broom        * 0.8.0      2022-04-13 [1] CRAN (R 4.2.0)
#>  cellranger     1.1.0      2016-07-27 [1] CRAN (R 4.2.0)
#>  class          7.3-20     2022-01-16 [1] CRAN (R 4.2.0)
#>  cli            3.3.0      2022-04-25 [1] CRAN (R 4.2.0)
#>  codetools      0.2-18     2020-11-04 [1] CRAN (R 4.2.0)
#>  colorspace     2.0-3      2022-02-21 [1] CRAN (R 4.2.0)
#>  crayon         1.5.1      2022-03-26 [1] CRAN (R 4.2.0)
#>  DBI            1.1.2      2021-12-20 [1] CRAN (R 4.2.0)
#>  dbplyr         2.1.1      2021-04-06 [1] CRAN (R 4.2.0)
#>  dials        * 0.1.1      2022-04-06 [1] CRAN (R 4.2.0)
#>  DiceDesign     1.9        2021-02-13 [1] CRAN (R 4.2.0)
#>  digest         0.6.29     2021-12-01 [1] CRAN (R 4.2.0)
#>  dplyr        * 1.0.9      2022-04-28 [1] CRAN (R 4.2.0)
#>  ellipsis       0.3.2      2021-04-29 [1] CRAN (R 4.2.0)
#>  evaluate       0.15       2022-02-18 [1] CRAN (R 4.2.0)
#>  fansi          1.0.3      2022-03-24 [1] CRAN (R 4.2.0)
#>  fastmap        1.1.0      2021-01-25 [1] CRAN (R 4.2.0)
#>  forcats      * 0.5.1      2021-01-27 [1] CRAN (R 4.2.0)
#>  foreach        1.5.2      2022-02-02 [1] CRAN (R 4.2.0)
#>  fs             1.5.2      2021-12-08 [1] CRAN (R 4.2.0)
#>  furrr          0.3.0      2022-05-04 [1] CRAN (R 4.2.0)
#>  future         1.25.0     2022-04-24 [1] CRAN (R 4.2.0)
#>  future.apply   1.9.0      2022-04-25 [1] CRAN (R 4.2.0)
#>  generics       0.1.2      2022-01-31 [1] CRAN (R 4.2.0)
#>  ggplot2      * 3.3.6      2022-05-03 [1] CRAN (R 4.2.0)
#>  glmnet       * 4.1-4      2022-04-15 [1] CRAN (R 4.2.0)
#>  globals        0.15.0     2022-05-09 [1] CRAN (R 4.2.0)
#>  glue           1.6.2      2022-02-24 [1] CRAN (R 4.2.0)
#>  gower          1.0.0      2022-02-03 [1] CRAN (R 4.2.0)
#>  GPfit          1.0-8      2019-02-08 [1] CRAN (R 4.2.0)
#>  gtable         0.3.0      2019-03-25 [1] CRAN (R 4.2.0)
#>  hardhat        0.2.0.9000 2022-05-20 [1] Github (tidymodels/hardhat@66d8f7f)
#>  haven          2.5.0      2022-04-15 [1] CRAN (R 4.2.0)
#>  highr          0.9        2021-04-16 [1] CRAN (R 4.2.0)
#>  hms            1.1.1      2021-09-26 [1] CRAN (R 4.2.0)
#>  htmltools      0.5.2      2021-08-25 [1] CRAN (R 4.2.0)
#>  httr           1.4.3      2022-05-04 [1] CRAN (R 4.2.0)
#>  infer        * 1.0.0      2021-08-13 [1] CRAN (R 4.2.0)
#>  ipred          0.9-12     2021-09-15 [1] CRAN (R 4.2.0)
#>  iterators      1.0.14     2022-02-05 [1] CRAN (R 4.2.0)
#>  jsonlite       1.8.0      2022-02-22 [1] CRAN (R 4.2.0)
#>  knitr          1.39       2022-04-26 [1] CRAN (R 4.2.0)
#>  lattice        0.20-45    2021-09-22 [1] CRAN (R 4.2.0)
#>  lava           1.6.10     2021-09-02 [1] CRAN (R 4.2.0)
#>  lhs            1.1.5      2022-03-22 [1] CRAN (R 4.2.0)
#>  lifecycle      1.0.1      2021-09-24 [1] CRAN (R 4.2.0)
#>  listenv        0.8.0      2019-12-05 [1] CRAN (R 4.2.0)
#>  lubridate      1.8.0      2021-10-07 [1] CRAN (R 4.2.0)
#>  magrittr       2.0.3      2022-03-30 [1] CRAN (R 4.2.0)
#>  MASS           7.3-57     2022-04-22 [1] CRAN (R 4.2.0)
#>  Matrix       * 1.4-1      2022-03-23 [1] CRAN (R 4.2.0)
#>  modeldata    * 0.1.1.9000 2022-05-20 [1] Github (tidymodels/modeldata@82e3320)
#>  modelr         0.1.8      2020-05-19 [1] CRAN (R 4.2.0)
#>  munsell        0.5.0      2018-06-12 [1] CRAN (R 4.2.0)
#>  nnet           7.3-17     2022-01-16 [1] CRAN (R 4.2.0)
#>  parallelly     1.31.1     2022-04-22 [1] CRAN (R 4.2.0)
#>  parsnip      * 0.2.1.9002 2022-05-20 [1] Github (tidymodels/parsnip@f2f24a0)
#>  pillar         1.7.0      2022-02-01 [1] CRAN (R 4.2.0)
#>  pkgconfig      2.0.3      2019-09-22 [1] CRAN (R 4.2.0)
#>  prodlim        2019.11.13 2019-11-17 [1] CRAN (R 4.2.0)
#>  purrr        * 0.3.4      2020-04-17 [1] CRAN (R 4.2.0)
#>  R.cache        0.15.0     2021-04-30 [1] CRAN (R 4.2.0)
#>  R.methodsS3    1.8.1      2020-08-26 [1] CRAN (R 4.2.0)
#>  R.oo           1.24.0     2020-08-26 [1] CRAN (R 4.2.0)
#>  R.utils        2.11.0     2021-09-26 [1] CRAN (R 4.2.0)
#>  R6             2.5.1      2021-08-19 [1] CRAN (R 4.2.0)
#>  Rcpp           1.0.8.3    2022-03-17 [1] CRAN (R 4.2.0)
#>  readr        * 2.1.2      2022-01-30 [1] CRAN (R 4.2.0)
#>  readxl         1.4.0      2022-03-28 [1] CRAN (R 4.2.0)
#>  recipes      * 0.2.0.9001 2022-05-20 [1] Github (tidymodels/recipes@fbda193)
#>  reprex         2.0.1      2021-08-05 [1] CRAN (R 4.2.0)
#>  rlang          1.0.2      2022-03-04 [1] CRAN (R 4.2.0)
#>  rmarkdown      2.14       2022-04-25 [1] CRAN (R 4.2.0)
#>  rpart          4.1.16     2022-01-24 [1] CRAN (R 4.2.0)
#>  rsample      * 0.1.1      2021-11-08 [1] CRAN (R 4.2.0)
#>  rstudioapi     0.13       2020-11-12 [1] CRAN (R 4.2.0)
#>  rvest          1.0.2      2021-10-16 [1] CRAN (R 4.2.0)
#>  scales       * 1.2.0      2022-04-13 [1] CRAN (R 4.2.0)
#>  sessioninfo    1.2.2      2021-12-06 [1] CRAN (R 4.2.0)
#>  shape          1.4.6      2021-05-19 [1] CRAN (R 4.2.0)
#>  stringi        1.7.6      2021-11-29 [1] CRAN (R 4.2.0)
#>  stringr      * 1.4.0      2019-02-10 [1] CRAN (R 4.2.0)
#>  styler         1.7.0      2022-03-13 [1] CRAN (R 4.2.0)
#>  survival       3.3-1      2022-03-03 [1] CRAN (R 4.2.0)
#>  tibble       * 3.1.7      2022-05-03 [1] CRAN (R 4.2.0)
#>  tidymodels   * 0.2.0      2022-03-19 [1] CRAN (R 4.2.0)
#>  tidyr        * 1.2.0      2022-02-01 [1] CRAN (R 4.2.0)
#>  tidyselect     1.1.2      2022-02-21 [1] CRAN (R 4.2.0)
#>  tidyverse    * 1.3.1      2021-04-15 [1] CRAN (R 4.2.0)
#>  timeDate       3043.102   2018-02-21 [1] CRAN (R 4.2.0)
#>  tune         * 0.2.0.9001 2022-05-20 [1] Github (tidymodels/tune@1856421)
#>  tzdb           0.3.0      2022-03-28 [1] CRAN (R 4.2.0)
#>  utf8           1.2.2      2021-07-24 [1] CRAN (R 4.2.0)
#>  vctrs          0.4.1      2022-04-13 [1] CRAN (R 4.2.0)
#>  withr          2.5.0      2022-03-03 [1] CRAN (R 4.2.0)
#>  workflows    * 0.2.6.9001 2022-05-20 [1] Github (tidymodels/workflows@9a9e231)
#>  workflowsets * 0.2.1      2022-03-15 [1] CRAN (R 4.2.0)
#>  xfun           0.31       2022-05-10 [1] CRAN (R 4.2.0)
#>  xml2           1.3.3      2021-11-30 [1] CRAN (R 4.2.0)
#>  yaml           2.3.5      2022-02-21 [1] CRAN (R 4.2.0)
#>  yardstick    * 0.0.9.9000 2022-05-20 [1] Github (tidymodels/yardstick@e56b69f)
#> 
#>  [1] C:/Users/grc3b/R-4.2.0/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

Created on 2022-05-31 by the reprex package (v2.0.1)

@EmilHvitfeldt
Copy link
Member

I got it! More minimal reprex below:

Much like how we have multi_predict._elnet() we should have mutli_predict. _glmnetfit()

library(parsnip)

lasso <- linear_reg(penalty = 1) %>% 
  set_engine(
    engine = 'glmnet',
    family = gaussian(link = 'log')
  )

fit(lasso, hp ~ ., data = mtcars) %>%
  multi_predict(new_data = mtcars)
#> Error in `multi_predict()`:
#> ! No `multi_predict` method exists for objects with classes '_glmnetfit', 'model_fit'

lasso <- linear_reg(penalty = 1) %>% 
  set_engine(
    engine = 'glmnet'
  )

fit(lasso, hp ~ ., data = mtcars) %>%
  multi_predict(new_data = mtcars)
#> # A tibble: 32 × 1
#>    .pred           
#>    <list>          
#>  1 <tibble [1 × 2]>
#>  2 <tibble [1 × 2]>
#>  3 <tibble [1 × 2]>
#>  4 <tibble [1 × 2]>
#>  5 <tibble [1 × 2]>
#>  6 <tibble [1 × 2]>
#>  7 <tibble [1 × 2]>
#>  8 <tibble [1 × 2]>
#>  9 <tibble [1 × 2]>
#> 10 <tibble [1 × 2]>
#> # … with 22 more rows

Created on 2022-05-31 by the reprex package (v2.0.1)

@EmilHvitfeldt EmilHvitfeldt added the bug an unexpected problem or unintended behavior label May 31, 2022
@topepo
Copy link
Member

topepo commented Jun 1, 2022

Yes. The object type going out of glmnet::glmnet() is not well documented:

An object with S3 class "glmnet","*" , where "*" is "elnet", "lognet", "multnet", "fishnet" (poisson), "coxnet" or "mrelnet" for the various types of models. If the model was created with relax=TRUE then this class has a prefix class of "relaxed".

It implies that there would always be a more specific class for us to work off of.

@frankhezemans
Copy link

I'm facing similar problems using the "mgaussian" family argument to glmnet.
Is a solution to this issue in the works, or should I use the glmnet package's own predict methods for the time being?

@hfrick
Copy link
Member

hfrick commented Jan 21, 2023

#483 is the reason for this

@frankhezemans this is on my todo list, and I'll take a look at the "mgaussian" option as well.

@hfrick
Copy link
Member

hfrick commented Feb 21, 2023

@frankhezemans regarding the "mgaussian" option, we could look into supporting this family as well but that would likely be restricted to model fitting via parsnip: parsnip can support a multivariate response but the packages for performance metrics and tuning currently don't have infrastructure for multivariate responses. If such a "parsnip only" solution would be helpful to you (without the tuning), please open a separate issue for this here on the parsnip repo. Thank you!

@frankhezemans
Copy link

Thank you @hfrick for following up on this. I am aware of the limited support for evaluation of models with multivariate response data. In the meantime, I have written some slightly hacky functions and scripts to serve the needs of my specific project. Thus, from my perspective, a "parsnip only" solution is not urgently needed. But thanks again for your support!

@hfrick hfrick mentioned this issue Feb 23, 2023
Merged
@github-actions
Copy link

This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.

@github-actions github-actions bot locked and limited conversation to collaborators Mar 17, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
bug an unexpected problem or unintended behavior
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support glmnet models with base-R families tidymodels/parsnip
5 participants
@topepo @hfrick @EmilHvitfeldt @frankhezemans @cb12991