Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tune_bayes() errors when tuning a recipe plus a finalized model #168

Closed
hplieninger opened this issue Feb 12, 2020 · 3 comments · Fixed by #169
Closed

tune_bayes() errors when tuning a recipe plus a finalized model #168

hplieninger opened this issue Feb 12, 2020 · 3 comments · Fixed by #169

Comments

@hplieninger
Copy link

The problem

I want to tune parameters of a recipe (e.g., step_poly(degree = tune())) and a model simultaneously.
This errors with tune_bayes(), when the model has parameters that need to be finalized (e.g., mtry) although I used update() or finalize() beforehand.

All other potential variations of the example below work as expected. For example, it works with tune_grid(), it works with min_n instead of mtry, and it works without the tuning parameter in the recipe.

Reproducible example

library("tidymodels")
#> -- Attaching packages -------------------------------------- tidymodels 0.0.3 --
#> v broom     0.5.4     v purrr     0.3.3
#> v dials     0.0.4     v recipes   0.1.9
#> v dplyr     0.8.4     v rsample   0.0.5
#> v ggplot2   3.2.1     v tibble    2.1.3
#> v infer     0.5.1     v yardstick 0.0.5
#> v parsnip   0.0.5
#> -- Conflicts ----------------------------------------- tidymodels_conflicts() --
#> x purrr::discard()    masks scales::discard()
#> x dplyr::filter()     masks stats::filter()
#> x dplyr::lag()        masks stats::lag()
#> x ggplot2::margin()   masks dials::margin()
#> x recipes::step()     masks stats::step()
#> x recipes::yj_trans() masks scales::yj_trans()
library("tune")
library("workflows")

set.seed(2341)

ex_rec <- recipe(mpg ~ ., data = mtcars) %>%
  step_poly(disp, degree = tune())

ex_vfold <- vfold_cv(mtcars, 3)

rf_model <-
  rand_forest("regression", mtry = tune()) %>%
  set_engine("ranger")

rf_wflow <-
  workflow() %>%
  add_model(rf_model) %>%
  add_recipe(ex_rec)

rf_param <-
  rf_wflow %>%
  parameters() %>%
  update(mtry = mtry(range = c(1L, 10L)),
         degree = degree_int(range = c(1L, 4L)))

rf_param
#> Collection of 2 parameters for tuning
#> 
#>      id parameter type object class
#>    mtry           mtry    nparam[+]
#>  degree         degree    nparam[+]

rf_search <- tune_bayes(rf_wflow, initial = 2,
                        resamples = ex_vfold,
                        param_info = rf_param,
                        control = control_bayes(no_improve = 2))
#> Error: Some tuning parameters require finalization but there are recipe parameters
#>  that require tuning. Please use `parameters()` to finalize the parameter ranges.

Created on 2020-02-12 by the reprex package (v0.3.0)

Session info
devtools::session_info()
#> - Session info ---------------------------------------------------------------
#>  setting  value                       
#>  version  R version 3.6.2 (2019-12-12)
#>  os       Windows 8.1 x64             
#>  system   x86_64, mingw32             
#>  ui       RTerm                       
#>  language (EN)                        
#>  collate  German_Germany.1252         
#>  ctype    German_Germany.1252         
#>  tz       Europe/Berlin               
#>  date     2020-02-12                  
#> 
#> - Packages -------------------------------------------------------------------
#>  package       * version    date       lib source                              
#>  assertthat      0.2.1      2019-03-21 [1] CRAN (R 3.6.0)                      
#>  backports       1.1.5      2019-10-02 [1] CRAN (R 3.6.1)                      
#>  base64enc       0.1-3      2015-07-28 [1] CRAN (R 3.6.0)                      
#>  bayesplot       1.7.1      2019-12-01 [1] CRAN (R 3.6.1)                      
#>  boot            1.3-24     2019-12-20 [1] CRAN (R 3.6.2)                      
#>  broom         * 0.5.4      2020-01-27 [1] CRAN (R 3.6.2)                      
#>  callr           3.4.1      2020-01-24 [1] CRAN (R 3.6.2)                      
#>  class           7.3-15     2019-01-01 [2] CRAN (R 3.6.2)                      
#>  cli             2.0.1      2020-01-08 [1] CRAN (R 3.6.2)                      
#>  codetools       0.2-16     2018-12-24 [2] CRAN (R 3.6.2)                      
#>  colorspace      1.4-1      2019-03-18 [1] CRAN (R 3.6.0)                      
#>  colourpicker    1.0        2017-09-27 [1] CRAN (R 3.6.0)                      
#>  crayon          1.3.4      2017-09-16 [1] CRAN (R 3.6.2)                      
#>  crosstalk       1.0.0      2016-12-21 [1] CRAN (R 3.6.0)                      
#>  desc            1.2.0      2018-05-01 [1] CRAN (R 3.6.0)                      
#>  devtools        2.2.1      2019-09-24 [1] CRAN (R 3.6.1)                      
#>  dials         * 0.0.4      2019-12-02 [1] CRAN (R 3.6.1)                      
#>  DiceDesign      1.8-1      2019-07-31 [1] CRAN (R 3.6.1)                      
#>  digest          0.6.23     2019-11-23 [1] CRAN (R 3.6.1)                      
#>  dplyr         * 0.8.4      2020-01-31 [1] CRAN (R 3.6.2)                      
#>  DT              0.11       2019-12-19 [1] CRAN (R 3.6.2)                      
#>  dygraphs        1.1.1.6    2018-07-11 [1] CRAN (R 3.6.0)                      
#>  ellipsis        0.3.0      2019-09-20 [1] CRAN (R 3.6.1)                      
#>  evaluate        0.14       2019-05-28 [1] CRAN (R 3.6.0)                      
#>  fansi           0.4.1      2020-01-08 [1] CRAN (R 3.6.2)                      
#>  fastmap         1.0.1      2019-10-08 [1] CRAN (R 3.6.1)                      
#>  foreach         1.4.7      2019-07-27 [1] CRAN (R 3.6.1)                      
#>  fs              1.3.1      2019-05-06 [1] CRAN (R 3.6.0)                      
#>  furrr           0.1.0      2018-05-16 [1] CRAN (R 3.6.0)                      
#>  future          1.16.0     2020-01-16 [1] CRAN (R 3.6.2)                      
#>  generics        0.0.2      2018-11-29 [1] CRAN (R 3.6.1)                      
#>  ggplot2       * 3.2.1      2019-08-10 [1] CRAN (R 3.6.2)                      
#>  ggridges        0.5.2      2020-01-12 [1] CRAN (R 3.6.2)                      
#>  globals         0.12.5     2019-12-07 [1] CRAN (R 3.6.1)                      
#>  glue            1.3.1      2019-03-12 [1] CRAN (R 3.6.0)                      
#>  gower           0.2.1      2019-05-14 [1] CRAN (R 3.6.0)                      
#>  GPfit           1.0-8      2019-02-08 [1] CRAN (R 3.6.1)                      
#>  gridExtra       2.3        2017-09-09 [1] CRAN (R 3.6.0)                      
#>  gtable          0.3.0      2019-03-25 [1] CRAN (R 3.6.0)                      
#>  gtools          3.8.1      2018-06-26 [1] CRAN (R 3.6.0)                      
#>  hardhat         0.1.1      2020-01-08 [1] CRAN (R 3.6.2)                      
#>  highr           0.8        2019-03-20 [1] CRAN (R 3.6.0)                      
#>  htmltools       0.4.0      2019-10-04 [1] CRAN (R 3.6.1)                      
#>  htmlwidgets     1.5.1      2019-10-08 [1] CRAN (R 3.6.1)                      
#>  httpuv          1.5.2      2019-09-11 [1] CRAN (R 3.6.1)                      
#>  igraph          1.2.4.2    2019-11-27 [1] CRAN (R 3.6.1)                      
#>  infer         * 0.5.1      2019-11-19 [1] CRAN (R 3.6.1)                      
#>  inline          0.3.15     2018-05-18 [1] CRAN (R 3.6.0)                      
#>  ipred           0.9-9      2019-04-28 [1] CRAN (R 3.6.0)                      
#>  iterators       1.0.12     2019-07-26 [1] CRAN (R 3.6.1)                      
#>  janeaustenr     0.1.5      2017-06-10 [1] CRAN (R 3.6.0)                      
#>  knitr           1.27       2020-01-16 [1] CRAN (R 3.6.2)                      
#>  later           1.0.0      2019-10-04 [1] CRAN (R 3.6.1)                      
#>  lattice         0.20-38    2018-11-04 [2] CRAN (R 3.6.2)                      
#>  lava            1.6.6      2019-08-01 [1] CRAN (R 3.6.1)                      
#>  lazyeval        0.2.2      2019-03-15 [1] CRAN (R 3.6.0)                      
#>  lhs             1.0.1      2019-02-03 [1] CRAN (R 3.6.1)                      
#>  lifecycle       0.1.0      2019-08-01 [1] CRAN (R 3.6.1)                      
#>  listenv         0.8.0      2019-12-05 [1] CRAN (R 3.6.1)                      
#>  lme4            1.1-21     2019-03-05 [1] CRAN (R 3.6.0)                      
#>  loo             2.2.0      2019-12-19 [1] CRAN (R 3.6.2)                      
#>  lubridate       1.7.4      2018-04-11 [1] CRAN (R 3.6.0)                      
#>  magrittr        1.5        2014-11-22 [1] CRAN (R 3.6.0)                      
#>  markdown        1.1        2019-08-07 [1] CRAN (R 3.6.1)                      
#>  MASS            7.3-51.5   2019-12-20 [1] CRAN (R 3.6.2)                      
#>  Matrix          1.2-18     2019-11-27 [2] CRAN (R 3.6.2)                      
#>  matrixStats     0.55.0     2019-09-07 [1] CRAN (R 3.6.1)                      
#>  memoise         1.1.0      2017-04-21 [1] CRAN (R 3.6.0)                      
#>  mime            0.8        2019-12-19 [1] CRAN (R 3.6.2)                      
#>  miniUI          0.1.1.1    2018-05-18 [1] CRAN (R 3.6.0)                      
#>  minqa           1.2.4      2014-10-09 [1] CRAN (R 3.6.0)                      
#>  munsell         0.5.0      2018-06-12 [1] CRAN (R 3.6.0)                      
#>  nlme            3.1-143    2019-12-10 [1] CRAN (R 3.6.2)                      
#>  nloptr          1.2.1      2018-10-03 [1] CRAN (R 3.6.0)                      
#>  nnet            7.3-12     2016-02-02 [2] CRAN (R 3.6.2)                      
#>  parsnip       * 0.0.5      2020-01-07 [1] CRAN (R 3.6.2)                      
#>  pillar          1.4.3      2019-12-20 [1] CRAN (R 3.6.2)                      
#>  pkgbuild        1.0.6      2019-10-09 [1] CRAN (R 3.6.1)                      
#>  pkgconfig       2.0.3      2019-09-22 [1] CRAN (R 3.6.1)                      
#>  pkgload         1.0.2      2018-10-29 [1] CRAN (R 3.6.0)                      
#>  plyr            1.8.5      2019-12-10 [1] CRAN (R 3.6.1)                      
#>  prettyunits     1.1.1      2020-01-24 [1] CRAN (R 3.6.2)                      
#>  pROC            1.16.1     2020-01-14 [1] CRAN (R 3.6.2)                      
#>  processx        3.4.2      2020-02-09 [1] CRAN (R 3.6.2)                      
#>  prodlim         2019.11.13 2019-11-17 [1] CRAN (R 3.6.1)                      
#>  promises        1.1.0      2019-10-04 [1] CRAN (R 3.6.1)                      
#>  ps              1.3.0      2018-12-21 [1] CRAN (R 3.6.0)                      
#>  purrr         * 0.3.3      2019-10-18 [1] CRAN (R 3.6.1)                      
#>  R6              2.4.1      2019-11-12 [1] CRAN (R 3.6.1)                      
#>  ranger          0.12.1     2020-01-10 [1] CRAN (R 3.6.2)                      
#>  Rcpp            1.0.3      2019-11-08 [1] CRAN (R 3.6.1)                      
#>  recipes       * 0.1.9      2020-01-07 [1] CRAN (R 3.6.2)                      
#>  remotes         2.1.0      2019-06-24 [1] CRAN (R 3.6.0)                      
#>  reshape2        1.4.3      2017-12-11 [1] CRAN (R 3.6.0)                      
#>  rlang           0.4.4      2020-01-28 [1] CRAN (R 3.6.2)                      
#>  rmarkdown       2.1        2020-01-20 [1] CRAN (R 3.6.2)                      
#>  rpart           4.1-15     2019-04-12 [2] CRAN (R 3.6.2)                      
#>  rprojroot       1.3-2      2018-01-03 [1] CRAN (R 3.6.0)                      
#>  rsample       * 0.0.5      2019-07-12 [1] CRAN (R 3.6.1)                      
#>  rsconnect       0.8.16     2019-12-13 [1] CRAN (R 3.6.1)                      
#>  rstan           2.19.2     2019-07-09 [1] CRAN (R 3.6.1)                      
#>  rstanarm        2.19.2     2019-10-03 [1] CRAN (R 3.6.1)                      
#>  rstantools      2.0.0      2019-05-09 [1] Github (stan-dev/rstantools@03a9de3)
#>  rstudioapi      0.11       2020-02-07 [1] CRAN (R 3.6.2)                      
#>  scales        * 1.1.0      2019-11-18 [1] CRAN (R 3.6.1)                      
#>  sessioninfo     1.1.1      2018-11-05 [1] CRAN (R 3.6.0)                      
#>  shiny           1.4.0      2019-10-10 [1] CRAN (R 3.6.1)                      
#>  shinyjs         1.1        2020-01-13 [1] CRAN (R 3.6.2)                      
#>  shinystan       2.5.0      2018-05-01 [1] CRAN (R 3.6.0)                      
#>  shinythemes     1.1.2      2018-11-06 [1] CRAN (R 3.6.0)                      
#>  SnowballC       0.6.0      2019-01-15 [1] CRAN (R 3.6.0)                      
#>  StanHeaders     2.21.0-1   2020-01-19 [1] CRAN (R 3.6.2)                      
#>  stringi         1.4.5      2020-01-11 [1] CRAN (R 3.6.2)                      
#>  stringr         1.4.0      2019-02-10 [1] CRAN (R 3.6.0)                      
#>  survival        3.1-8      2019-12-03 [1] CRAN (R 3.6.1)                      
#>  testthat        2.3.1      2019-12-01 [1] CRAN (R 3.6.1)                      
#>  threejs         0.3.3      2020-01-21 [1] CRAN (R 3.6.2)                      
#>  tibble        * 2.1.3      2019-06-06 [1] CRAN (R 3.6.0)                      
#>  tidymodels    * 0.0.3      2019-10-04 [1] CRAN (R 3.6.2)                      
#>  tidyposterior   0.0.2      2018-11-15 [1] CRAN (R 3.6.1)                      
#>  tidypredict     0.4.3      2019-09-03 [1] CRAN (R 3.6.1)                      
#>  tidyr         * 1.0.2      2020-01-24 [1] CRAN (R 3.6.2)                      
#>  tidyselect      1.0.0      2020-01-27 [1] CRAN (R 3.6.2)                      
#>  tidytext        0.2.2      2019-07-29 [1] CRAN (R 3.6.1)                      
#>  timeDate        3043.102   2018-02-21 [1] CRAN (R 3.6.0)                      
#>  tokenizers      0.2.1      2018-03-29 [1] CRAN (R 3.6.0)                      
#>  tune          * 0.0.1      2020-02-03 [1] Github (tidymodels/tune@3c14c13)    
#>  usethis         1.5.1      2019-07-04 [1] CRAN (R 3.6.0)                      
#>  vctrs           0.2.2      2020-01-24 [1] CRAN (R 3.6.2)                      
#>  withr           2.1.2      2018-03-15 [1] CRAN (R 3.6.0)                      
#>  workflows     * 0.1.0      2019-12-30 [1] CRAN (R 3.6.2)                      
#>  xfun            0.12       2020-01-13 [1] CRAN (R 3.6.2)                      
#>  xtable          1.8-4      2019-04-21 [1] CRAN (R 3.6.0)                      
#>  xts             0.12-0     2020-01-19 [1] CRAN (R 3.6.2)                      
#>  yaml            2.2.0      2018-07-25 [1] CRAN (R 3.6.0)                      
#>  yardstick     * 0.0.5      2020-01-23 [1] CRAN (R 3.6.2)                      
#>  zoo             1.8-7      2020-01-10 [1] CRAN (R 3.6.2)                      
#> 
#> [1] C:/Users/plieninger/Documents/R/win-library/3.6
#> [2] C:/Program Files/R/R-3.6.2/library
@hplieninger
Copy link
Author

Was this closed on purpose? IMHO, the problem seems to persist in the master branch. Or am I overlooking something?

@topepo topepo reopened this Feb 21, 2020
@topepo
Copy link
Member

topepo commented Feb 21, 2020

Sorry. That was closed in error. I'll take a look at this in the next few days.

hplieninger pushed a commit to hplieninger/tune that referenced this issue Feb 26, 2020
@github-actions
Copy link

github-actions bot commented Mar 7, 2021

This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.

@github-actions github-actions bot locked and limited conversation to collaborators Mar 7, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants