-
Notifications
You must be signed in to change notification settings - Fork 42
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tuning with list-columns in grid
#625
Comments
Hey @SHo-JANG, thanks for the issue. This is a really interesting use case, and I appreciate the helpful reprex! You’ve worked to integrate with the machinery in a very intuitive way. I think the hitch you’re running into here is that tune expects the thing to be tuned to be an argument to the training function. This is where the As such, you’ll need to specify the partials of the objective function as your I realize this is sub-optimal, and means that you have to be careful in keeping track of which partial is which, though I don’t anticipate we’ll develop this interface further as this is an uncommon use case and would require some fundamental changes to tune. You can use the Mirroring your reprex: library(tidymodels)
# focal_loss --------------------------------------------------------------
focal_loss <- function(preds, dtrain, alpha = 0.5,focal_gamma = 0) {
labels <- getinfo(dtrain, "label")
preds <- 1 / (1 + exp(-preds))
p<- preds
y <- labels
grad <- (y*alpha*(1-p)^focal_gamma*(focal_gamma*log(p)*p-(1-p))-
(1-y)*(1-alpha)*p^(focal_gamma)*(focal_gamma*log(1-p)*(1-p)-p))
du <- y*alpha*(1-p)^(focal_gamma-1)*(log(p)*(-focal_gamma^2 *p + (1-p)*focal_gamma)+ 2*focal_gamma*(1-p)+(1-p))
dv <- -(1-y)*(1-alpha)*p^(focal_gamma-1)*(log(1-p)*(focal_gamma^2*(1-p)-p*focal_gamma)-2*focal_gamma*p-p)
hess <- (du+dv)*p*(1-p)
return(list(grad = grad, hess = hess))
}
# data setup ----------------------------------------------------------------
data <- two_class_dat %>%
rename(y = Class)
set.seed(100)
splits <- initial_split(data,prop = 0.8, strata = y)
train_data <- training(splits)
test_data <- testing(splits)
resamples <- vfold_cv(data = train_data,v = 5,strata = y)
# specifications ----------------------------------------------------
xgb_spec <-
boost_tree(mode = "classification", tree_depth = tune(), trees = tune()) %>%
# note that i just set `objective = tune()` here
set_engine(object = ., engine = "xgboost", objective = tune())
xgb_recipe <- train_data %>%
recipe(y ~ .) |>
#step_mutate_at(all_numeric_predictors(), fn = list(orig = ~.)) %>%
step_normalize(all_predictors(), -all_outcomes())
xgb_workflow <-
workflow() %>%
add_recipe(xgb_recipe) %>%
add_model(xgb_spec)
# grid setup ------------------------------------------------------------------
partial_grid <-
expand_grid(
alpha = seq(0, 1, length.out = 3),
focal_gamma = seq(0, 5, length.out = 3)
)
partial_grid$partials <-
map2(
partial_grid$alpha,
partial_grid$focal_gamma,
~partial(focal_loss, alpha = !!.x, focal_gamma = !!.y)
)
xgb_grid <-
extract_parameter_set_dials(xgb_spec) %>%
filter(id != "objective") %>%
grid_latin_hypercube(size = 9) %>%
bind_cols(partial_grid %>% select(objective = partials))
# tune! -----------------------------------------------------------------------
res <- tune_grid(xgb_workflow, resamples = resamples, grid = xgb_grid)
#> → A | error: Error in xgb.iter.update(bst$handle, dtrain, ...
#> There were issues with some computations A: x5
#>
#> Warning: All models failed. Run `show_notes(.Last.tune.result)` for more
#> information. Created on 2023-03-09 with reprex v2.0.2 That said, as you can see, there's an issue here: supplying partials of the function as part of a tibble column means that we need to wrap that collection of functions as a list. So, tune passes each possible value of
and xgboost thus trips up by saying that it doesn't know what to do with a list for an objective function. This is where that change from Line 379 in e61a83b
but that change might actually need to happen in workflows. So, for now, this does not work, but we're on it. :) |
grid
A possible fix in #633! See updated reprex there—you can install those changes with |
pak::pak("tidymodels/tune@625")
#> Error: ! error in pak subprocess
#> Caused by error:
#> ! Could not solve package dependencies:
#> * tidymodels/tune@625: ! pkgdepends resolution error for tidymodels/tune@625.
#> Caused by error:
#> ! Can't find reference @625 in GitHub repo tidymodels/tune. Created on 2023-03-10 with reprex v2.0.2 .Last.error
Backtrace:
Subprocess backtrace:
It doesn't work for me! Thank you for your reply. |
Oh, woops! That ref is |
It works with I sincerely thank you for creating such a useful ecosystem. If there is a necessary function, I will continue to suggest it. (I don't speak English very well, so I communicate through a translator. I'm really sorry if there was any violation of etiquette.) |
This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue. |
I want to customize objective function.
For example, what I want to implement now is a loss function called Focal loss, which can be used when there is a class imbalance.
Focal Loss
Because focal loss is a generalization of cross-entropy, setting certain hyperparameters will result in exactly the same result as CE.
I want to tuning alpha , focal_gamma
Created on 2023-02-28 with reprex v2.0.2
I checked that it works well on the existing xgb.train, but I don't know how to apply the tune function.
From now on, the code below is the way I tried.
Created on 2023-02-28 with reprex v2.0.2
The
extract_parameter_set_dials()
function does not recognize thefocal_gamma
argument.How I can tuning objective function?
Ultimately, I want to tune not only one parameter but also several parameters together.
The text was updated successfully, but these errors were encountered: