metric_tweak and metric_set doesn't work together

[EmilHvitfeldt]

Originally found here https://stackoverflow.com/questions/75034496/tidymodels-f-meas-metric-tweak-error-on-metric-set

library(yardstick)
#> For binary classification, the first factor level is assumed to be the event.
#> Use the argument `event_level = "second"` to alter this as needed.

f_meas_weighted <- metric_tweak("f_meas_weighted", f_meas, estimator = "macro_weighted")

f_meas_weighted(iris, truth = Species, estimate = Species)
#> # A tibble: 1 × 3
#>   .metric         .estimator     .estimate
#>   <chr>           <chr>              <dbl>
#> 1 f_meas_weighted macro_weighted         1

metric_set(f_meas_weighted)(iris, truth = Species, estimate = Species)
#> Error in `metric_set()`:
#> ! Failed to compute `f_meas_weighted()`.
#> Caused by error in `f_meas.data.frame()`:
#> ! formal argument "estimator" matched by multiple actual arguments

#> Backtrace:
#>     ▆
#>  1. └─metric_set(f_meas_weighted)(iris, truth = Species, estimate = Species)
#>  2.   └─base::mapply(...) at yardstick/R/metrics.R:351:6
#>  3.     └─yardstick (local) `<fn>`(dots[[1L]][[1L]], dots[[2L]][[1L]])
#>  4.       └─base::tryCatch(...) at yardstick/R/metrics.R:561:2
#>  5.         └─base (local) tryCatchList(expr, classes, parentenv, handlers)
#>  6.           └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
#>  7.             └─value[[3L]](cond)
#>  8.               └─rlang::abort(message, parent = cnd, call = call("metric_set")) at yardstick/R/metrics.R:567:6

^{Created on 2023-01-09 with reprex v2.0.2}

[DavisVaughan]

Is this really a bug? The metric_set() result has an argument for estimator that gets passed through. You are supposed to do

library(yardstick)
#> For binary classification, the first factor level is assumed to be the event.
#> Use the argument `event_level = "second"` to alter this as needed.

f_meas_weighted <- metric_tweak("f_meas_weighted", f_meas)

set <- metric_set(f_meas_weighted)

set(iris, truth = Species, estimate = Species, estimator = "macro_weighted")
#> # A tibble: 1 × 3
#>   .metric         .estimator     .estimate
#>   <chr>           <chr>              <dbl>
#> 1 f_meas_weighted macro_weighted         1

^{Created on 2023-02-08 with reprex v2.0.2.9000}

[EmilHvitfeldt]

But that assumes that you want to set the same estimator for all the metrics in the set, which I don't know is something we should force if we can avoid it

Main

library(yardstick)
#> For binary classification, the first factor level is assumed to be the event.
#> Use the argument `event_level = "second"` to alter this as needed.

j_index_macro <- metric_tweak("j_index_macro", j_index, estimator = "macro")
j_index_micro <- metric_tweak("j_index_micro", j_index, estimator = "micro")

set <- metric_set(accuracy, j_index_macro, j_index_micro)

set(iris, truth = Species, estimate = Species)
#> Error in `metric_set()`:
#> ! Failed to compute `j_index_macro()`.
#> Caused by error in `j_index.data.frame()`:
#> ! formal argument "estimator" matched by multiple actual arguments

#> Backtrace:
#>     ▆
#>  1. └─yardstick (local) set(iris, truth = Species, estimate = Species)
#>  2.   └─base::mapply(...) at yardstick/R/metrics.R:351:6
#>  3.     └─yardstick (local) `<fn>`(dots[[1L]][[2L]], dots[[2L]][[2L]])
#>  4.       └─base::tryCatch(...) at yardstick/R/metrics.R:561:2
#>  5.         └─base (local) tryCatchList(expr, classes, parentenv, handlers)
#>  6.           └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
#>  7.             └─value[[3L]](cond)
#>  8.               └─rlang::abort(message, parent = cnd, call = call("metric_set")) at yardstick/R/metrics.R:567:6

PR

library(yardstick)
#> For binary classification, the first factor level is assumed to be the event.
#> Use the argument `event_level = "second"` to alter this as needed.

j_index_macro <- metric_tweak("j_index_macro", j_index, estimator = "macro")
j_index_micro <- metric_tweak("j_index_micro", j_index, estimator = "micro")

set <- metric_set(accuracy, j_index_macro, j_index_micro)

set(iris, truth = Species, estimate = Species)
#> # A tibble: 3 × 3
#>   .metric       .estimator .estimate
#>   <chr>         <chr>          <dbl>
#> 1 accuracy      multiclass         1
#> 2 j_index_macro macro              1
#> 3 j_index_micro micro              1

[github-actions]

This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.