Skip to content

better error message when validating a calibration using a validation set #182

Description

@topepo

This doesn't work because there is not a complete set of predictions on the entire data set.

library(tidymodels)
library(probably)

"https://github.com/aml4td/website/raw/refs/heads/main/RData/mushrooms.RData" |> 
  url() |> 
  load()

# a validation set: 
shroom_rs
#> # A tibble: 1 × 2
#>   splits               id        
#>   <list>               <chr>     
#> 1 <split [42748/6107]> validation

nb_res <-
  logistic_reg() %>%
  fit_resamples(
    class ~ .,
    resamples = shroom_rs,
    control = control_resamples(save_pred = TRUE, save_workflow = TRUE)
  )
#> → A | warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
#> There were issues with some computations   A: x1
#> There were issues with some computations   A: x1
#> 

nb_res |> cal_validate_beta() 
#> Error in `purrr::map()`:
#> ℹ In index: 1.
#> Caused by error in `vctrs::vec_slice()`:
#> ! Can't subset elements past the end.
#> ℹ Locations 6108, 6109, 6110, …, 42747, and 42748 don't exist.
#> ℹ There are only 6107 elements.

Created on 2025-07-30 with reprex v2.1.1

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions