Elapsed Time Measure works only on response predictions?

[bblodfon]

Hi,

Shouldn't the time required for a learner to train and predict on a (resampled) task be independent of the learner's predicted measure type? For example, the following doesn't work, though the times to train and predict the model are available via the resample object:

library(mlr3verse)
#> Loading required package: mlr3
library(mlr3proba)

task = tsk('rats')
poe  = po("encode", method = 'treatment')
task = poe$train(list(task))[[1]]

rr   = resample(task, lrn('surv.coxph'), rsmp('cv', folds = 5))
#> INFO  [10:19:40.631] [mlr3] Applying learner 'surv.coxph' on task 'rats' (iter 4/5) 
#> INFO  [10:19:40.703] [mlr3] Applying learner 'surv.coxph' on task 'rats' (iter 2/5) 
#> INFO  [10:19:40.749] [mlr3] Applying learner 'surv.coxph' on task 'rats' (iter 1/5) 
#> INFO  [10:19:40.777] [mlr3] Applying learner 'surv.coxph' on task 'rats' (iter 3/5) 
#> INFO  [10:19:40.807] [mlr3] Applying learner 'surv.coxph' on task 'rats' (iter 5/5)

rr$score(msr('time_train')) # time_train is `NaN`
#>              task task_id                learner learner_id         resampling
#> 1: <TaskSurv[54]>    rats <LearnerSurvCoxPH[36]> surv.coxph <ResamplingCV[20]>
#> 2: <TaskSurv[54]>    rats <LearnerSurvCoxPH[36]> surv.coxph <ResamplingCV[20]>
#> 3: <TaskSurv[54]>    rats <LearnerSurvCoxPH[36]> surv.coxph <ResamplingCV[20]>
#> 4: <TaskSurv[54]>    rats <LearnerSurvCoxPH[36]> surv.coxph <ResamplingCV[20]>
#> 5: <TaskSurv[54]>    rats <LearnerSurvCoxPH[36]> surv.coxph <ResamplingCV[20]>
#>    resampling_id iteration           prediction time_train
#> 1:            cv         1 <PredictionSurv[20]>        NaN
#> 2:            cv         2 <PredictionSurv[20]>        NaN
#> 3:            cv         3 <PredictionSurv[20]>        NaN
#> 4:            cv         4 <PredictionSurv[20]>        NaN
#> 5:            cv         5 <PredictionSurv[20]>        NaN

# learners have that information though:
for (learner in rr$learners) {
  print(learner$timings)
}
#>   train predict 
#>   0.011   0.010 
#>   train predict 
#>   0.015   0.012 
#>   train predict 
#>   0.011   0.011 
#>   train predict 
#>   0.018   0.013 
#>   train predict 
#>   0.011   0.011

^{Created on 2022-06-20 by the reprex package (v2.0.1)}

[mllg]

Should be fixed now.