failing to pair roc curves builed with formula

[Marc-Flo]

Hello !

I'm currently working on a database whicj contains missing values for the predictors.
When applying are.paired function to two roc objects built with the formula method, i get a negative result...
I have tried to apply the are.paired function to the same roc object built with the default method and this time i get a positive result.

Is there any reason that can explain why the roc object built with the formula method aren't recognized as paired ?

Thank you !

[xrobin]

Works for me:

> data(aSAH)
> roc1 <- roc(outcome ~ wfns, aSAH)
Setting levels: control = Good, case = Poor
Setting direction: controls < cases
> roc2 <- roc(outcome ~ ndka, aSAH)
Setting levels: control = Good, case = Poor
Setting direction: controls < cases
> are.paired(roc1, roc2)
[1] TRUE

Please provide the steps to reproduce as requested when you created this bug report.

[Marc-Flo]

Here's an exemple of the code i used yesterday :

test <- data.frame(
  response = c(1,0,1,0,0,0,1,0,0,0,0,0,0,1,1,0,0,0,1,1),
  pred_1 = c(15,0,30,32,0,10,64,NA,10,0,0,0,0,0,108,0,0,0,56,22),
  pred_2 = c(0.7,0.1,0.2,0.2,1.4,0.1,0.2,1.9,0.8,0.4,NA,0.5,0.6,NA,3.7,1.7,0.3,0.7,0.7,NA)
  )

stand_1 <- roc(test$response, test$pred_1, algorithm = 1, ci = T)
stand_2 <- roc(test$response, test$pred_2, algorithm = 1, ci = T)

are.paired(stand_1, stand_2)

form_1 <- roc(response ~ pred_1, data = test, algorithm = 1, ci = T)
form_2 <- roc(response ~ pred_2, data = test, algorithm = 1, ci = T)

are.paired(form_1, form_2) 

[xrobin]

Thanks for the update.
It's probably due to model.frame taking care of the NAs by itself. It works again if you set it to pass NAs, and let the main roc method take care of them.

na_1 <- roc(response ~ pred_1, data = test, algorithm = 1, ci = T, na.action=na.pass)
na_2 <- roc(response ~ pred_2, data = test, algorithm = 1, ci = T, na.action=na.pass)
are.paired(na_1, na_2) 
[1] TRUE

I will see how this can be handled. At the moment I'm thinking setting a default na.action=na.pass and documenting the behavior if a different na.action is used, but there might be a better way.

[xrobin]

It's working now. The na.action is set to na.pass internally. Nothing changed from a user's perspective, except a warning if you pass a different na.action (typically na.omit or na.exclude).

> form_1 <- roc(response ~ pred_1, data = test, algorithm = 1, ci = T)
> form_2 <- roc(response ~ pred_2, data = test, algorithm = 1, ci = T)
> 
> are.paired(form_1, form_2) 
[1] TRUE

It will be available in pROC 1.16 that I'm going to release within the next couple of weeks. Thanks again for the report!