Optimal cut points on a k fold cv df #65
Replies: 1 comment
-
Hi Miguel, yes, you can calculate cutpoints for predicted values. Since you are doing k-fold CV, I assume you want to simulate the out-of-sample performance. In that case, you could do a nested CV, because you have two optimization tasks: Your first model and the optimal cutpoints. You already have the out-of-sample model predictions and true labels from the first CV routine. Now you could run a second k-fold CV routine on these predictions for the optimal cutpoints. This will result in the CV metrics of the chosen estimation method for optimal cutpoints.
Without knowing the details of your project, I am not sure whether you need additional transformation steps. Generally, you would not need any transformation steps, assuming that your model predictions are for the biological values. Then, the cutpoints are also being calculated on biological values. You simply have to keep in mind that these cutpoints are optimized on the outputs of the model, so for new data, the cutpoints would have to be applied to model predictions again, not raw measurement values. |
Beta Was this translation helpful? Give feedback.
-
Hi all
I have calculated predictions for a number of models using a k fold validation set for my project. My question is can I use predicted values (instead of biological values) to estimate optimal cut-points and whether there are any additional steps I need to take to convert to the original biological value?
Many thanks in advance
Best wishes
Miguel
Beta Was this translation helpful? Give feedback.
All reactions