How does cutpointr classify tp/fn and tn/fp if a sample has the same numeric values as the cutpoints?

[AnVCR]

When using cutpointr to determine cut-off on qPCR Ct-values (numeric) based on a non-quantitative PCR (as factor) the summary(cutpoint) gives the number of tp/fn/tn/fp used. If I try to plot Ct-values of centrifuged and non-centrifuged samples plotting the derived cutoffs (by cutpointr) the number of dots on the plot does not always count. In one case, a point is positive (factor) and its Ct-values for centrifuged and non-centrifuged are the exact values of both cutoffs. The cutpointr package seems to classify this point as both true positive (most likely) and fals negative (although color=factor gives it a positive label, which is correct when looking into the dataset). Is it possible that the cutpointr package uses >= instead of > to classify samples as being tn? or is there a different explanation possible?

[Thie1e]

Hi,

you are correct, cutpointr uses >= or <=. You should be able to confirm that by looking at the direction column of the output.

If you don't supply >= or <= as direction when calling cutpointr, it will choose the direction automatically so that the AUC > 0.5. If you don't supply the positive or negative class, it will also determine automatically which class is the positive one.

Do your results seem plausible then? Otherwise, could you post a reproducible example?

[AnVCR]

Hi Christian, Thank you for your fast reply. Indeed, we used direction <= in our cutpointr function since the data are Ct-values where big values mean less DNA in the sample to detect. Is there an alternative to changing the direction? Thank you for your help! Best regards Anne CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. Hi, you are correct, cutpointr uses >= or <=. You should be able to confirm that by looking at the direction column of the output. If you don't supply >= or <= as direction when calling cutpointr, it will choose the direction automatically so that the AUC > 0.5. Do your results seem plausible then? Otherwise, could you post a reproducible example? - Reply to this email directly, view it on GitHub<#64 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BD26CFOPED5NHT63WAN7YHTYDTM4VAVCNFSM6AAAAAA7ESKUXOVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TKMRSGY2DM>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[AnVCR]

Here's the plot: ***@***.*** Van: Christian Thiele ***@***.***> Verzonden: donderdag 9 november 2023 14:45 Aan: Thie1e/cutpointr ***@***.***> CC: Anne Van Caesbroeck ***@***.***>; Author ***@***.***> Onderwerp: Re: [Thie1e/cutpointr] How does cutpointr classify tp/fn and tn/fp if a sample has the same numeric values as the cutpoints? (Discussion #64) CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. Hi, you are correct, cutpointr uses >= or <=. You should be able to confirm that by looking at the direction column of the output. If you don't supply >= or <= as direction when calling cutpointr, it will choose the direction automatically so that the AUC > 0.5. Do your results seem plausible then? Otherwise, could you post a reproducible example? - Reply to this email directly, view it on GitHub<#64 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BD26CFOPED5NHT63WAN7YHTYDTM4VAVCNFSM6AAAAAA7ESKUXOVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TKMRSGY2DM>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[AnVCR]

Hi Christian, Here's the code we use: for(i in 1:nrow(todo_df)){ val1<-todo_df[i,1] val2<-todo_df[i,2] df_val1<-clean_IARC[,val1] df_val2<-clean_IARC[,val2] cutpoints<-cutpointr(df_val1,class=df_val2,"youden", pos_class = "positive", neg_class = "negative", na.rm=T, direction = "<=") cutpoints_summary[i] <- cutpoints$optimal_cutpoint print(colnames(clean_IARC[val1])) print(summary(cutpoints)) } for (i in "plot_names") { for(i in 1:nrow(plotorder_df)){ val1_plot<-plotorder_df[i,1] val2_plot<-plotorder_df[i,2] val3_plot<-plotorder_df[i,3] df_val1_plot<-clean_IARC[,val1_plot] df_val2_plot<-clean_IARC[,val2_plot] df_val3_plot<-clean_IARC[,val3_plot] print(ggplot(data=clean_IARC,aes(df_val1_plot,df_val2_plot, color=df_val3_plot))+labs(x="Ct-values direct",y="Ct-values centrifuged",color="Gp5+6+")+geom_jitter(position="jitter")+ggtitle(plot_names[i])+geom_hline(yintercept=hvector[i])+geom_vline(xintercept = vvector[i])) }} And the output: [1] "IARC.AllD_hpv16ct" Method: maximize_metric Predictor: x Outcome: class Direction: <= Subgroups: youden Subgroup: youden ------------------------------------------------------------------------------------------------------------ AUC n n_pos n_neg 0.9771 698 22 676 optimal_cutpoint sum_sens_spec acc sensitivity specificity tp fn fp tn 39.2 1.9545 0.9986 0.9545 1 21 1 0 676 Predictor summary: Data Min. 5% 1st Qu. Median Mean 3rd Qu. 95% Max. SD NAs Overall 19.33 46.000 46.00 46.00 45.49307 46.000 46.0000 46 2.8935410 0 negative 39.39 46.000 46.00 46.00 45.96820 46.000 46.0000 46 0.4231588 0 positive 19.33 21.126 27.69 29.51 30.89364 35.665 39.1845 46 6.4478166 0 [1] "IARC.AllC_hpv16ct" Method: maximize_metric Predictor: x Outcome: class Direction: <= Subgroups: youden Subgroup: youden ------------------------------------------------------------------------------------------------------------ AUC n n_pos n_neg 0.9771 700 22 678 optimal_cutpoint sum_sens_spec acc sensitivity specificity tp fn fp tn 36.61 1.9545 0.9986 0.9545 1 21 1 0 678 Predictor summary: Data Min. 5% 1st Qu. Median Mean 3rd Qu. 95% Max. SD NAs Overall 17.59 46.000 46.00 46.00 45.41257 46.0000 46.000 46 3.2006055 0 negative 37.63 46.000 46.00 46.00 45.93699 46.0000 46.000 46 0.6798454 0 positive 17.59 19.471 25.22 28.66 29.25091 33.9225 36.594 46 6.5998571 0 Van: Christian Thiele ***@***.***> Verzonden: donderdag 9 november 2023 14:45 Aan: Thie1e/cutpointr ***@***.***> CC: Anne Van Caesbroeck ***@***.***>; Author ***@***.***> Onderwerp: Re: [Thie1e/cutpointr] How does cutpointr classify tp/fn and tn/fp if a sample has the same numeric values as the cutpoints? (Discussion #64) CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. Hi, you are correct, cutpointr uses >= or <=. You should be able to confirm that by looking at the direction column of the output. If you don't supply >= or <= as direction when calling cutpointr, it will choose the direction automatically so that the AUC > 0.5. Do your results seem plausible then? Otherwise, could you post a reproducible example? - Reply to this email directly, view it on GitHub<#64 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BD26CFOPED5NHT63WAN7YHTYDTM4VAVCNFSM6AAAAAA7ESKUXOVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TKMRSGY2DM>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[AnVCR]

[Thie1e]

If you do not want the cutoffs to be exactly equal to one of the values found in your data, you can set use_midpoints = TRUE in cutpointr. The metrics will stay the same, but the new cutoffs will be shifted half the distance to the next lowest or next highest value, depending on direction.

It is a bit difficult to understand your code without having access to the data. I am a bit confused, because there seems to be only one subgroup returned in summary(), called "youden". So setting the subgroup parameter does not seem necessary here.