Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update residual-checking.Rmd #268

Closed
wants to merge 3 commits into from
Closed

Conversation

ericward-noaa
Copy link
Collaborator

Added diagnostics for delta/presence-absence models: AUC, TSS

TSS addition motivated by discussion here:
#84

It'd be easy to turn the TSS code into a function if it's more broadly useful

Added diagnostics for delta/presence-absence models: AUC, TSS
@seananderson
Copy link
Member

Thanks - can you add pROC to .github/workflows/pkgdown.yaml in the extra packages for web-only vignettes? And maybe remove the extra spaces in roc <- pROC::roc( actual_values, plogis(pred$est) )?

I'm trying to decide if this fits with the residual checking vignette given it doesn't involve residuals. Would the cross-validation vignette be a more natural place maybe adding a quick example of AUC on the cross-validation predictions?

@ericward-noaa
Copy link
Collaborator Author

  • Deleted the spaces
  • I was hoping to avoid another dependency on pROC, so I made both the chunks that used it not evaluated (only 1 was off before, hence the warning)
  • I can go either way on using this here or in the cross validation. I think there's lots of examples of people using AUC on train or test data

@seananderson
Copy link
Member

I've incorporated a version of this into the cross validation vignette, including an example of AUC in cross validation. I'll push it shortly.

seananderson added a commit that referenced this pull request Sep 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants