New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plotting fails on ROCs for large data sets #1

Closed
jerheff opened this Issue Jul 20, 2015 · 2 comments

Comments

Projects
None yet
2 participants
@jerheff
Copy link

jerheff commented Jul 20, 2015

I have a data set with ~2.7 million entries heavily skewed toward negative samples. I made a boot.roc object via this call:

boot.roc(data$prediction, data$actual, n.boot=1000, use.cache = TRUE)

Calling plot on the object then fails with this error immediately:

plot(bootrocobject, show.metric="auc")
Error in .Call("fbroc_tpr_at_fpr_cached", PACKAGE = "fbroc", tpr, fpr, :
negative length vectors are not allowed

I then tried a small number of bootstrap samples (100), which does not provide an error message, but appears to hang.

@erikpeter

This comment has been minimized.

Copy link
Owner

erikpeter commented Jul 23, 2015

Thank you for the report. Unfortunately (or fortunately for me) I am in the first week of a three week vacation. I suspect I won't get to work for it before the second half of August.

Some questions:

  • Which version are you using (R and fbroc)?
  • What happens in uncached mode? It is more easy on the memory, so it might help

Is the data set confidential or can you share it? If it is confidential I still would like to know:

  • How many positive and negative samples?
  • Are there ties in the data? If so, how many?
  • What AUC do you get?

@erikpeter erikpeter self-assigned this Jul 23, 2015

@erikpeter

This comment has been minimized.

Copy link
Owner

erikpeter commented Sep 28, 2015

Now fixed on github. Was caused by a stupid default for the number of points at which the ROC curve was calculated. Should also increase performance when plotting.

Until a new version is on CRAN (a while yet) either use the newest version from Github or switch the positive and negative class as a workaround.

@erikpeter erikpeter closed this Sep 28, 2015

@erikpeter erikpeter added the bug label Sep 28, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment