Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Threshold Used For Experimental Results #6

Closed
ZeroCool2u opened this issue Feb 27, 2020 · 2 comments
Closed

Threshold Used For Experimental Results #6

ZeroCool2u opened this issue Feb 27, 2020 · 2 comments
Labels
question Further information is requested

Comments

@ZeroCool2u
Copy link

Hi there, I was attempting to replicate your results on the Darpa dataset, but realized you didn't specify the threshold you used. I understand the threshold is user defined, but would like to know what value was used in the experimental setup. Could you please clarify how you calculate the MIDAS(R) ROC and what threshold you used?

Thanks!
@bhatiasiddharth
Copy link
Member

bhatiasiddharth commented Feb 27, 2020 •
edited

Hi Theo, thanks for trying out MIDAS. I have added auc.py (used to get the roc auc scores), original darpa dataset and darpa dataset in MIDAS format in the repository.
We don't use a threshold to evaluate. We directly evaluate on the scores using Sklearn's roc_auc_score.

@bhatiasiddharth bhatiasiddharth added the question Further information is requested label Feb 27, 2020
@ZeroCool2u
Copy link
Author

Ah, I understand now. Thanks for clarifying and nice work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ZeroCool2u @bhatiasiddharth