Skip to content
This repository has been archived by the owner. It is now read-only.

#271 LUNA16 evaluation #292

Merged
merged 2 commits into from Jan 22, 2018
Merged

#271 LUNA16 evaluation #292

merged 2 commits into from Jan 22, 2018

Conversation

@vessemer
Copy link
Contributor

@vessemer vessemer commented Jan 22, 2018

This allows to compute FROC and CPM metrics with bootstrapping for the classification and detection tasks.

Description

Free-Response Receiver Operating Characteristic (FROC) and competition performance metric (CPM). It computes an average of the seven sensitivities measured at several false positives per scan (FPPS) thresholds, more concretely, at each FPPS ∈ {0.125, 0.25, 0.5, 1, 2, 4, 8} true positive rate was computed. Mean of which forms the CPM as discussed in #271.

Reference to official issue

This reference to the #271.

How Has This Been Tested?

froc_3dlrcnn

CPM over 10-Fold cross validation:

0.125 0.25 0.5 1 2 4 8 Score (CPM)
0.595 0.670 0.731 0.793 0.835 0.868 0.887 0.76

CLA

  • I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well
@WGierke
Copy link
Contributor

@WGierke WGierke commented Jan 22, 2018

Nice! You mentioned that you used 10-fold cross validation. Can you document somewhere how you trained and evaluated the model using your newly introduced methods?

@vessemer
Copy link
Contributor Author

@vessemer vessemer commented Jan 22, 2018

I've used my old pipeline since it fits my currently available one TITAN X GPU, though the pipeline code in my repo looks not that good :)

@vessemer
Copy link
Contributor Author

@vessemer vessemer commented Jan 22, 2018

Three months ago I've worked on algorithms unification and training parts, but then master branch diverges from mine and I abandoned that idea for a while. The code I've written lies here. I hope to return to this soon.

@vessemer
Copy link
Contributor Author

@vessemer vessemer commented Jan 22, 2018

Btw, for 10 fold cross-validation was used split from the data page suggested by the LUNA16 authors.

@lamby
Copy link
Contributor

@lamby lamby commented Jan 22, 2018

This is cool!

@lamby lamby merged commit b924809 into drivendataorg:master Jan 22, 2018
1 of 2 checks passed
1 of 2 checks passed
continuous-integration/travis-ci/pr The Travis CI build could not complete due to an error
Details
@drivendata
concept-to-clinic/cla @vessemer has signed the CLA.
Details
@vessemer vessemer deleted the vessemer:271_LUNA_evaluation branch Jan 22, 2018
This was referenced Jan 23, 2018
@vessemer vessemer mentioned this pull request Feb 1, 2018
1 of 1 task complete
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants