Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add active learning #445

Open
dzenanz opened this issue Apr 17, 2022 · 2 comments
Open

Add active learning #445

dzenanz opened this issue Apr 17, 2022 · 2 comments
Assignees
Projects

Comments

@dzenanz
Copy link
Member

dzenanz commented Apr 17, 2022

Create a new web component and expose a few parameters which control re-learning:

  • Number of epochs
  • Percentage of data to use for re-training.

When only a subset of data is selected, focus on cases which had the largest discrepancy between automatic and manual decisions.

@dzenanz dzenanz self-assigned this Apr 17, 2022
@dzenanz dzenanz added this to Backlog in scrum board via automation Apr 17, 2022
@dzenanz
Copy link
Member Author

dzenanz commented Apr 20, 2022

My idea: add a "Retrain" button somewhere. When clicked, it shows a new web form with a spin-box for number of epochs prepopulated with a default value (e.g. 5), a 0% to 100% slider initialized to some default (e.g. 100%), and a "Start" button.

When the start button is clicked, invoke a (yet to be written) update(model_path, csv_path, epochs, data_percent) function from nn_training.py. The first is the path to the NN weights file (e.g. ./models/miqaT1-val0.pth), the second a path to a CSV file (e.g. /tmp/20220420_135506.csv). The last two are just numbers from the GUI.

The CSV file should be similar to this:

series_type,normal_variants,lesions,full_brain_coverage,misalignment,swap_wraparound,ghosting_motion,inhomogeneity,susceptibility_metal,flow_artifact,truncation_artifact,overall_qa_assessment,file_path
T1-15,,TRUE,TRUE,FALSE,FALSE,FALSE,FALSE,FALSE,FALSE,FALSE,3,/data/sub-000410/ses-43317/anat/sub-000410_ses-43317_run-003_T1w.nii.gz
T1-30,,FALSE,TRUE,FALSE,FALSE,FALSE,TRUE,FALSE,FALSE,FALSE,8,/data/sub-000527/ses-20877/anat/sub-000527_ses-20877_run-004_T1w.nii.gz
T1-30,,FALSE,TRUE,FALSE,FALSE,FALSE,TRUE,FALSE,FALSE,FALSE,10,/data/sub-000527/ses-20877/anat/sub-000527_ses-20877_run-007_T1w.nii.gz

Namely, normal_variants is always empty as we don't check that. For other artifacts: true if indicated as present otherwise false. OverallQA: 3 for bad, 8 for usable, 10 for "usable extra". This mapping is somewhat arbitrary.

@dzenanz dzenanz moved this from Backlog to Sprint Ready in scrum board Apr 20, 2022
@dzenanz
Copy link
Member Author

dzenanz commented Apr 21, 2022

update method needs to take in the original model, current, and updated model (to be produced).

@curtislisle curtislisle self-assigned this May 26, 2022
@annehaley annehaley moved this from Sprint Ready to Backlog in scrum board Jul 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

No branches or pull requests

3 participants