Processor!

[tkornuta-ibm]

• Refactored method that loads default configuration for components (IMPORTANT!)
  • The new approach relies on path saved by setup.py to ~/.ptp/configs
    => As a result, one might run ptp-* in any folder and point configuration file from any other folder as well!
    => Added option to load many configuration files from command line
    (WARNING!) After merge, everyone will have to run python setup.py develop/install
• Refactored Processor, enabled option to process any set (section containing problem) from the config file
• Added StreamFileExporter component that exports values of streams to file
• Create default_extend_answer.yml config that we will use as "secondary" config loaded from command line:
  • Config contains three new sections (training_answers,validation_answers,test_answers) that sample items one by one (without shuffling)
  • It also adds component StreamFileExporter that exports all values from all streams to file
    (INFO: to be used with ptp-processor only that makes a single pass over the whole set)

Exemplary usage: (loads the pretrained question categorization pipeline and uses it to get answers for test set defined in 'test_answers' section of default_extend_answers.yml)

ptp-processor --load ~/image-clef-2019/experiments/q_categorization/20190416_120801/checkpoints/vqa_med_question_categorization_rnn_ffn_best.pt --c configs/vqa_med_2019/question_categorization/question_categorization_rnn_ffn.yml,configs/vqa_med_2019/default_extend_answers.yml --set test_answers

[lgtm-com]

This pull request introduces 1 alert when merging 715fa6a into 19657b8 - view on LGTM.com

new alerts:

• 1 for Unused local variable

---

Comment posted by LGTM.com

[lgtm-com]

This pull request introduces 1 alert when merging 47fd27e into 19657b8 - view on LGTM.com

new alerts:

• 1 for Unused local variable

---

Comment posted by LGTM.com

[lgtm-com]

This pull request introduces 1 alert when merging 25683d3 into 19657b8 - view on LGTM.com

new alerts:

• 1 for Unused local variable

---

Comment posted by LGTM.com