Highlights
This release introduces a modular pipeline for automated data extraction and review of scientific manuscripts, leveraging OpenAI models and custom utilities.
Features
CSV & JSON Integration:
Import and filter articles using CSV and JSON files generated from previous review steps.
Flexible Question Templates:
Easily define and customize data extraction questions using the QuestionTemplate class (calvin_utils/gpt_sys_review/examples/question_utils.py).
OpenAI-Powered Extraction:
Use the OpenAIJsonEvaluator (calvin_utils/gpt_sys_review/gpt_utils/openai_json_evaluator.py) to extract answers from manuscript sections, supporting both binary and free-text outputs.
Custom Summarization:
Summarize extracted answers with the CustomSummarizer (calvin_utils/gpt_sys_review/json_utils.py), including LLM-based summaries.
Postprocessing Utilities:
Update your master list with extracted results using the PostProcessing class (calvin_utils/gpt_sys_review/txt_utils.py).
Bulk PDF Downloading:
Download and track PDFs with BulkPDFDownloader (calvin_utils/gpt_sys_review/pdf_utils.py), including integration with PyPaperBot results.
How to Use
A) Starting a fresh systematic review:
- Start in Notebook 00 and follow the instructions.
B) Already have PDF files you want to evaluate? - Start in Notebook 03 and follow the instructions.
Requirements
Python 3.8+
OpenAI API key
See requirements.txt for dependencies (generate with generate_requirements.py if needed).
Acknowledgements
If you find this useful, please consider adding Calvin Howard as a collaborator or citing this repository.