COMP545_Final_Project

Generate dataset

python generate_cik_dataset.py

python run_baseline_evaluation.py

python run_llm_evaluation.py --method dp --use-context

Note that you can use --method llmp and --no-context as options as well

Run run_all_domains_mistral.sh for complete run; If a run on a specific domain is needed, please consult usage within run_mistral_standalone.py

Run run_all_domains_gpt4o.sh for complete run; If a run on a specific domain is needed, please consult usage within run_domain_gpt4o.py

python xgboost_selector.py

Results can bee found in /results, for further analysis, please follow the REEADME.md in /results

Name		Name	Last commit message	Last commit date
Benchmarks		Benchmarks
LLM		LLM
Relevant Papers		Relevant Papers
Results		Results
datasets		datasets
results		results
.gitignore		.gitignore
README.md		README.md
context-helps-but-only-at-a-scale.pdf		context-helps-but-only-at-a-scale.pdf
generate_cik_dataset.py		generate_cik_dataset.py
merge_results.py		merge_results.py
model_summary.py		model_summary.py
run_all_domains_gpt4o.sh		run_all_domains_gpt4o.sh
run_all_domains_mistral.sh		run_all_domains_mistral.sh
run_baseline_evaluation.py		run_baseline_evaluation.py
run_domain_gpt4o.py		run_domain_gpt4o.py
run_llm_evaluation.py		run_llm_evaluation.py
run_mistral_standalone.py		run_mistral_standalone.py
validate_data.py		validate_data.py
verify_baseline_results.py		verify_baseline_results.py
xgboost_selector.py		xgboost_selector.py