`ReASC`: Reliability-Aware Adaptive Self-Consistency for Efficient Sampling in LLM Reasoning

$ conda env create -f environment.yaml
$ conda activate reasc

run run_self_certainty_calibration.sh to get result for calibration set in offline settings
edit run_sc_self_certainty.sh and run it for the result of standard Self Consistency.
run each notebook within ./notebooks for the result of ASC, ESC, ReASC Offline and ReASC Online

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
assets		assets
data_generation		data_generation
math500		math500
omnimath		omnimath
.gitignore		.gitignore
asc_code.ipynb		asc_code.ipynb
asc_code_dirichlet.ipynb		asc_code_dirichlet.ipynb
compute_correctness.py		compute_correctness.py
compute_prediction.py		compute_prediction.py
confsc_offline_code.ipynb		confsc_offline_code.ipynb
confsc_online_code.ipynb		confsc_online_code.ipynb
environment.yaml		environment.yaml
esc_code.ipynb		esc_code.ipynb
readme.md		readme.md
run_sc_self_certainty.sh		run_sc_self_certainty.sh
run_self_certainty_calibration.sh		run_self_certainty_calibration.sh
sc_code.ipynb		sc_code.ipynb
sc_with_self_certainty.py		sc_with_self_certainty.py
self_certainty_calibration.py		self_certainty_calibration.py
utils_hf.py		utils_hf.py

Provide feedback