Self-Polish

Codes for the paper Self-Polsih: Enhance Reasoning in Large Language Models via Problem Refining.

Code Structure

• We use object-oriented style programming, and decouple the codes on the question side and the answer side. We can combine them for better performance. We implement the following methods not.

  • Problem Side: There are two strategies for problems:
    • Self-Polish (The proposed method): The proposed method that optimizes problems till an converged answer is got.
    • Normal: The original method. No optimization is used upon problems.
    • 
  • Answer Side: There are three strategies for answer generation:
    • Few Shot: Generate Answer with several examples.
    • Few Shot CoT: Generate Answer with several examples, and each example contains the chain-of-thought reasoining process.
    • Least-to-Most: Reduce problems to sub problems and then solve them one by one。
    • 

• We also list some test prompts to rewrite problems in the prompt directory.

Run Code

sh test_normal.sh
sh test_sp.sh

• Note that you should set your OpenAI API key in the test_Normal_with_exception.py or test_SP_with_exception.py:

keys = ["Your Key"]

Method Overview