Exploring self-improvement in LLMs.
Motivation: https://arxiv.org/abs/2410.04444 (Gödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement)
-
Short self-improvement in iterations with one LLM. (2/4/2025) Model chosen is the DeepSeek-R1-Distill-Qwen-14B, quantised 4-bit for memory efficiency. In each iteration, the LLM modifies a code file and note file to progress towards its prompted goal. Output of model which includes the note and code file is fed into itself.