Official code release for the paper Test-Time Learning with an Evolving Library by Weijia Xu, Alessandro Sordoni, Chandan Singh, Zelalem Gero, Michel Galley, Xingdi Yuan, and Jianfeng Gao (Microsoft Research).
EvoLib is a test-time learning (TTL) framework that enables black-box large language models to accumulate, reuse, and evolve knowledge across problem instances without any parameter updates or external supervision. Instead of adapting the model weights, EvoLib maintains a shared, evolving library of knowledge abstractions extracted from the model's own inference trajectories:
- Modular skills — reusable procedures (functions for coding, sub-problem solutions for reasoning, sub-task workflows for agentic tasks).
- Reflective insights — natural-language rules distilled from past mistakes.
The library is updated through abstraction extraction, consolidation, and a dynamic weighting scheme based on Information Gain (IG) and Future IG, which jointly capture the immediate usefulness of an abstraction and its long-term value for generating future useful abstractions.
- No gradient updates. Works with any black-box LLM behind an API.
- No ground-truth supervision. Self-evaluated scores (synthetic tests, majority voting, LLM-as-a-Judge) drive learning.
- Two complementary abstraction types. Skills supply reusable solution components; insights guide future attempts away from common errors.
- Dynamic credit assignment. IG rewards abstractions for immediate utility; Future IG propagates credit backward along evolution chains so that abstractions that enable useful future abstractions are also up-weighted.
- Consolidation mechanism. Semantically/functionally similar abstractions are merged, controlling library growth and producing increasingly general, reusable knowledge units.
- Continual learning friendly. Robust to task ordering, including random streams of heterogeneous tasks.
Cost–performance curves comparing EvoLib with competitive test-time scaling (Best-of-N, RSA) and test-time learning (Dynamic Cheatsheet) baselines. The x-axis is the weighted token count (input tokens weighted by 1, output tokens by 4); the y-axis is task performance. EvoLib achieves consistently higher accuracy under varying compute budget and improves more rapidly per token spent.
Left: BigCodeBench Hard (GPT-4o). Middle: LiveCodeBench v6 Hard (o4-mini).
Right: HMMT 2025–2026 (o4-mini).
See the paper for the full results table (including results on agentic tasks), ablations, and further analysis.
.
├── EvoLib/ # EvoLib agent + evaluation on math and code generation
│ ├── evolib_agent.py # Core EvoLib algorithm.
│ ├── eval_main.py # CLI entry point and per-benchmark task prompts.
│ ├── llm.py # Azure OpenAI chat wrapper with token accounting.
│ ├── embed.py # Embedding model wrapper.
│ └── utils.py # Utility functions.
│
└── AgentBoard/ # AgentBoard fork with the EvoLib agent integrated for multi-turn agentic tasks
├── agentboard/agents/evo_agent.py
├── eval_configs/evo_agent_all_tasks.yaml
└── README.md # AgentBoard setup and evaluation instructions.
EvoLib is split into two sub-projects because they have different upstream dependencies. Install only the parts you need.
cd EvoLib
pip install -r requirements.txtTask-specific dependencies (install only for the benchmarks you want to run):
- LiveCodeBench:
lcb_runner - BigCodeBench:
bigcodebench - HMMT:
matharena
See AgentBoard/README.md for the full environment setup, including dataset download.
All math/code experiments run through EvoLib/eval_main.py. Authentication uses
Azure Managed Identity / Azure CLI tokens via azure-identity.
cd EvoLib
# BigCodeBench Hard (GPT-4o)
python eval_main.py \
--task bigcodebench \
--k_q_per_problem 3 \
--endpoint <chat_completions_endpoint> \
--embedding_endpoint <embeddings_endpoint> \
--syn_test_data_file <bigcodebench_synthetic_tests.json> \
--log_file ./runs/bigcode.log
# LiveCodeBench v6 Hard (o4-mini, high reasoning)
python eval_main.py \
--task livecodebench \
--k_q_per_problem 3 \
--endpoint <chat_completions_endpoint> \
--embedding_endpoint <embeddings_endpoint> \
--syn_test_data_file <livecodebench_synthetic_tests.json> \
--log_file ./runs/livecode.log
# HMMT 2025-2026 (o4-mini, high reasoning)
python eval_main.py \
--task hmmt \
--k_q_per_problem 10 \
--endpoint <chat_completions_endpoint> \
--embedding_endpoint <embeddings_endpoint> \
--log_file ./runs/hmmt.logRe-running with the same --log_file resumes from the last [Iter N] recorded
in the log. Library and solution snapshots are written next to the log file as
*.skill.json, *.insight.json, *.score.json, and *.solution.json every
20 iterations.
cd AgentBoard
python agentboard/eval_main.py \
--cfg-path eval_configs/evo_agent_all_tasks.yaml \
--tasks pddl \
--model o4-mini \
--wandb \
--log_path ./results/pddl-evolib-o4-mini \
--project_name evaluate-pddl-evolib-o4-mini \
--baseline_dir ./data/baseline_resultsSwitch --tasks pddl to --tasks scienceworld for the ScienceWorld experiment.
If you use EvoLib in your research, please cite:
@article{xu2026evolib,
title = {Test-Time Learning with an Evolving Library},
author = {Xu, Weijia and Sordoni, Alessandro and Singh, Chandan and Gero, Zelalem and Galley, Michel and Yuan, Xingdi and Gao, Jianfeng},
journal = {arXiv preprint arXiv:2605.14477},
year = {2026},
url = {https://arxiv.org/abs/2605.14477}
}The agentic-task experiments are built on top of AgentBoard; the code/math experiments use evaluation harnesses from LiveCodeBench, BigCodeBench, and MathArena. We thank the authors for releasing their code.
This project is released under the MIT License. Code from the upstream projects listed above retains its original license.
Better performance can be achieved by selecting a strong and well-governed base model, carefully configuring prompting and evaluation settings, validating behavior on the target task distribution, and monitoring library growth and abstraction quality over time.
We strongly encourage users to use LLMs/MLLMs that support robust Responsible AI mitigations, such as Azure
Open AI (AOAI) services. Such services continually update their safety and RAI mitigations with the latest industry standards for responsible use. For more on AOAI’s best practices when employing foundations models for scripts and applications:
What is Azure AI Content Safety?
Overview of Responsible AI practices for Azure OpenAI models
Azure OpenAI Transparency Note
Azure OpenAI’s Code of Conduct
Users are responsible for sourcing their datasets legally and ethically. This could include securing appropriate rights, ensuring consent for use of audio/images, and/or the anonymization of data prior to use in research.
Users are reminded to be mindful of data privacy concerns and are encouraged to review the privacy policies associated with any models and data storage solutions interfacing with EvoLib.
It is the user's responsibility to ensure that the use of EvoLib complies with relevant data protection regulations, license terms, and organizational guidelines.
Developers should follow transparency best practices and inform end-users they are interacting with an AI system.
We do not recommend using EvoLib in commercial or real-world applications without further testing and development. It is being released for research purposes. EvoLib was not designed or evaluated for all possible downstream purposes. On tasks where the base model cannot evaluate and verify solutions reliably, the iterative nature of the EvoLib algorithm may introduce accumulated errors that degrade downstream performance. Developers should consider its inherent limitations as they select use cases, and evaluate and mitigate for accuracy, safety, and fairness concerns specific to each intended downstream use.
Without further testing and development, EvoLib should not be used in sensitive domains where inaccurate outputs could suggest actions that lead to injury or negatively impact an individual's legal, financial, or life opportunities. We do not recommend using EvoLib in the context of high-risk decision making (e.g. in law enforcement, legal, finance, or healthcare).
This research was conducted by Microsoft Research. We welcome feedback and collaboration from our audience. If you have suggestions, questions, or observe unexpected or offensive behavior in our technology, please contact Weijia Xu (weijiaxu@microsoft.com).
If the team receives reports of undesired behavior or identifies issues independently, we will update this repository with appropriate mitigations.