An open-source, resource-efficient autonomous LLM agent framework, built as an evaluation harness for benchmarking model routing, tool-calling transport, and context-management strategies on the cost–latency–accuracy Pareto frontier.
Research paper (preprint): "The Agentic Recipe: Exploring Cost-Performance Trade-offs Beyond Model Routing in LLM Agents."
Autonomous LLM agents default to the largest, most expensive model for every turn of the ReAct loop. Small models are cheap but fail at complex tool-calling. There is no agreed "agentic recipe" for balancing them. Elias is the instrument we use to measure that trade-off empirically, and the open artifact that makes the measurements reproducible.
elias/
├── paper/
│ ├── arxiv/ ← preprint (LaTeX, TMLR template) — grows into TMLR
│ │ ├── main.tex
│ │ ├── references.bib
│ │ └── figures/
│ └── joss/ ← JOSS software paper (~1000 words)
│ ├── paper.md
│ └── paper.bib
├── src/ ← Elias source (Go single-binary harness) — built fresh
├── experiments/
│ ├── tasks/ ← Golden Transcripts (YAML task definitions)
│ ├── recipes/ ← Agentic recipe configs (baselines + Elias configs)
│ ├── results/ ← Raw telemetry output (JSON) + derived tables
│ └── analysis/ ← Statistical tests + plotting scripts
├── docs/ ← installation, tutorial, api
├── tests/
├── CITATION.cff ← academic citation metadata
├── CONTRIBUTING.md ← 4-level contributor model
├── GOVERNANCE.md ← team decision-making + CRediT roles
├── LICENSE ← MIT (code) / CC-BY 4.0 (data, see experiments/)
└── README.md
Early. The harness is being built from scratch. The critical path is the minimal agent loop that can emit one real telemetry row (tokens, latency, cost, tool success) — everything downstream (recipes, benchmarks, the preprint's figure) depends on it.
- Code: MIT (see
LICENSE). - Data / telemetry / golden transcripts: CC-BY 4.0, published openly.
If you use this software or benchmark data in your research, please cite it using
the metadata in CITATION.cff.