LLMTDDLoop
title: LLM TDD Loop type: technique created: 2026-05-16 last_updated: 2026-05-16 related: ["Playradar", "radar/tools/TDDGuard"] sources: ["https://codeinthehole.com/tips/llm-tdd-loop-script/"] radar_quadrant: Techniques radar_ring: Assess radar_position: center
A technique for AI-assisted code generation in which a shell script feeds a test file to a language model, executes the tests, and iterates — sending failure output back to the model — until all tests pass or a maximum attempt count is reached.
The script (tdd.sh) implements an epsilon-style loop:
- Feed the test file to the LLM with an instruction to generate passing implementation code.
- Write the generated code to disk.
- Run
pytestagainst the tests. - If tests pass: exit successfully.
- If tests fail: send the failure output back to the LLM requesting a fix.
- Repeat up to a configured maximum (4 attempts in the reference implementation).
A conventions.txt file constrains LLM output format: plain executable Python only, no Markdown fences, no YAML, no explanatory prose. This prevents the model from returning decorated output that would fail as raw Python.
| Tool | Role |
|---|---|
llm (Simon Willison) |
CLI for invoking language models |
files-to-prompt |
Prepares file contents as LLM context |
pytest |
Executes and validates the test suite |
bat |
Syntax-highlighted terminal output |
The author documents a known failure case: when test assertions are contradictory, the LLM generates code that technically passes the tests while producing nonsensical behaviour. Test quality is a prerequisite for output quality — the loop enforces test compliance, not semantic correctness.
The technique remains a proof-of-concept. The author describes it as "just a fun experiment" rather than a production methodology.
The loop inverts the normal developer role in TDD. In conventional TDD, the developer writes tests and then implements code to pass them. In the LLM TDD loop, the developer writes tests and the LLM implements the code. The developer's job shifts from implementation to test quality and output review.
LLM TDD Loop sits in the Assess ring of the Techniques quadrant, at center position. First studied via David Winterbottom's article on codeinthehole.com on 2026-05-16; no personal implementation to date. The technique is experimental — explicitly described as a proof-of-concept — and requires investment in the llm CLI toolchain. Center position reflects a feasible but not immediately actionable trial: the toolchain is accessible, but the technique has limited scope (works best for pure function implementations with clean test boundaries) and carries open questions about code quality and maintainability beyond test compliance.
