An AI agent that writes code, runs tests, spots failures, fixes bugs, and retries until everything passes.
Ever had an AI write code that looks right but fails every test? FixLoop closes that loop:
- You describe the problem (like "write a Fibonacci function")
- FixLoop asks an LLM to write the code
- It runs the tests automatically
- If something fails, it feeds the error back to the LLM and says "fix it"
- It repeats until every test passes
You don't sit there copying error messages into a chat window. The machine does the grinding.
# 1. Clone the repo
git clone <repo-url> && cd FixLoop
# 2. Install dependencies
pip install -r requirements.txt
# 3. Set a free API key
export GROQ_API_KEY="gsk_..." # get one at https://console.groq.com/keys
# 4. Run your first challenge
python3 main.py fibonacciWhat you'll see:
============================================================
Challenge: fibonacci
============================================================
[1/3] Generating initial code...
Code generated (122 chars).
[2/3] Running tests (iteration 1)...
All tests PASSED after 1 iteration(s)!
Final solution (solution.py):
----------------------------------------
def fib(n):
if n <= 0:
return 0
elif n == 1:
return 1
else:
return fib(n-1) + fib(n-2)
----------------------------------------
Solution saved to solutions/fibonacci_solution.py
The AI wrote the code, the tests ran, everything passed. That's it.
# Install dependencies
pip install -r requirements.txt
# Set your API key (pick one provider)
export GROQ_API_KEY="gsk_your_key_here" # free, fast
# export GEMINI_API_KEY="your_key_here" # free, Google
# export OPENAI_API_KEY="sk-your_key_here" # paid, best qualityNo config files to edit. No accounts to create beyond getting a free API key.
# See what's available
python3 main.py --list
# Run one
python3 main.py fibonacciThat's it. The app does the rest. You'll see live output as it generates code, runs tests, and (if needed) fixes bugs.
FixLoop prints every step so you know exactly what's happening:
============================================================
Challenge: fibonacci
============================================================
[1/3] Generating initial code... ← AI is writing the code
Code generated (122 chars). ← Success! 122 chars written
[2/3] Running tests (iteration 1)... ← pytest is running
All tests PASSED after 1 iteration(s)! ← All green!
Final solution (solution.py): ← Here's what it wrote
----------------------------------------
def fib(n):
if n <= 0:
return 0
elif n == 1:
return 1
else:
return fib(n-1) + fib(n-2)
----------------------------------------
Solution saved to solutions/fibonacci_solution.py ← saved!
FixLoop doesn't give up — it debug loops:
[2/3] Running tests (iteration 1)...
Tests FAILED (1/5) ← pytest found failures
FAILED test_solution.py::test_large ← specific test that broke
[3/3] Debugging (iteration 1)... ← AI is fixing the code
Code updated (355 chars). ← Rewrote 355 chars
[2/3] Running tests (iteration 2)... ← retesting
All tests PASSED after 2 iteration(s)! ← fixed!
The AI sees the error message and the code, figures out what went wrong, and rewrites the buggy parts. If it still fails, it tries again up to --max-iterations times.
Start simple, then level up:
python3 main.py fibonacci # easy - warm up
python3 main.py valid_parentheses # medium - has edge cases
python3 main.py valid_sudoku # hard - complex logic# Same challenge, different AI
python3 main.py fibonacci --provider groq # fast, free
python3 main.py fibonacci --provider gemini # free, Google
python3 main.py fibonacci --provider openai # paid, powerfulJust change the --provider flag. Your API key is read from the matching environment variable.
# Override the default model for any provider
python3 main.py fibonacci --provider openai --model gpt-4o
python3 main.py fibonacci --provider groq --model llama-3.1-8b-instant
python3 main.py fibonacci --provider ollama --model codellama:7bIf you don't specify --model, FixLoop picks a sensible default for each provider.
# Aggressive: give up fast if it's not working
python3 main.py two_sum --max-iterations 2
# Patient: let it keep trying hard problems
python3 main.py valid_sudoku --max-iterations 10Default is 5. Increase for tough problems, decrease for simple ones.
Every passing solution is saved automatically:
ls solutions/
# fibonacci_solution.py prime_checker_solution.py ...These are the final, tested, working solutions — ready to copy into your project.
You don't have to use the CLI. Import FixLoop directly:
from fixloop.llm import LLMClient
from fixloop.runner import FixLoop
from fixloop.challenges import Challenge
llm = LLMClient(model="llama-3.3-70b-versatile", provider="groq")
loop = FixLoop(llm, max_iterations=5)
my_challenge = Challenge(
name="my_problem",
description="Write a function `add(a, b)` that returns a + b.",
tests="""from solution import add
def test_add():
assert add(2, 3) == 5
assert add(-1, 1) == 0
""",
)
success = loop.solve(my_challenge)Here's a typical session from start to finish:
# 1. Install
pip install -r requirements.txt
# 2. Set API key
export GROQ_API_KEY="gsk_abc123"
# 3. List challenges
python3 main.py --list
# 4. Start with an easy one
python3 main.py fibonacci
# 5. Check the saved solution
cat solutions/fibonacci_solution.py
# 6. Try something harder
python3 main.py valid_sudoku --max-iterations 8
# 7. Try a different AI
python3 main.py valid_parentheses --provider gemini
# 8. Add your own challenge (edit challenges.py, then:)
python3 main.py my_challenge- Python 3.10 or higher (check with
python3 --version) - pip (usually comes with Python)
- An API key from one of the supported providers (most have a free tier)
No GPU, no Docker, no cloud account required.
Run python3 main.py --list to see all challenges:
| Challenge | The AI needs to... |
|---|---|
fibonacci |
Return the nth Fibonacci number (fib(10) == 55) |
prime_checker |
Check if a number is prime |
valid_parentheses |
Validate bracket pairing — ({[]}) is valid, ([)] is not |
two_sum |
Find two indices in a list whose values add up to a target |
valid_sudoku |
Validate a 9x9 Sudoku board (check rows, columns, and 3x3 boxes) |
# See what challenges are available
python3 main.py --list
# Solve a challenge
python3 main.py fibonacci
# Pick which AI to use
python3 main.py valid_sudoku --provider groq
# Use a specific model
python3 main.py two_sum --provider openai --model gpt-4o
# Give the AI more chances to fix bugs
python3 main.py fibonacci --max-iterations 10
# Fewer chances (fail faster)
python3 main.py prime_checker --max-iterations 2| Provider | Free? | How to start | Set this env var |
|---|---|---|---|
| Groq | ✅ Free tier (no credit card) | Grab an API key from their console | GROQ_API_KEY |
| Gemini | ✅ Free tier (no credit card) | Google's free AI Studio key | GEMINI_API_KEY |
| OpenAI | 💰 Paid | Requires a paid OpenAI account | OPENAI_API_KEY |
| Ollama | 🏠 Runs locally | Install Ollama, pull a model like llama3.1 |
(none needed) |
# Examples for each provider (only set ONE):
export GROQ_API_KEY="gsk_your_key_here"
export GEMINI_API_KEY="your_key_here"
export OPENAI_API_KEY="sk-your_key_here"
# Run with your chosen provider:
python3 main.py fibonacci --provider groqPro tip: Groq is the fastest free option. Ollama is great if you want everything offline. Gemini's free tier is generous but has rate limits.
Here's the full flow, step by step:
┌──────────────────────────────────────────────────┐
│ 1. GENERATE │
│ LLM writes solution.py based on description │
└──────────────────────┬───────────────────────────┘
↓
┌──────────────────────┴───────────────────────────┐
│ 2. TEST │
│ pytest runs against solution.py │
│ ┌───── PASS? ────┐ │
│ │ ✅ Yes │ ❌ No │
│ │ Save & exit │ Capture errors │
│ └────────────────┘ │
└──────────────────────┬───────────────────────────┘
↓ (if failed)
┌──────────────────────┴───────────────────────────┐
│ 3. FIX │
│ LLM sees the code + test errors │
│ Rewrites the code to fix the bugs │
└──────────────────────┬───────────────────────────┘
↓
Go back to step 2 (retry)
(up to --max-iterations times)
That's when FixLoop shines. Here's what happens when the code has bugs:
[1/3] Generating initial code...
Code generated (340 chars).
[2/3] Running tests (iteration 1)...
Tests FAILED (1/5)
FAILED test_solution.py::test_edge_case - AssertionError
[3/3] Debugging (iteration 1)...
Code updated (355 chars).
[2/3] Running tests (iteration 2)...
All tests PASSED after 2 iteration(s)!
The AI got it wrong → the test caught it → the AI fixed it → tests pass now. No human intervention needed.
FixLoop/
├── fixloop/ # The core agent
│ ├── __init__.py # Exports FixLoop class
│ ├── llm.py # Talks to OpenAI, Gemini, Groq, Ollama
│ ├── coder.py # Asks the LLM to write code
│ ├── tester.py # Runs pytest, parses failures
│ ├── debugger.py # Asks the LLM to fix bugs
│ ├── runner.py # The main loop (orchestrates everything)
│ ├── utils.py # Cleans up LLM output (strips markdown)
│ └── challenges.py # Challenge definitions (add yours here!)
├── main.py # CLI entry point
├── requirements.txt # Python dependencies
├── solutions/ # Successful solutions land here
└── README.md # This file
Each module is small — the longest is under 100 lines. Easy to read, easy to change.
Open fixloop/challenges.py and add a new Challenge to the CHALLENGES dict:
CHALLENGES["double"] = Challenge(
name="double",
description="""Write a function `double(n)` that returns twice the input.
- n is an integer
- double(3) should return 6
- double(-5) should return -10
Put the function in a file called solution.py.
""",
tests="""from solution import double
def test_positive():
assert double(3) == 6
def test_negative():
assert double(-5) == -10
def test_zero():
assert double(0) == 0
""",
)Then run it:
python3 main.py doubleTips for writing good challenges:
- The description is what the LLM sees. Be clear about function name, parameters, and expected behavior.
- The tests are standard pytest functions. Cover edge cases (empty input, zero, negatives, etc.).
- Start with a challenge you know the answer to, so you can verify the output.
You can tune how FixLoop behaves without touching code:
| Flag | Default | What it does |
|---|---|---|
--provider |
gemini |
Which LLM to use (openai, gemini, groq, ollama) |
--model |
(provider default) | Specific model name (e.g., gpt-4o, gemini-2.0-flash) |
--max-iterations |
5 |
How many times to retry before giving up |
Environment variables:
GROQ_API_KEY— for GroqGEMINI_API_KEYorGOOGLE_API_KEY— for GeminiOPENAI_API_KEY— for OpenAIOLLAMA_HOST— for Ollama (defaults tohttp://localhost:11434/v1)
- Start with Groq — it's free, fast, and the default model works well
- For hard problems, use
gpt-4oorclaude-3-opus— they handle complex logic better - Increase
--max-iterationsfor tougher challenges (the AI might need several attempts) - Write thorough tests — the more edge cases you cover, the more robust the final solution
- Be specific in the description — include the exact function signature, parameter types, and examples
Error: GROQ_API_KEY not set.
→ You need to set your API key as an environment variable before running:
export GROQ_API_KEY="gsk_your_key_here"Tests fail with ERROR collecting test_solution.py
→ The AI generated code with syntax errors. This happens sometimes. FixLoop should fix it in the next iteration. If it doesn't, try a more capable model.
429 ResourceExhausted (Gemini)
→ You've hit the free tier rate limit. Wait 60 seconds and try again, or switch to Groq.
Ollama is very slow
→ Make sure your model fits in your GPU memory. Try a smaller model like qwen2.5-coder:1.5b or llama3.2:3b.
The AI keeps generating the same broken code
→ The debugger prompt might not be giving it enough context. Check the debugger.py prompt — it includes the current code, test output, and error summary. If the model keeps making the same mistake, try a different provider.
PRs are welcome! Here's how to help:
- Add challenges — The more the merrier. Good challenges have clear descriptions and thorough tests.
- Support more LLMs — Add Anthropic Claude, Cohere, or any OpenAI-compatible provider.
- Improve the prompts — Small tweaks to
coder.pyanddebugger.pycan dramatically improve results. - CI/CD integration — Make FixLoop runnable as a GitHub Action or pre-commit hook.
To contribute:
# Fork the repo, then:
git clone <your-fork>
cd FixLoop
pip install -r requirements.txt
# Make your changes, then test:
python3 main.py fibonacci --provider groq
# Open a PR with a clear description of what you changed and whyFixLoop is built around four simple steps:
coder.pysends the problem description to an LLM with a system prompt asking for raw Python codetester.pywrites the tests to a file and runspytest -v --tb=short, capturing stdout/stderrdebugger.pysends the failing code + test output to the LLM with a "fix this" promptrunner.pyloops steps 2-3 until tests pass or the iteration limit is hit
The only "clever" part is utils.py — it strips markdown code fences from LLM output because models love wrapping code in ```python blocks.
Everything runs in a temporary directory that's cleaned up automatically. Successful solutions are copied to solutions/ before cleanup.
MIT. Do what you want.