A multi-agent AI improvement system where three specialized agents Generator, Critic, and Improver work in an iterative loop to progressively enhance any text-based task. Features convergence detection, pattern tracking, and support for 20+ LLM providers including Ollama, Groq, DeepSeek, OpenAI, and Anthropic.
- What This System Does
- Why It Exists — The Problem It Solves
- Who Benefits From This System
- How It Works — The Core Loop
- Real-World Use Cases
- Installation
- Quick Start
- CLI Reference
- Provider Setup
- Model Discovery
- Architecture — For Developers
- How Each Iteration Works
- Convergence and Early Stopping
- Output Files
- Extending the System — For Developers
- Environment Variables
- Running Tests
- Examples
- FAQ
This system solves a fundamental problem: a single AI response is rarely the best possible response.
When you ask an AI to write something, generate code, or solve a problem, it gives you one answer. That answer might be good, but it's not optimal. It might have gaps, weaknesses, or missed opportunities.
This system takes that first answer and makes it better — automatically, repeatedly, and intelligently.
Here's what happens when you run it:
- You give it a task (e.g., "Write a guide about Python decorators")
- The Generator Agent creates an initial response
- The Critic Agent reviews it like an expert editor — finding weaknesses, scoring it on 6 dimensions, and listing specific improvements
- The Improver Agent analyzes all the feedback, identifies patterns, and creates a strategic improvement plan
- The Generator creates an improved version based on all that feedback
- The Critic reviews the new version
- This cycle repeats until the output reaches the quality target you set
The result: an output that has been reviewed, critiqued, and improved by multiple AI experts — producing something significantly better than any single AI pass could achieve.
When you use a single AI model, you face these limitations:
| Problem | Without CriticLoopAI | With CriticLoopAI |
|---|---|---|
| Single perspective | One AI's view, with its biases and blind spots | Multiple AI perspectives — Generator, Critic, Improver each bring different strengths |
| No quality gate | You get what you get, no verification | Every output is scored, critiqued, and verified before delivery |
| No iteration | One-shot generation, manual re-prompting | Automatic iterative improvement with smart stopping |
| No pattern tracking | You forget what worked and what didn't | Full history of every iteration, score, critique, and improvement tracked |
| Inconsistent quality | Output varies wildly between runs | Quality converges to a consistent high standard through repeated refinement |
| Manual feedback loop | You have to manually review and re-prompt | The AI reviews and improves itself — you just watch the scores climb |
In short: This system turns AI from a "one-shot tool" into a self-improving quality machine.
If you use AI for writing, research, content creation, or any text-based task, this system gives you:
1. Better Results, Automatically
- Instead of getting one AI response and hoping it's good, you get a response that has been iteratively improved 5+ times
- Each iteration catches issues the previous one missed
- The final output is scored and verified — you know exactly how good it is
2. No Manual Re-Prompting
- Instead of copying the AI's response, finding issues, and re-prompting ("make it more detailed", "add examples", "fix the structure"), the system does this automatically
- The Critic identifies exactly what needs to change
- The Generator implements those changes precisely
3. Transparency
- You can see every iteration — what was generated, what the Critic found, what improved, and what didn't
- The scoring system (0.0 to 1.0) gives you a concrete quality metric
- You know exactly when to stop — when the score hits your target or when improvements plateau
4. Works With Any AI Provider
- Use Ollama (local, free), Groq (free tier), DeepSeek (cheap), OpenAI, Anthropic, or any provider
- No vendor lock-in — switch providers with one flag
- Local providers mean your data never leaves your machine
5. Saves Time on Complex Tasks
- For tasks that need high quality (blog posts, documentation, code reviews, analysis), the system produces polished output without manual refinement
- Set it to run and come back to a finished, scored result
Example scenarios where end users benefit:
| Task | What the system does |
|---|---|
| Writing a blog post | Generator writes draft -> Critic finds weak sections -> Improver plans fixes -> Generator rewrites -> Final: polished post |
| Code review | Generator reviews code -> Critic finds security issues -> Improver prioritizes fixes -> Final: comprehensive review |
| Technical documentation | Generator drafts docs -> Critic finds gaps -> Improver plans additions -> Final: complete documentation |
| Marketing copy | Generator writes ad copy -> Critic evaluates persuasion -> Improver refines messaging -> Final: high-converting copy |
| Research summary | Generator summarizes paper -> Critic checks accuracy -> Improver fills gaps -> Final: accurate summary |
If you're building AI-powered applications, this system provides:
1. A Production-Ready Multi-Agent Framework
- Clean, modular Python architecture
BaseAgentclass handles all LLM communication, retries, and error handling- Add new agent types by extending
BaseAgent— implementget_system_prompt()and you're done - Async throughout — built on
httpxfor non-blocking API calls
2. Provider-Agnostic Design
- Any OpenAI-compatible API works out of the box — Ollama, LM Studio, vLLM, Groq, DeepSeek, Fireworks, Together, OpenRouter, and 15+ more
- Built-in Anthropic support (separate API format handled internally)
- Model discovery via
/v1/modelsendpoint — know exactly what's available before you run - 20+ provider presets with verified URLs and signup links
3. Intelligent Loop Management
- Convergence detection — stops when quality plateaus
- Early stopping — no wasted iterations when improvements stall
- Diminishing returns detection — the Improver predicts when further iterations won't help
- Configurable patience, thresholds, and targets
4. Pattern Tracking and Analysis
KnowledgeStoretracks every iteration's output, critique, improvements, and score- Extracts recurring issues from critique text automatically
- Calculates improvement trends, consistency, and effectiveness
- Persists full session history to JSON for analysis
5. Easy Integration
- Use as a CLI tool:
python main.py "your task" - Import as a library:
from core.orchestrator import LoopOrchestrator - Configuration via dataclasses, environment variables, or CLI flags
- All results saved as JSON — easy to parse, analyze, or display
6. Extensible Architecture
- Add new agents: extend
BaseAgent, implement system prompt - Add new providers: add API format handler in
base_agent.py - Add new evaluation criteria: modify
CriticAgentprompts - Add new stopping conditions: modify
LoopConfigandorchestrator.py
Developer-friendly patterns used:
- Dataclass-based configuration (typed, validated, composable)
- Abstract base class for agents (clean inheritance)
- Async/await throughout (non-blocking, scalable)
- JSON-structured LLM outputs (parseable, fallback-handled)
- Session-based storage (isolated, timestamped, auditable)
You give a task
|
v
+------------------+
| GENERATOR | Creates initial output
| (Creativity: 0.8)|
+------------------+
|
v
+------------------+
| CRITIC | Evaluates on 6 criteria
| (Precision: 0.3) | Returns: critique + improvements + score
+------------------+
|
v
+------------------+
| IMPROVER | Analyzes patterns across all iterations
| (Balance: 0.5) | Creates strategic improvement plan
+------------------+
|
v
+------------------+
| GENERATOR | Creates improved version
+------------------+
|
v
Score >= target? ----YES----> DONE: Final polished output
|
NO
|
v
No improvement for 2 iterations? ----YES----> DONE: Best output so far
|
NO
|
v
Back to CRITIC (next iteration)
The Critic evaluates on 6 dimensions:
| Dimension | What it measures |
|---|---|
| Accuracy | Is the information correct and factual? |
| Completeness | Does it fully address the task? |
| Clarity | Is it well-organized and easy to understand? |
| Depth | Does it provide sufficient detail? |
| Creativity | Does it offer unique insights or approaches? |
| Usability | Is it practical and actionable? |
Each dimension is scored 0.0-1.0. The overall score is a weighted combination. This means you get a detailed quality breakdown, not just a single number.
# Write a professional blog post
python main.py "Write a 1500-word blog post about the future of AI in healthcare" \
--provider groq --api-key gsk_xxx \
--model llama-4-maverick \
--max-iterations 6 --target-score 0.93What happens: Generator writes draft -> Critic finds weak arguments -> Improver plans evidence additions -> Generator rewrites with evidence -> Critic confirms improvement -> Final: publish-ready post
# Generate production-quality code
python main.py "Write a Python function that implements rate limiting with sliding window algorithm" \
--requirements "Include type hints" \
--requirements "Add docstring" \
--requirements "Handle edge cases" \
--provider deepseek --api-key sk-xxx --model deepseek-chatWhat happens: Generator writes code -> Critic finds missing edge cases -> Improver prioritizes fixes -> Generator adds handling -> Final: production-ready code
# Create a comprehensive literature review section
python main.py "Write a literature review section on transformer architectures for NLP" \
--requirements "Cite key papers" \
--requirements "Compare approaches" \
--requirements "Identify research gaps" \
--provider openai --api-key sk-xxx --model gpt-5.6What happens: Generator drafts review -> Critic finds missing citations -> Improver plans additions -> Generator adds citations and comparisons -> Final: thorough review
# Create marketing materials
python main.py "Write a product launch email for a new SaaS tool that helps teams collaborate remotely" \
--requirements "Professional tone" \
--requirements "Include social proof" \
--requirements "Strong CTA" \
--max-iterations 4What happens: Generator writes email -> Critic evaluates persuasion elements -> Improver refines messaging -> Final: high-converting email
Requirements: Python 3.9+
git clone https://github.com/v2Talal/CriticLoopAI.git
cd CriticLoopAI
pip install -r requirements.txtDependencies:
httpx(>=0.27.0) — async HTTP client for API callspytest(>=7.0.0) — testing frameworkpytest-asyncio(>=0.21.0) — async test support
No API keys needed for local providers (Ollama, LM Studio, vLLM).
# Install Ollama: https://ollama.com
ollama pull llama-4-maverick
python main.py "Write a guide about Python decorators"The system auto-detects Ollama, lists your models, and lets you pick one.
# Get free API key: https://console.groq.com
python main.py "Write a guide about Docker" \
--provider groq --api-key gsk_your_key_here \
--model llama-4-maverickpython main.py "Your task" \
--base-url https://your-provider.com/v1 \
--api-key your_key --model model-namepython main.py [task] [options]
| Flag | Short | Default | Description |
|---|---|---|---|
--requirements |
-r |
— | Add requirements. Repeatable: -r "use examples" -r "be concise" |
--initial-output |
-i |
— | Start from your own output (skip initial generation) |
--target-score |
-t |
0.95 | Quality target (0.0-1.0). Loop stops when reached. |
--max-iterations |
-m |
5 | Maximum improvement iterations |
--provider |
-p |
custom |
Provider: ollama, groq, openai, deepseek, anthropic, etc. |
--model |
— | — | Model name. Auto-detected for local providers. |
--api-key |
-k |
not-needed |
API key (not needed for local) |
--base-url |
-b |
— | Custom API endpoint URL |
--output-dir |
-o |
storage |
Where to save results |
--quiet |
-q |
off | Minimal output |
--discover |
-d |
off | Interactively select a model before running |
--list-models |
-M |
off | List available models (no loop) |
--list-presets |
-l |
off | List all provider presets |
Run on your machine. No API key. No internet needed.
| Provider | Preset | URL | Install |
|---|---|---|---|
| Ollama | ollama |
localhost:11434/v1 |
ollama.com |
| LM Studio | lmstudio |
localhost:1234/v1 |
lmstudio.ai |
| vLLM | vllm |
localhost:8000/v1 |
github.com/vllm-project/vllm |
| Text Gen WebUI | textgen |
localhost:5000/v1 |
oobabooga/text-generation-webui |
| KoboldCpp | kobold |
localhost:5001/v1 |
LostRuins/koboldcpp |
| Provider | Preset | URL | Details |
|---|---|---|---|
| Groq | groq |
api.groq.com/openai/v1 |
Free tier, rate-limited. Very fast. |
| Cerebras | cerebras |
api.cerebras.ai/v1 |
$5 free credits on signup |
| SambaNova | sambanova |
api.sambanova.ai/v1 |
Free tier available |
| OpenRouter | openrouter |
openrouter.ai/api/v1 |
25+ free models. 5.5% fee on paid. |
| Provider | Preset | URL |
|---|---|---|
| DeepSeek | deepseek |
api.deepseek.com/v1 |
| OpenAI | openai |
api.openai.com/v1 |
| Fireworks AI | fireworks |
api.fireworks.ai/inference/v1 |
| Together AI | together |
api.together.ai/v1 |
| OpenCode | opencode |
api.opencode.ai/v1 |
| Anthropic | anthropic |
api.anthropic.com/v1 |
| Mistral AI | mistral |
api.mistral.ai/v1 |
| Cohere | cohere |
api.cohere.ai/compatibility/v1 |
| Google AI | google |
generativelanguage.googleapis.com/v1beta/openai/ |
# Using --api-key directly
python main.py "Your task" --base-url https://your-api.com/v1 --api-key key --model name
# Using environment variable (avoids key in command history)
$env:API_KEY="your-key-here" # PowerShell
python main.py "Your task" --base-url https://your-api.com/v1 --model name
# Linux/macOS
export API_KEY="your-key-here"
python main.py "Your task" --base-url https://your-api.com/v1 --model nameBefore running, discover what models are available:
# List models from any provider
python main.py --list-models --provider ollama
python main.py --list-models --provider groq --api-key gsk_xxx
# Interactive selection
python main.py "Your task" --discover --provider ollama
# Auto-discovery (no --model needed with local providers)
python main.py "Your task" --provider ollama
# List all provider presets
python main.py --list-presetsCriticLoopAI/
|-- main.py # CLI entry point
|-- config.py # Configuration dataclasses
|-- requirements.txt # Dependencies
|-- test_system.py # Test suite
|
|-- agents/ # AI Agent implementations
| |-- base_agent.py # Base class: LLM communication, retry, history
| |-- generator.py # Generator: creates and improves outputs
| |-- critic.py # Critic: evaluates, scores, provides feedback
| |-- improver.py # Improver: synthesizes feedback, plans strategy
|
|-- core/ # Core system logic
| |-- orchestrator.py # Loop controller: manages iterations
| |-- knowledge.py # Pattern tracking and analysis
|
|-- utils/ # Utilities
| |-- logger.py # Colored console output, file saving
| |-- model_discovery.py # Fetches models from any provider API
|
|-- storage/ # Auto-created at runtime (session data)
All agents inherit from BaseAgent:
BaseAgent (abstract)
|-- GeneratorAgent (temperature: 0.8, creative)
|-- CriticAgent (temperature: 0.3, precise)
|-- ImproverAgent (temperature: 0.5, balanced)
BaseAgent provides:
- LLM communication (OpenAI-compatible, Anthropic, Groq formats)
- Retry logic with exponential backoff (1s, 2s, 4s)
- Conversation history management (last 5 exchanges)
- Configurable timeout (default: 60s)
To add a new agent:
- Create a new file in
agents/ - Extend
BaseAgent - Implement
get_system_prompt()returning your agent's role - Add methods for your agent's specific tasks
- Wire it into
orchestrator.py
Each agent's system prompt is designed for a specific role:
| Agent | Prompt Focus | Temperature |
|---|---|---|
| Generator | "Create comprehensive, well-structured outputs. Incorporate feedback precisely. Build upon previous iterations." | 0.8 |
| Critic | "Be thorough but constructive. Score fairly. Identify strengths AND weaknesses. Suggest concrete improvements." | 0.3 |
| Improver | "Prioritize by impact. Identify root causes, not symptoms. Create actionable plans. Track patterns across iterations." | 0.5 |
LoopOrchestrator in core/orchestrator.py manages the entire cycle:
from core.orchestrator import LoopOrchestrator
from config import SystemConfig
config = SystemConfig()
config.llm.base_url = "http://localhost:11434/v1"
config.llm.model = "llama-4-maverick"
orchestrator = LoopOrchestrator(config)
result = await orchestrator.run(
task="Write a guide about Python",
requirements=["include examples", "be concise"]
)
# result contains: task, final_output, best_score, iteration_history, etc.Available methods:
run(task, requirements, initial_output)— Full improvement looprun_with_specific_goal(task, target_score, max_iterations)— Temporary override of loop config
KnowledgeStore in core/knowledge.py tracks patterns across iterations:
from core.knowledge import KnowledgeStore
store = KnowledgeStore("storage")
store.add_iteration({"iteration": 1, "output": "...", "score": 0.7})
summary = store.get_summary()
# Returns: total_iterations, current_score, best_score, average_score,
# improvement_trend, recurring_issues, successful_strategies
trend = store.get_improvement_trend()
# Returns: trend ("improving"/"stable"/"declining"), average_improvement, etc.
effectiveness = store.analyze_effectiveness()
# Returns: best_improvement_iteration, total_improvement, consistencyFive dataclasses in config.py:
from config import SystemConfig
# From environment variables
config = SystemConfig.from_env()
# Or build manually
config = SystemConfig()
config.llm.provider = "openai_compatible"
config.llm.model = "llama-4-maverick"
config.llm.base_url = "http://localhost:11434/v1"
config.llm.api_key = "not-needed"
config.loop.max_iterations = 5
config.loop.convergence_score = 0.95
config.loop.early_stop_patience = 2| Config Class | Controls |
|---|---|
LLMConfig |
Provider, model, API key, base URL, temperature, max tokens |
AgentConfig |
Agent name, temperature, retry attempts, timeout |
LoopConfig |
Max iterations, convergence score, early stop patience, thresholds |
StorageConfig |
Storage directory, save settings, export format |
SystemConfig |
Wraps all configs + verbose flag |
Console output:
- Color-coded: Cyan (info), Green (success), Yellow (warning), Red (error), Magenta (header)
- Icons:
✓(success),⚠(warning),✗(error),▶(header)
File storage per session:
storage/YYYYMMDD_HHMMSS/
iteration_1.json # Output + critique + improvements + score
iteration_2.json # Same for iteration 2
...
final_result.json # Complete result with all history
summary.json # Session summary
knowledge_store.json # Pattern analysis
Iteration 1:
- Generator creates initial output for the task
- Critic evaluates -> returns critique + improvements + score
- Output and score saved
Iteration 2+:
- Improver analyzes all previous data:
- Recurring issues from critique text
- Score trend (improving/stable/declining)
- Convergence proximity
- Generator creates improved output:
- Keeps what works
- Fixes all issues from critique
- Implements all improvements
- Must not introduce new issues
- Critic evaluates new output:
- Receives previous critiques for progress tracking
- Returns new critique, improvements, score
- System updates best score/output if improved
- Checks stopping conditions
| Condition | Default | What happens |
|---|---|---|
| Score >= target | 0.95 | Loop stops. Final output generated. |
| No improvement | 2 iterations | Loop stops. Best output used. |
| Diminishing returns | improvement < 0.05 | Warning printed. Loop continues. |
| Max iterations | 5 | Loop stops. Safety limit. |
The Improver also predicts diminishing returns after each iteration — warning you before the loop would naturally stop.
Each session produces:
| File | Purpose | Open with |
|---|---|---|
final_output.md |
Your final result in readable Markdown | Any text editor, browser |
session_summary.md |
Score progression and improvements overview | Any text editor, browser |
final_result.json |
Raw data for developers | Code editor, API integrations |
iteration_N.json |
Detailed data per iteration | Code editor |
knowledge_store.json |
Pattern tracking across sessions | Code editor |
summary.json |
Session metadata | Code editor |
Example session output:
============================================================
SESSION COMPLETE
============================================================
Task: Write a Python sorting algorithm...
Final Score: 0.82/1.00
Iterations: 2
Improvement: +0.07 (0.75 → 0.82)
Files saved:
final_output.md ← Open this to read your result
session_summary.md ← Overview of the improvement process
final_result.json ← Raw data for developers
============================================================
session_summary.md example:
# Session Summary
| Field | Value |
|-------|-------|
| Task | Write a Python sorting algorithm |
| Final Score | 0.82/1.00 |
| Total Iterations | 2 |
| Convergence | No |
## Score Progression
| Iteration | Score | Change |
|-----------|-------|--------|
| 1 | 0.75 | --- |
| 2 | 0.82 | +0.07 |
## Key Improvements Applied
### Iteration 2 (Score: 0.82)
- Focus on one primary sorting algorithm
- Add a brief summary section
- Simplify explanations for complex parts# agents/summarizer.py
from agents.base_agent import BaseAgent
from config import AgentConfig, LLMConfig
class SummarizerAgent(BaseAgent):
def __init__(self, llm_config: LLMConfig):
config = AgentConfig(name="Summarizer", temperature=0.4)
super().__init__(config, llm_config)
def get_system_prompt(self) -> str:
return "You are an expert summarizer. Be concise and accurate."
async def summarize(self, text: str) -> str:
return await self.generate(f"Summarize this:\n{text}")Then add it to orchestrator.py.
Add the API format handler in base_agent.py:
async def _call_my_provider(self, messages, temperature):
async with httpx.AsyncClient() as client:
response = await client.post(
f"{self.llm_config.base_url}/chat/completions",
headers={"Authorization": f"Bearer {self.llm_config.api_key}"},
json={"model": self.llm_config.model, "messages": messages, "temperature": temperature},
timeout=self.config.timeout_seconds
)
return response.json()["choices"][0]["message"]["content"]Then add the routing in call_llm().
Modify the Critic's system prompt in critic.py to include your domain-specific criteria (e.g., SEO optimization, accessibility, performance).
import asyncio
from config import SystemConfig
from core.orchestrator import LoopOrchestrator
async def improve_my_output():
config = SystemConfig.from_env()
orchestrator = LoopOrchestrator(config)
result = await orchestrator.run("Your task here")
print(f"Best score: {result['best_score']}")
print(f"Output: {result['final_output']}")
asyncio.run(improve_my_output())| Variable | Default | Description |
|---|---|---|
AI_PROVIDER |
openai_compatible |
Provider type |
AI_MODEL |
llama-4-maverick |
Model name |
API_KEY |
not-needed |
API key |
AI_BASE_URL |
http://localhost:11434/v1 |
API endpoint |
AI_TEMPERATURE |
0.7 |
Generation temperature |
MAX_ITERATIONS |
5 |
Max iterations |
VERBOSE |
true |
Verbose output |
OPENAI_API_KEY |
— | OpenAI key (auto-detected) |
ANTHROPIC_API_KEY |
— | Anthropic key (auto-detected) |
GROQ_API_KEY |
— | Groq key (auto-detected) |
python test_system.pyVerifies: all imports, default configuration, knowledge store operations.
python main.py "Write a guide about Python decorators" --provider ollamapython main.py "Create a REST API" \
-r "Use RESTful conventions" -r "Include auth" -r "Support pagination" \
--provider deepseek --api-key sk-xxxpython main.py "Improve this code" \
-i "def add(a, b): return a + b" \
--provider groq --api-key gsk_xxx --model llama-4-maverickpython main.py "Write marketing copy" \
--provider openai --api-key sk-xxx --model gpt-5.6 \
--max-iterations 8 --target-score 0.98python main.py "Your task" --provider ollama -qDo I need an API key? No, not for local providers (Ollama, LM Studio, vLLM). For cloud providers, yes — some have free tiers (Groq, OpenRouter).
Which provider should I use?
Depends on your needs. Local providers (Ollama, LM Studio) are free and private. Cloud providers vary in speed, cost, and model quality. Use --list-presets to see available options with their details.
What happens if an API call fails? Retries 3 times with exponential backoff. If all fail, the error is raised.
Can I use multiple providers? Not in one run, but switch between runs to compare.
How do I stop early? Press Ctrl+C. The system saves progress up to that point.
When does the loop stop? Score reaches target (0.95), no improvement for 2 iterations, or max iterations (5).
What tasks can I use this for? Any text-based task: writing, code, analysis, translation, documentation, marketing, research.
Can I use this as a Python library?
Yes. Import LoopOrchestrator and call run() programmatically.
How do I extend the system?
Add new agents by extending BaseAgent. Add providers by adding API handlers. Modify prompts for domain-specific evaluation.