LoopLlama

Local-first AI chat with strict self-verification loops — powered by Ollama, grounded with free web retrieval, and designed to reduce hallucinations before answers reach the user.

Beta. LoopLlama prioritizes evidence-grounded factual sections and honest uncertainty over speed. Mixed prompts with strict settings can take 8–25 minutes on a consumer GPU.

Features

• Verify → revise loops — draft, adversarial verify, revise, repeat (configurable)
• Confirmation passes — consecutive code-law checks before release
• Deterministic anti-hallucination gates — structural audits, evidence grounding, requirement validators (authoritative over LLM self-scores)
• Smart retrieval — DuckDuckGo + Wikipedia by default; optional Brave Search API
• Mixed prompts — fantasy + factual + public-opinion sections in one request, clearly labeled
• Multi-turn chat — follow-ups use conversation history; creative story expansions skip factual web search
• Expanded-safe reasoning summary — optional audit metadata (loop counts, stop reason, sources) without chain-of-thought
• No paid LLM APIs — runs entirely on your Ollama instance

How it works

flowchart TD
    A[User message] --> B[Retrieval policy]
    B -->|factual / mixed| C[Web search]
    B -->|creative follow-up| D[Skip search]
    C --> E[Draft answer]
    D --> E
    E --> F[Improvement loops]
    F --> G[Confirmation loops]
    G --> H[Final confirmation]
    H --> I[Gatekeeper]
    I --> J[Code-law + sanitize]
    J --> K[Response]

Loading

Each verify pass returns structured JSON (quality_score, issues_found, revised_answer). Code-law gates (fake citations, ungrounded proper nouns, missing sections, Mars/AI contamination) can block or rewrite answers even when the verifier is optimistic.

Requirements

• Python 3.11+
• Ollama with a chat model pulled (e.g. ollama pull llama3.1:8b)
• ~8 GB VRAM recommended for llama3.1:8b
• Network access for DuckDuckGo / Wikipedia retrieval (factual prompts)

Quick start

git clone https://github.com/YOUR_USERNAME/LoopLlama.git
cd LoopLlama

# Backend
cd backend
python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -r requirements.txt

# Config
cp ../.env.example ../.env
# Edit ../.env if Ollama is not on localhost:11434

# Run (serves API + web UI on port 8010)
uvicorn app.main:app --host 0.0.0.0 --port 8010 --reload

Open http://localhost:8010

Preload the model (optional, faster first request):

ollama run llama3.1:8b
# Ctrl+D to exit — model stays loaded when OLLAMA_KEEP_ALIVE is set

Configuration

Copy .env.example to .env at the project root.

Variable	Default	Description
DRAFT_MODEL	llama3.1:8b	Ollama model for drafting
VERIFY_MODEL	(same as draft)	Model for verify/gatekeeper passes
OLLAMA_CHAT_URL	http://127.0.0.1:11434/api/chat	Ollama chat endpoint
OLLAMA_KEEP_ALIVE	30m	Keep model loaded between loop calls
MAX_SELF_CHECK_LOOPS	5	Improvement verify cycles
CONFIRMATION_LOOPS	5	Consecutive confirmation passes required
FINAL_CONFIRMATION_LOOPS	5	Final confirmation passes required
MAX_CONFIRMATION_ATTEMPTS	30	Max tries per confirmation block
MAX_FINAL_CONFIRMATION_ATTEMPTS	30	Max tries for final confirmation
MIN_QUALITY_SCORE	0.94	Improvement gate threshold
MIN_CONFIRMATION_QUALITY_SCORE	0.97	Confirmation quality hint
MIN_FINAL_CONFIRMATION_QUALITY_SCORE	0.99	Final confirmation quality hint
MIN_CONFIDENCE_SCORE	0.90	Confidence threshold
STRICT_FINAL_REQUIRES_CONFIRMATION	true	Require confirmation before confident release
STAGNATION_ROUNDS_LIMIT	4	Stop improvement if quality plateaus
MIN_QUALITY_IMPROVEMENT_DELTA	0.02	Minimum delta to count as improvement
DDG_ENABLED	true	Enable DuckDuckGo retrieval
DDG_TIMEOUT_SECONDS	8	DuckDuckGo timeout
REQUEST_TIMEOUT_SECONDS	180	Ollama HTTP timeout per call
BRAVE_SEARCH_API_KEY	(empty)	Optional Brave Search API key

Strict vs balanced profiles

Profile	Loops	Typical time (llama3.1:8b)
Balanced (default in .env.example)	5 + 5 + 5	~8–12 min mixed prompt
Maximum strict	10 + 10 + 10	~15–25 min mixed prompt

For maximum strictness, set MAX_SELF_CHECK_LOOPS, CONFIRMATION_LOOPS, and FINAL_CONFIRMATION_LOOPS to 10 and raise attempt limits to 80.

Remote Ollama

If Ollama runs on another machine on your LAN:

OLLAMA_CHAT_URL=http://192.168.1.100:11434/api/chat

Retrieval policy

Prompt type	Web search	Example
Factual	Yes	"Explain Mars habitability with sources"
Public belief	Yes	"What do people think about AI safety in 2025?"
Mixed (fantasy + facts)	Yes	Dragons on Mars + Mars science + AI opinions
Creative / fantasy only	No	"Write a fantasy story about dragons"
Creative follow-up	No	"Expand on the storyline" (after a fantasy turn)
Opinion (no facts requested)	No	"What's your favorite sci-fi trope?"

API

POST /api/chat

{
  "message": "Your question",
  "show_reasoning_summary": true,
  "history": [
    { "role": "user", "content": "..." },
    { "role": "assistant", "content": "..." }
  ]
}

Response:

{
  "answer": "...",
  "reasoning_summary": {
    "loops_run": 15,
    "stop_reason": "final_confirmation_complete",
    "sources": ["https://..."],
    "final_quality_score": 0.83,
    "final_confidence": 0.92
  }
}

Example prompts

Mixed (stress test):

Write a fantasy story about dragons ruling Mars, then separately explain what
scientists currently believe about Mars habitability, and finally summarize what
most people think about AI safety in 2025. Label each part and cite web sources
only for the factual/public-belief sections.

Factual:

Explain how DNS caching works and include common failure modes.

Public opinion:

What do people think about remote work productivity in 2025?

Creative follow-up (after a fantasy section):

Can you expand on the storyline?

Project structure

LoopLlama/
├── backend/
│   ├── app/
│   │   ├── api/           # FastAPI routes
│   │   ├── core/          # Config, schemas
│   │   └── services/      # Loop engine, retrieval, prompts
│   ├── scripts/           # Smoke check, eval loops (optional)
│   └── tests/
├── frontend/              # Vanilla HTML/CSS/JS chat UI
├── .env.example
└── LICENSE

Tests

cd backend
source .venv/bin/activate
pytest -q -m "not integration"

Smoke check (retrieval + Ollama reachability):

python scripts/smoke_check.py

Performance notes

• First request after idle may add 30–60s for model load (mitigated by OLLAMA_KEEP_ALIVE)
• Strict loops mean many Ollama calls; GPU utilization spikes during verify passes
• Creative follow-ups use fewer loops (~7) and skip web search
• Conversation history is kept in the browser only (cleared on refresh)

Limitations

• Beta quality — reduces many hallucination patterns but does not guarantee correctness
• Local model limits — llama3.1:8b verifier scores are noisy; code-law gates are the real safety net
• Retrieval variance — DuckDuckGo/Wikipedia availability affects factual sections
• Speed — strict verification trades latency for grounding
• No persistence — no database; chat history is session-only in the UI

Optional maintainer scripts

Script	Purpose
scripts/smoke_check.py	Quick health check
scripts/testing_loop.py	Repeated test + smoke cycles
scripts/strict_eval_loop.py	Long-running quality eval
scripts/overnight_safe_repair.py	Unattended test/repair with git rollback

See CONTRIBUTING.md for development guidelines.

License

MIT