*A lightweight, deterministic LLM agent framework built on llama.cpp with tool calling, RAG, persistent memory, and structured tracing."
Logician is a production-grade agent framework for building tool-using AI agents on top of llama.cpp. It provides:
- Deterministic tool execution (JSON or TOON format)
- Persistent memory with semantic search over conversations
- RAG integration for external knowledge
- Structured logging & tracing for debugging and monitoring
- Self-reflection loops for iterative improvement
- llama.cpp backend for local, privacy-preserving inference
# Clone and install
git clone https://github.com/lseman/logician.git
cd logician
pip install -e .
# Run a demo
python apps/runners/demo.py
# Or start a REPL
python apps/runners/repl_demo.py
# Or ingest a repo without the TUI
python apps/runners/repo_ingest.py /path/to/repo
# Or clone + ingest a git URL into .logician/repos/_checkouts
python apps/runners/repo_ingest.py https://github.com/org/repo.git --base-dir /path/to/workspacefrom agent import Agent
from agent.tools import ToolRegistry
# Register your tools
tool_registry = ToolRegistry()
# Create agent
agent = Agent(
model="llama3:8b",
tools=[tool_registry],
memory_path=".memory",
)
# Run with reflection
result = agent.run(
prompt="Analyze this time series and forecast next 30 days",
max_turns=5,
use_reflection=True,
)
print(result.answer)
print(result.trace_md) # Markdown timeline┌─────────────────────────────────────────────────────────────┐
│ Agent Core │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ Planning │→│ Acting │→│ Observing │ │
│ │ (think) │ │ (tools) │ │ (verify) │ │
│ └──────────────┘ └──────────────┘ └──────────────────┘ │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Backends │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ llama.cpp │ │ vLLM │ │ MCP Client │ │
│ │ (local) │ │ (GPU) │ │ (model context) │ │
│ └──────────────┘ └──────────────┘ └──────────────────┘ │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Memory & Storage │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ MessageDB │ │ DocumentDB │ │ RAG Vector Store │ │
│ │ (SQLite) │ │ (Chroma) │ │ (Chroma) │ │
│ └──────────────┘ └──────────────┘ └──────────────────┘ │
└─────────────────────────────────────────────────────────────┘
- Typed parameters for safe tool calling
- Dynamic tool loading via
add_tool(...)API - Dual format support: JSON or TOON (Token-Oriented Object Notation)
from agent.tools import ToolParameter, ToolRegistry
tool_registry.add_tool(
name="fetch_data",
description="Fetch time series data from CSV",
parameters={
"filepath": ToolParameter(type="string", required=True),
"date_column": ToolParameter(type="string", required=True),
},
)- SQLite storage for conversation history
- Semantic search via Chroma + SentenceTransformers
- Auto-summarization of long histories into SYSTEM prompts
- Separate Chroma collection for external documents
- Chunking & metadata with top-k retrieval
- Injected as SYSTEM context in prompts
from agent import DocumentDB
db = DocumentDB(collection_name="docs")
db.add_directory("./docs/")
# Documents appear as SYSTEM context in agent prompts- Second-pass critique of final answers
- Optional refinement or additional tool calls
- Configurable prompt and token budget
result = agent.run(
prompt="Debug this code",
use_reflection=True, # Enable self-reflection
reflection_prompt="Review your answer for accuracy and completeness",
)- Structured logging via
AGENT_LOG_LEVELandAGENT_LOG_JSON - Per-module loggers:
agent,agent.tools,agent.db,agent.rag - Trace output:
debug: structured JSON trace (events, timings, config)trace_md: markdown timeline for UI rendering
- Dual API support:
/v1/chat/completionsand/completion - Streaming with token callbacks
- Retry with backoff, configurable stop tokens
- Temperature, max tokens, and other inference parameters
agent/
├── src/
│ ├── agent/ # Core agent logic
│ │ ├── core.py # Main agent loop
│ │ ├── trace.py # Structured tracing
│ │ ├── memory.py # Memory management
│ │ └── tools/ # Tool registry
│ ├── backends/ # Model backends
│ ├── db/ # Database layer
│ ├── eoh/ # Evolution of Heuristics (meta-learning)
│ ├── mcp/ # Model Context Protocol client
│ └── reasoners/ # Reasoning strategies (CoT, ToT, etc.)
├── apps/
│ ├── runners/ # Demo and diagnostic runners
│ └── plotting/ # Visualization helpers
├── skills/ # Tool definitions
├── tests/ # Unit and integration tests
├── docs/ # Documentation
├── pyproject.toml # Dependencies and build config
└── README.md
| Variable | Description | Default |
|---|---|---|
AGENT_LOG_LEVEL |
Log level (DEBUG, INFO, WARNING, ERROR) | INFO |
AGENT_LOG_JSON |
Emit JSON logs | false |
AGENT_MODEL_PATH |
Path to llama.cpp model | ./models/ |
AGENT_MEMORY_PATH |
Memory storage path | .memory |
AGENT_TOON_FORMAT |
Use TOON format | false |
agent = Agent(
model="llama3:8b",
tools=[tool_registry],
memory_path=".memory",
max_turns=10,
temperature=0.7,
use_reflection=True,
toon_format=False,
)# Run all tests
pytest tests/
# Run specific test
pytest tests/test_agent.py
# Run with coverage
pytest tests/ --cov=src/agent- Python SDK for easier integration
- UI dashboard for tracing and monitoring
- Multi-agent collaboration workflows
- Fine-tuning support for custom models
- Event streaming for real-time tracing
- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
MIT License - see LICENSE for details.
- llama.cpp for the inference backend
- Chroma for vector storage
- SentenceTransformers for embeddings
Need help? Open an issue or join the discussion.