Local AI Agent, From Scratch

A personal AI agent built step by step, with no agent framework, on top of a model running entirely on your own machine (Ollama + Qwen3, 8B). No cloud API calls, no data leaving the machine, and no external cost per query.

Every layer, from raw model calls to tool use, memory, retrieval, and live web access, is hand-written first, so nothing is a black box. Once the hand-built version worked end to end, it was rebuilt a second time in LangGraph (see langgraph/), to make it concrete exactly what a framework replaces versus what it doesn't.

Full write-up: [link to the blog post once published]

The build, step by step

Step	What it adds	File(s)
1	Environment: WSL2, Ubuntu, Ollama, GPU	— (see blog post)
2	Python → Ollama: chat, streaming, system prompts	step2a_basic.py … step2d_chat.py
3	Python → tool: a function, a schema, a dispatcher	step3_tools_demo.py, tools.py
4	The model decides to call a tool (single round)	step4_tool_calling.py
5	The agent loop: repeat until no more tool calls	step5_agent_loop.py
6	Long-term memory: facts that survive a restart	step6_memory_agent.py, memory_store.py
7	RAG: retrieval by meaning, not keywords	step7_ingest.py, step7_rag_agent.py, rag_store.py
8	Live web search + page fetching	step8_web_agent.py, web_tools.py
9	Everything consolidated, with session persistence	agent.py
9′	The same agent, rebuilt in LangGraph	langgraph/

Quick start

python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

ollama pull qwen3:8b
ollama pull nomic-embed-text   # for RAG (Step 7+)

# Build the knowledge base once
python3 agent.py --ingest company_handbook.txt

# Run the full agent
python3 agent.py

Other entry points:

python3 agent.py --session work          # a separate, named conversation
python3 agent.py --ask "what time is it in Tokyo?"   # one-shot, scriptable
python3 step5_agent_loop.py chat         # just the agent loop, no memory/RAG/web

Each stepN_*.py file also runs standalone, so you can follow the build in order rather than jumping straight to the finished agent.

Hand-built vs. LangGraph

The langgraph/ folder is the same agent, rebuilt with a framework. The honest comparison, confirmed by actually running both:

What was hand-written (agent.py)	What LangGraph replaced it with (langgraph/)
A while loop calling the model, checking for tool calls, running them, looping	Two nodes ("agent", "tools") + one line: add_conditional_edges("agent", tools_condition)
A hand-written JSON schema for every tool function	Inferred from type hints + docstring via a single @tool decorator
A dispatcher dict mapping tool names to functions	ToolNode(ALL_TOOLS) — built in
~30 lines to serialize conversation history to JSON and reload it on restart	One line: SqliteSaver.from_conn_string("sessions.db")

What did not change: the system prompt, every tool function, and the conversation-trimming policy — because how to trim a conversation is a design decision, not boilerplate, and no framework should make it for you.

Why local, why no framework first

• Data never leaves the machine. Every tool call, every retrieved document, every fact remembered stays on disk, useful for anyone evaluating this in a regulated or data-sensitive context.
• No per-query API cost. The model runs once, locally, on hardware already owned.
• Understanding before abstraction. Building the loop, the tool dispatcher, and the session persistence by hand first means the LangGraph version isn't a black box — it's a known list of what got automated, with every "would this hide a decision I need to make" question already answered.

Requirements

• Python 3.10+
• Ollama, running locally
• ~5–8GB free (model weights + embedding model)
• For the LangGraph edition: pip install -r langgraph/requirements-langgraph.txt

License

MIT — see LICENSE.