JIT Context OS: Using local SQLite WAL (<3ms) as an epistemic context runtime for coding agents

[wojciechwiesner]

Background & Philosophy

When running autonomous multi-turn agents or coding loops, the standard pattern has been simple: append previous turns to conversation history and pass the growing haystack back to the LLM.

This leads to three failure modes:

1. Lost-in-the-Middle: Important system constraints buried in 30k+ token histories get ignored.
2. Epistemic Self-Poisoning: Assistant speculations and hallucinated variables from Turn 3 become "facts" in Turn 7.
3. The Haystack Tax: Slower TTFT, higher token costs, and 429 rate limit errors.

---

Why SQLite WAL (<3ms) Fits Epistemic Context

Instead of treating conversational history as active context, we built an epistemic runtime: JIT Context OS.

• L0 Hot-Path (SQLite WAL): A local SQLite database (, ) stores atomic mutation events, working-set file hashes, and verified runtime proofs with sub-3ms Read-Your-Own-Writes.
• Epistemic Authority Weighting:
  • User instructions: Weight 1.0 (Invariant I1)
  • Deterministic tool execution proofs (, test pass): Weight 1.0
  • Assistant conversational reasoning: Weight 0.0 (preventing self-poisoning)
• Lean Dynamic Working-Set Capsule (<1.8k tokens):
  Prompts never accumulate conversational history. Each turn receives only the static tool schema (100% prefix cache hits), the immediate prior tool result, and an adaptively compiled capsule (1.2k – 1.8k tokens) containing active goals, current interface contracts, and verified assertions.

---

Empirical Proof: EXP-009 Benchmark (99 Mac Mini vs Cloud Frontier)

We evaluated this on an Apple Mac Mini M2 Pro (16GB RAM) fixing 4 distinct root causes across 3 interconnected Python modules, verified by an independent blind ============================= test session starts ==============================
platform darwin -- Python 3.14.6, pytest-9.0.2, pluggy-1.6.0
rootdir: /Users/wojciechwiesner/Projects/active/theones.io
plugins: anyio-4.12.1, asyncio-1.3.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 6 items

forms-api/test_forms_api.py ...... [100%]

============================== 6 passed in 0.59s =============================== suite:

Setup & Model	Platform	Turns to Complete	Wall-Clock Time	Pytest Verification	Compute Cost
🥇 Local Qwen 3.8 9B + JIT	Mac Mini M2 Pro (Ollama 32k)	4 turns	~2 min (126s)	4/4 PASSED (exit 0)	/bin/bash.00 (100% local)
🥈 Gemini 3.8 Flash + JIT	Google Cloud API	9 turns	17.7s	4/4 PASSED (exit 0)	Paid Cloud API
🥉 Gemini 3.8 Flash (No JIT / Chat)	Google Cloud API	10 turns (exhausted)	34.5s	❌ 0/4 FAILED (lost in tests/)	Paid Cloud API

Takeaway: A 9B open-weight model running locally on consumer Apple Silicon solved the multi-file task in fewer turns than Gemini 3.8 Flash because its context was kept surgically lean (<1.8k tokens). Small models aren't incapable of complex reasoning; they have been suffocated by chat history.

---

Specification & Code

• GitHub Repository: github.com/wojciechwiesner/jit-context
• CERN Zenodo Permanent Record: DOI: 10.5281/zenodo.22649542
• Technical Writeup: theones.io/blog/why-we-killed-chat-history-in-coding-agents/

Curious to hear thoughts from the and SQLite community on using SQLite WAL as the epistemic state layer for LLM agents!