Most LLM failures aren't intelligence failures, they're honesty failures: the model says it read the file, the agent says the feature is wired up, the eval says things improved, the memory says it remembers. None of those claims came with evidence. I build the machinery that makes them checkable.
Verification-first LLM development. Every layer of an LLM system makes claims — about what reached the model, what the pipeline actually did, what "done" means, what a result proves. I build small, sharp tools where every one of those claims carries a receipt:
| verifies | ||
|---|---|---|
| dummybot | what your pipeline feeds the model | A scripted LLM test double with prompt assertions — most real LLM bugs are feed bugs, and a return-only mock can't see them. Zero deps. pip install dummybot |
| provenmap | what deserves to be called done | A project map where DONE is computed from evidence, never filed — "connected" must be grep-proven, and agent-inflated progress has nowhere to hide. Zero deps. pip install provenmap |
| memvault (shipping this week) | what your app actually remembers | Durable, provenanced memory for LLM apps: explicit gated writes, budgeted recall packets with receipts. The context window is a cache, not a database. |
| honest-eval (in design) | what an eval result can honestly claim | Fair baselines, negative controls, ceiling tiers that are supposed to fail, and claim boundaries stamped into every report. |
All of these are extracted from the internals of Ticos — a local-first LLM runtime I'm building around a deliberately small frozen model, where every run must end in an answer, a receipt-cited refusal, or a typed fault. Silence is structurally impossible. It goes public when it's ready; the tools ship as they earn it.
I direct AI coding agents under an adversarial verification regime: contract tests authored before implementation, every change attacked by independent review lenses before it lands, and progress tracked by evidence rather than assertion (provenmap exists because I refused to let my own tooling tell me flattering lies). The scarce skill isn't writing code faster anymore — it's building verification that can't be fooled by confident machines, including your own.
LLM systems: local-first (Ollama, llama.cpp-class small models), OpenAI-compatible APIs, RAG & embeddings, deterministic eval harnesses, MCP · Core: Python (stdlib-first — every published package has zero runtime dependencies), SQLite, pytest + hypothesis, GitHub Actions
Before LLMs: full-stack web — React/Next.js, Angular, Node/NestJS, Python/FastAPI, Postgres/MongoDB, Docker/K8s, AWS/GCP/Azure. I still build the web parts when a system needs a face.
Open to LLM reliability and agent-infrastructure work — reach me here on GitHub.