VeloGuide — AI Cycling Trip Planner for the Netherlands

AI-powered system for planning 1–3 day cycling trips in the Netherlands, built on the pi-agent framework.

Input: text, images (photo of a place you'd like to visit — its city is identified and used as the start), and voice. Output: a grounded, multi-day itinerary; refine it across turns ("make day 2 shorter"), or hit Stop to cancel and + New trip to start fresh.

Voice has two modes: the default browser speech-to-text (Chrome/Edge/Safari, no server call) or, set via STT_BACKEND, server-side STT (Deepgram, or Gemini through the same OpenRouter key) for much better accuracy — see STT options. Either way the transcript becomes ordinary text before the agent sees it, so every grounding rule applies to all modalities.

Docs: DECISIONS.md (one-page decisions/assumptions/limitations/scaling) · ARCHITECTURE.md (deep-dive) · EVALUATION.md (quality evaluation plan)

How this maps to the evaluation criteria

Criterion	Where it's addressed
Runnable, testable prototype	This README — Quick Start (make setup && make run), make smoke (one real headless plan + latency trace), make test (offline unit tests), make eval (scored scenario suite). Local Overpass for usable latency.
Result quality & LLM-issue handling	The tool-grounding invariant (the LLM never emits a falsifiable fact) + the full LLM-Specific Issue table in ARCHITECTURE.md; verified end-to-end by make eval (programmatic grounding + geo-sanity checks) and JUDGE=1 make eval (LLM-as-judge). Plan in EVALUATION.md.
Adequacy of assumptions	DECISIONS.md → Assumptions (recreational profiles, fitness drives distance, accommodation out of scope, NL OSM coverage), with the technical boundary explained in ARCHITECTURE.md.
Paths toward scaling	DECISIONS.md → Scaling (×100 / ×10,000 summary) and the ARCHITECTURE.md → Scalability deep-dive (where it breaks first and the fix at each tier).

Prerequisites

• Node.js 22+ — nodejs.org
• OpenRouter API key — set in .env (one key routes to Claude/Gemini/GPT)
• OpenRouteService API key (optional) — set ORS_API_KEY in .env for cycling-network routing with elevation; without it, routing falls back to the keyless OSRM demo

Quick Start

cp .env.example .env
# Edit .env and add your OPENROUTER_API_KEY

make setup
make run

Open http://localhost:3000 in your browser.

⚠️ Strongly recommended for usable latency: set up a local Overpass (see below) before serious use. Without it, POI/junction lookups hit the rate-limited public OSM endpoint and a full plan takes ~1 minute when un-throttled, several minutes when rate-limited; with it, a fresh plan is ~30s (latency is model-generation-bound, not tool-bound — see ARCHITECTURE.md). The app falls back to the public endpoint automatically if it's not configured.

Note on long conversations: each turn replays the growing history to the model, so a long multi-turn session gets slower. Click + New trip to start a fresh session (resets latency) when you begin a genuinely different trip; refining the current plan is unaffected.

Tip: ⚡ Fast mode (next to the input) is on by default — the model batches its tool calls and writes a compact, scannable plan. Uncheck it for a fuller, more detailed multi-day itinerary.

Fast local Overpass (recommended)

POI and junction lookups use the public OSM Overpass API, which is rate-limited and slow under load. Running a local Overpass with just the Netherlands extract removes the limit; a fresh plan then lands around ~30s (measured: 1-day ~30s, 3-day ~32s — the bulk is model generation across the agent's sequential turns, not the geo lookups, which total ~5s).

One-time setup (needs Docker). Budget ~30–60 min total, mostly hands-off: ~1.3 GB download, then a PBF→OSM-XML conversion (single-threaded bzip2, the slow part), then the database import. It runs in the background in a persistent Docker volume, so you only do it once.

docker run -d --name overpass_nl \
  -e OVERPASS_META=yes \
  -e OVERPASS_MODE=init \
  -e OVERPASS_PLANET_URL=https://download.geofabrik.de/europe/netherlands-latest.osm.pbf \
  -e OVERPASS_PLANET_PREPROCESS='mv /db/planet.osm.bz2 /db/planet.osm.pbf && osmium cat -o /db/planet.osm.bz2 /db/planet.osm.pbf && rm /db/planet.osm.pbf' \
  -e OVERPASS_RULES_LOAD=10 \
  -v overpass_db_nl:/db \
  -p 12345:80 \
  wiktorn/overpass-api

Wait for the import to finish (docker logs -f overpass_nl), then add to .env:

OVERPASS_URL=http://localhost:12345/api/interpreter

The app uses the local instance first and falls back to the public endpoint automatically if it's unset or unreachable. (Geofabrik serves the NL extract as .osm.pbf; the PREPROCESS step converts it to the OSM-XML the Overpass importer expects.)

Two gotchas with this image (already accounted for above, but in case you hit them):

• The container runs the import in init mode and then exits. Once the import is done, docker start overpass_nl to serve it.
• If queries return runtime error: open64: 13 Permission denied /db/db/osm3s_osm_base, the DB directory is 0700 and the query process (a different user) can't reach the dispatcher socket. Fix once:

  docker exec -u root overpass_nl chmod 755 /db /db/db && docker restart overpass_nl

Verify it's serving real data:

curl -s -X POST http://localhost:12345/api/interpreter \
  --data-urlencode 'data=[out:json];node["amenity"="cafe"](52.36,4.88,52.40,4.93);out 3;'

Architecture

Single pi-agent with 7 custom tools, fronted by a deterministic intake gate, + web chat UI.

Browser (chat) ←WebSocket→ Express server → intake gate ─(start/days/date settled)→ pi-agent session
                                               │                                         ↓
                                  (anything missing/conflicting)                      7 tools
                                               │                                         ↓
                                     one targeted question        OpenRouteService / OSRM / OSM Overpass / Open-Meteo / Nominatim

Before any planning, the gate (a tool-free extraction — it cannot route or plan) settles three parameters: start location (text or photo), trip length (days), and start date (for a real forecast). A missing start or length → one targeted question, and the planning agent is never invoked. A missing date is never asked — tomorrow is assumed and stated at the top of the plan; only conflicting dates ("today" vs "from June 20") trigger the question. The gate asks at most once: decline to specify and it plans with stated defaults (1 day, from Amsterdam, starting tomorrow), disclosing the assumption. Refinement turns ("make day 2 shorter") skip the gate.

When the start comes from a photo, the gate identifies the city with a vision model (VISION_MODEL, default Gemini 2.5 Flash — a small text model is unreliable at this) and discloses it for confirmation ("I identified the photo as Groningen — tell me if that's wrong"); a photo of a clearly non-Dutch place is redirected rather than coerced into an NL city.

The UI also exposes Stop (cancel an in-flight plan via the SDK's session.abort()) and + New trip (dispose the session and start fresh — the latency reset for long conversations).

Tools

Tool	API	Purpose
geocode	Nominatim	Place names → coordinates
plan_route	OpenRouteService → OSRM	Cycling-network routes with real distances, elevation, turn-by-turn (ORS when ORS_API_KEY set, else OSRM fallback)
get_weather	Open-Meteo	Daily weather forecast
find_pois	OSM Overpass	Cafes, restaurants, museums, windmills
find_accommodation	OSM Overpass	Hotels, B&Bs, campsites
find_knooppunten	OSM Overpass	Dutch cycling junction network
web_search	DuckDuckGo	Local tips, events, seasonal info

Why pi-agent

Pi is an open-source (MIT) agent harness by Earendil. It ships three packages: pi-coding-agent (the agent runtime), pi-agent-core (tool calling + state management), and pi-ai (a unified multi-provider LLM API — Anthropic, OpenAI, Google, OpenRouter, etc.). In SDK/embedded mode (createAgentSession), it gives us:

• ReAct-style agentic loop — the model calls tools, reads results, and decides whether to call more or write the final answer; we supply only the tools and the system prompt.
• Typed tool definitions (defineTool + TypeBox schemas) — parameter types are derived end-to-end from the schema; pi-agent validates every tool call against it before our execute runs.
• Streaming events (session.subscribe) — text_delta, tool_execution_start/end, compaction, retry events; we relay these over WebSocket for live tool-activity chips and progressive rendering.
• Context compaction — automatic conversation summarisation when approaching the token limit, so multi-turn sessions don't silently truncate.
• Auto-retry — configurable retry logic for transient LLM failures (we set maxRetries: 3).
• Image support — session.prompt(text, { images }) passes images alongside text; the model sees them natively.
• Cancellation — session.abort() stops a running turn cleanly (powers the Stop button).
• Session lifecycle — session.dispose() frees resources (powers + New trip).
• noTools: "builtin" — disables pi's default coding tools (read/write/bash/grep) so the model sees only our 7 domain tools and cannot shell out or edit files.
• Model-agnostic — getModel("openrouter", id) resolves any OpenRouter-routed model; one env var switches between Haiku, Sonnet, Gemini Flash.

This means VeloGuide's own code is domain logic (tools, prompt, intake gate, pipeline guards) — the agentic loop, schema validation, streaming, compaction, retries, and multi-provider auth are all inherited from the SDK.

Stack

• LLM: Claude Haiku 4.5 via OpenRouter (default; ~2× Sonnet's throughput, reliable tool-calling — see DECISIONS.md). Set MODEL=anthropic/claude-sonnet-4.6 for higher reasoning quality, or MODEL=google/gemini-2.5-flash for the lowest cost.
• Vision (photo → city): VISION_MODEL, default google/gemini-2.5-flash; used by the intake step only when an image is attached.
• Agent: pi-agent SDK (@earendil-works/pi-coding-agent) — see Why pi-agent above
• Backend: TypeScript, Express, WebSocket
• Frontend: Vanilla HTML/CSS/JS, marked.js for markdown
• Speech-to-text: STT_BACKEND — browser (default, Web Speech API) · gemini (OpenRouter key) · deepgram (DEEPGRAM_API_KEY)
• Feedback loop (optional): set FEEDBACK_DB to capture 👍/👎 on plans; make feedback-report turns downvotes into regression cases (see EVALUATION.md)

Voice input (STT)

The 🎤 button's accuracy depends on STT_BACKEND:

STT_BACKEND	Engine	Key	Notes
browser (default)	Browser Web Speech API	none	No server call; accuracy varies, mishears place names
gemini	Gemini via OpenRouter	uses OPENROUTER_API_KEY	Records in-browser → server transcribes; domain-biased
deepgram	Deepgram STT	DEEPGRAM_API_KEY	Dedicated STT, robust on non-speech (no confabulation)

For server modes the browser records audio, re-encodes it to WAV, and POSTs to /transcribe; the transcript lands in the input box for review before sending. The agent only ever receives text, so all grounding rules hold regardless of mode.

Project Structure

velo_guide/
├── Makefile
├── .env.example
├── DECISIONS.md                 # 1-page: decisions, assumptions, limitations, scaling
├── ARCHITECTURE.md              # Deep-dive: agent loop, latency engineering, LLM-issue table
├── EVALUATION.md                # Quality evaluation plan (make eval runs the automated half)
├── backend/
│   ├── package.json
│   ├── tsconfig.json
│   ├── eval/                    # Test cases + eval harness (make eval) + feedback-report.ts
│   ├── test/                    # Offline unit tests (make test, CI)
│   └── src/
│       ├── main.ts              # Entry point
│       ├── server.ts            # Express + WebSocket + /transcribe, /feedback, /config
│       ├── agent.ts             # Pi-agent session factory
│       ├── system-prompt.ts     # Dutch cycling domain prompt + reasoning-strip backstop
│       ├── intake.ts            # Deterministic intake gate (text/photo, TypeBox-validated)
│       ├── pipeline.ts          # Shared conversation pipeline (gate → agent → guards)
│       ├── stt.ts               # Server STT (browser/gemini/deepgram)
│       ├── feedback.ts          # Off-by-default 👍/👎 capture (node:sqlite)
│       ├── tools/               # 7 custom tools
│       └── utils/               # Overpass client, formatters, geo-sanity, images
└── frontend/
    ├── index.html               # Chat interface
    ├── style.css
    └── app.js                   # WebSocket client

Development

make lint    # Type-check
make test    # Offline unit tests (no network, no API key)
make run     # Start dev server
make smoke   # One real headless agent turn with tool/latency trace + grounding checks
make eval    # Run the eval suite (backend/eval/test-cases.json) with a pass/fail scorecard
make feedback-report  # Satisfaction rate + downvotes → candidate regression cases (needs FEEDBACK_DB)

Optional feedback loop (off by default): set FEEDBACK_DB=./feedback.db in .env to show a 👍/👎 under each plan and capture it — with the tool trace that produced it — to a local SQLite file (Node's built-in node:sqlite, no extra dependency). make feedback-report turns downvotes into candidate regression cases for the eval suite. It's an evaluation hook, not session persistence — no auth, no PII, just an anonymous client id. See EVALUATION.md.

Add JUDGE=1 to make eval for the LLM-as-judge pass: a second model (Sonnet by default) verdicts the judgment-call assertions, scores quality dimensions 1–5, and flags soft hallucinations against the captured tool outputs. See EVALUATION.md.

CI (GitHub Actions): every push runs the type-check, unit tests, and a frontend syntax check. The live agent eval (make eval, real LLM + geo API calls) is a manually dispatched job — it spends API credits and depends on rate-limited public endpoints, so it's run from the Actions tab (with the OPENROUTER_API_KEY repo secret) before releases or after prompt/model changes.

make smoke accepts a custom prompt, an optional follow-up turn (multi-turn refinement check), and an optional image (multimodal check):

cd backend && npx tsx src/smoke.ts "Plan a 2-day trip from Utrecht" "make day 2 shorter"
cd backend && IMAGE=eval/fixtures/dutch-windmill.jpg npx tsx src/smoke.ts "I want to cycle somewhere that looks like this"