Roadmap: what 0.2.x and 0.3.0 hold

[moeen-mahmud]

What is planned, what could be picked up, what is known to be weak, and what is deliberately not planned. This is a plan rather than a promise: it changes as people use Dispach and tell us what broke. Comment here if something you need is missing.

Now: 0.2.1

Fixes found right after 0.2.0, three of the four from the first outside contributor: the phase route taking effect on an active session, its response shape, WhatsApp config errors with codes and hints, and the workspace template loading. Release notes.

Next: the rest of 0.2.x

Whatever early users hit first, in the order they hit it. Already open, some with a pull request in review:

• The phase route has a dead null branch, a stale hint and a comment describing a check that does not exist #20 the phase route's unreachable checks (PR fix(server): drop the phase route's dead null branch #41)
• Every generated agent.yaml says commented blocks are "refused at load until they ship", and they are not #25 the generated agent.yaml header claiming commented blocks are refused (PR fix(cli): the generated manifest stops claiming commented blocks are refused #46)
• The compaction notice tells the model the ladder runs in the old order #27 the compaction notice describing the old ladder order
• .env parsing: a quoted value with an inline comment keeps its quotes, and \\n round-trips as a newline #29 .env parsing of quoted values and escapes (PR fix(core): .env quoted values end at the close quote; escapes unescape in one pass #42)
• Remove the kv store, which nothing reads or writes, and drop its table #30 removing the unused kv store
• The SSRF address classifier lives in core and is tested only from tools-web #32 testing the SSRF address classifier under Node
• Report pass^k beside the pass rate in the harness benchmark #48 reporting pass^k, how many tasks pass every time, beside the benchmark's pass rate

The good first issue list is the place to start.

Small models: where Dispach is behind

The benchmark is honest about this, and it is the area most in need of work. Hermes scores higher than Dispach on qwen3.5-9b (112 against 104 of 125) and gpt-oss-20b (111 against 97), and the gpt-oss gap is the largest, with 95% intervals that only just overlap. What the runs show:

• gpt-oss-20b asks "do you want me to send it?" instead of sending what it was asked to send: gpt-oss-20b asks "do you want me to send it?" instead of sending #52
• gpt-oss-20b, GPT-6 Luna and qwen3.5-9b read the "Not enabled for you" list as covering tools they were given: Small models say a pinned tool "isn't enabled" #53
• Under the plain-text tool format (NLT), several mid-size models write an ACTION: block glued to a sentence, or promise a call and never make it: NLT failure shapes from the harness sweep: ACTION: glued to prose, and "Sending it now." with no call #36
• qwen3.5-9b under NLT was paused at 45 of its 52 runs; finishing it is about $0.30 of model spend
• Nothing has been run on a local model, though many self-hosters run one: Run the benchmark on local small models (Ollama qwen3.5:9b, gemma4:12b) #54
• Small-model rule compliance has no settled result: Run eval:rules against a hosted small open-weight model and settle the needs-run row #31
• Phase-scoped tools were built for small models and have only been measured on a frontier one, where they cost accuracy: Measure phase-scoped tools on a small model, the case the feature was built for #55

Evaluation

The benchmark already grades on the final state of the tools, isolates every run, commits every transcript, and balances "should act" against "should not" tasks. Following Anthropic's Demystifying evals for AI agents, what it lacks:

• Report pass^k beside the pass rate in the harness benchmark #48 pass^k beside the pass rate (above, in 0.2.x)
• Record what the runtime did in each benchmark run: nudges, repairs, how the turn ended #49 a record of what the runtime did in each run (nudges, repairs, how the turn ended), so "no change in the score" can be told apart from "the new behaviour never ran"
• A harder capability suite with trap tasks, now that five models score 125/125 #50 a harder suite with trap tasks: five models score 125/125, so for frontier models the current suite checks for regressions and can no longer show improvement
• A noise floor and an adoption rule for runtime changes, and a regression subset before each release #51 a measured noise floor, an adoption rule written before a change is measured, and a small regression run before each release

Skipped on purpose, for now:

• A model as grader. Every outcome here can be checked by code, and a judge adds variance of its own. It comes back if a task ever needs prose judged.
• Human grading. Transcripts are read by hand when a number looks wrong.
• pass@k (at least one success in k tries). For an agent people rely on, consistency is the number that matters.

Later: 0.3.0 candidates

Larger pieces, each with a design note and looking for an owner (help wanted):

• A published plugin entry point: dispach/plugin #33 a published plugin entry point, so a plugin can be written against npm instead of a checkout
• A new channel plugin: Discord (or Matrix) #34 a Discord or Matrix channel
• A native Anthropic Messages model transport #35 a native Anthropic Messages model transport
• npm trusted publishing (OIDC) to replace NPM_TOKEN #37 npm trusted publishing in place of a token

The experimental surfaces in 0.2.0 (rooms, stand-ins, team memory, the Slack, Teams and A2A channels, media) become stable once they have run against real use. Until then they may change within 0.2.x.

Known limits, not yet scheduled

• Endpoints: 0.2.0 was re-verified against DeepSeek's API and an open-weight model; OpenAI's and Anthropic's own endpoints were last checked on earlier releases. The benchmark reaches their models through OpenRouter.
• Windows has never been tested. macOS and Linux, including the container image, are.
• web_fetch and DNS rebinding: every resolved address is checked, but the connection resolves again; pinning the checked address is not possible through fetch.

Not planned

Asking for one of these is welcome, but expect the discussion before any code:

• A hosted service or trial environment. Dispach is self-hosted, and that is the product.
• Telemetry. The runtime reports nothing to anyone.
• Choosing the tool dialect automatically from the model. It stays explicit configuration, so behaviour never changes silently when the model does.
• Searching for tools at runtime. Tools are pinned at load, because search-then-call is two-hop reasoning, where small models fail.
• Embeddings for memory before lexical retrieval is shown to fall short (decision 5.6).
• MCP as the internal architecture. MCP stays one tool provider among several (decision 4.7).

The reasoning for the last four is in docs/00-DECISIONS.md.