Add diffusion models?

[vitali87]

Task Description

Explore using diffusion large language models (dLLMs) for Cypher generation and other agentic tasks requiring low latency.

Acceptance Criteria

• Fast
• Cheap
• Adequate reasoning

Updated Context (February 2026)

The dLLM landscape has evolved significantly since this was first opened in July 2025. Here is a summary of the current state:

Mercury 2 (Inception Labs)

On February 24, 2026, Inception Labs launched Mercury 2, the first reasoning dLLM. Key facts:

• ~1,000 tokens/sec output throughput (vs ~89 tok/s for Claude 4.5 Haiku Reasoning, ~71 tok/s for GPT-5 Mini)
• Tool calling and function calling are now supported, along with JSON structured output and a 128K context window
• OpenAI API compatible (drop in replacement for existing stacks)
• Pricing: $0.25/1M input tokens, $1.00/1M output tokens via OpenRouter
• Benchmarks: 91.1 AIME 2025, 73.6 GPQA, 67.3 LiveCodeBench
• Available on Azure AI Foundry for enterprise use

The original blocker (Mercury lacking tool calling) has been resolved with Mercury 2.

Gemini Diffusion (Google DeepMind)

Gemini Diffusion is Google's experimental diffusion model announced mid 2025. It generates entire blocks of tokens at once and matches the coding performance of their fastest autoregressive model. Currently still in waitlist/experimental status.

Code Specific dLLMs

• DiffuCoder (Apple): A 7B parameter dLLM trained on 130B tokens of code. Its global planning and iterative refinement properties are well suited for code generation tasks.
• CoDA (Salesforce): A lightweight 1.7B parameter diffusion code model that is competitive with some 7B models while using fewer parameters. Fully open sourced with training recipes.

PydanticAI Integration Path

PydanticAI supports OpenRouter natively, and Mercury is available on OpenRouter. This means integration could be as simple as:

agent = Agent("openrouter:inception/mercury")

Since Mercury 2 now supports tool calling and structured output, the PydanticAI integration path is viable for agentic workflows.

Ecosystem Maturity

While dLLMs have made major progress, the ecosystem is still maturing compared to autoregressive LLMs. Training and inference tooling is less established, and some benchmark gaps remain vs top autoregressive code models. However, the speed and cost advantages (5 to 10x on both fronts) make dLLMs very compelling for latency sensitive tasks like Cypher generation.

Recommendation

Mercury 2 is now the strongest candidate for integration. The original blocker (no tool calling) is gone, the model is fast, cheap, and accessible through OpenRouter. Worth prototyping with openrouter:inception/mercury to evaluate Cypher generation quality.

Converted from #46

[pratikrath126]

Great research summary! The recommendation to prototype with Mercury 2 seems solid. I'd also keep an eye on Gemini Diffusion if/when it becomes generally available, as Google's approach might offer different trade-offs. Looking forward to seeing how this integrates with the code graph!