[CONTRIBUTION]: Add Gemma 4 tool-call and reasoning parsers

[the-david-oy]

Type of Change

New feature

Problem Statement

Dynamo's frontend does not yet support Google Gemma 4 thinking models. A Gemma 4 deployment served by vLLM ≥ v0.19.0 (where Gemma 4 support landed via vllm-project/vllm#38826 and #39027) currently emits tool calls and reasoning content in a custom serialization that no existing Dynamo parser family can consume. The format uses bare unquoted keys, a custom <|"|> string delimiter, and dedicated <|tool_call>...<tool_call|> markers for tool calls, plus <|channel>thought\n...<channel|> for reasoning. Without this support, callers running Dynamo + vLLM against Gemma 4 must either (a) fall back to mode B (--dyn-chat-processor vllm, which delegates parsing to vLLM's Python preprocessor and gives up Dynamo's KV routing benefits), or (b) lose tool-calling and reasoning content entirely.

Proposed Solution

Introduce a new Gemma4 parser family alongside the existing JSON / XML / DSML / Pythonic / Harmony / KimiK2 / Glm47 families. Concretely:

• New tool-call parser at lib/parsers/src/tool_calling/gemma4/parser.rs — recursive-descent grammar parser into serde_json::Value (booleans, numbers, <|"|>-delimited strings, nested objects, arrays). Mirrors the precedent set by kimi_k2_parser.rs and dsml/parser.rs (each got their own ParserConfig variant when their grammar didn't fit the JSON/XML mold).
• New reasoning parser at lib/parsers/src/reasoning/gemma4_parser.rs — implements the ReasoningParser trait directly (mirrors granite_parser.rs shape) with the streaming thought\n prefix-stripping logic ported from upstream vLLM's Gemma4ReasoningParser.
• Vendored chat template at examples/chat_templates/gemma4_tool.jinja — verbatim copy of upstream vLLM's examples/tool_chat_template_gemma4.jinja (Apache-2.0). Required because the stock HuggingFace Gemma 4 chat template does not emit the <|"|>-delimited tool-definition encoding the parser expects. Used via --custom-jinja-template.
• Registry entries: gemma4 (alias gemma-4) for both --dyn-tool-call-parser and --dyn-reasoning-parser.
• is_reasoning_disabled_by_request polish in lib/llm/src/preprocessor.rs for parity with the existing kimi_k25 / deepseek_v4 / nemotron_nano short-circuits.
• Doc rows in docs/agents/{tool-calling,reasoning}.md and lib/parsers/README.md, plus a recommended-pairing line.

Recommended pairing for Gemma 4 deployments:

--dyn-tool-call-parser gemma4 \
--dyn-reasoning-parser gemma4 \
--custom-jinja-template examples/chat_templates/gemma4_tool.jinja

Estimated PR Size

XL (501-1000 lines)

(Net diff is ~1700 lines, but ~333 of those are the verbatim vendored jinja template, ~700 are inline #[cfg(test)] test coverage, and ~50 are docs. Net new logic is ~600 lines of Rust across the two new parser modules.)

Files/Components Affected

New files:

• lib/parsers/src/tool_calling/gemma4/{mod.rs, parser.rs}
• lib/parsers/src/reasoning/gemma4_parser.rs
• examples/chat_templates/gemma4_tool.jinja
• examples/chat_templates/README.md

Modified files:

• lib/parsers/src/tool_calling/{config.rs, parsers.rs, mod.rs} — ParserConfig::Gemma4 variant + factory + dispatch
• lib/parsers/src/reasoning/mod.rs — ReasoningParserType::Gemma4 variant + registry
• lib/parsers/{README.md, TEST_CASES.md} — parser-family cheat-sheet rows
• docs/agents/{tool-calling.md, reasoning.md} — parser-table rows + recommended-pairing line
• lib/llm/src/preprocessor.rs — is_reasoning_disabled_by_request gemma4 arm + parametric tests

Out of scope (deferred follow-ups):

• Native Rust chat-template formatter analogous to lib/llm/src/preprocessor/prompt/deepseek_v4.rs (multi-day port of the 333-line jinja).
• SGLang frontend wiring in components/src/dynamo/frontend/sglang_prepost.py (mode C; depends on whether SGLang has its own gemma4 parser).
• Streaming-jail integration tests under lib/llm/tests/data/vllm/gemma4/ (need real captured chunks from a Gemma 4 vLLM deployment).

[grahamking]

Yes!

[grahamking]

Landed.