[Bug] OLMo3 reasoning parser fails to detect </think> end tag, preventing GCD activation

[ivnle]

Description

The OLMo3 reasoning parser (olmo3_reasoning_parser.py) fails to correctly detect when thinking ends, causing all generated tokens to be classified as thinking tokens. This prevents grammar-constrained decoding (GCD) from activating for the post-thinking structured output.

Evidence

From experiments with OLMo3-7B-Think + GCD + thinking=high on the math500 task:

Question 0 (Polar coordinates conversion):

{
  "thinking_tokens": 1885,
  "total_tokens": 1885,
  "non_thinking_tokens": 0,
  "format_valid": false
}

Question 1 (Double sum problem):

{
  "thinking_tokens": 5521,
  "total_tokens": 5521,
  "non_thinking_tokens": 0,
  "format_valid": false
}

Question 2 (Function evaluation):

{
  "thinking_tokens": 2718,
  "total_tokens": 2718,
  "non_thinking_tokens": 0,
  "format_valid": false
}

Key observation: thinking_tokens == total_tokens in ALL cases means vLLM classified the ENTIRE output as thinking, so GCD was never applied to enforce JSON structure.

Expected Behavior

With thinking mode + GCD:

1. Model generates: <think>reasoning...</think>{"answer": "A"}
2. Thinking parser detects </think> at token position N
3. Tokens 0-N: classified as thinking (GCD disabled)
4. Tokens N+1 onwards: classified as non-thinking (GCD enabled)
5. Result: thinking_tokens < total_tokens, structured output is grammar-constrained, format_valid: true

Actual Behavior

1. Model generates: <think>reasoning...</think>[more text]
2. Thinking parser FAILS to detect </think>
3. ALL tokens classified as thinking (GCD never activates)
4. Result: thinking_tokens == total_tokens, free-form output instead of JSON, format_valid: false

Example Output

From Question 0, the model generated valid reasoning followed by free-form explanation instead of JSON:

Thinking portion (correctly detected):

Okay, so I need to convert the rectangular coordinate point (0, 3) to polar coordinates...
[1503 chars of step-by-step reasoning]
**Final Answer**
The polar coordinates are \boxed{(3, \frac{\pi}{2})}.

Actual output (should have been constrained to JSON):

To convert the point \((0, 3)\) in rectangular coordinates to polar coordinates \((r, \theta)\)...
Thus, the polar coordinates are \(\boxed{(3, \frac{\pi}{2})}\).

The model generated a full explanation instead of JSON because GCD never activated. Expected JSON:

{"answer": "(3, \\frac{\\pi}{2})"}

Reproduction

Setup:

• Model: allenai/Olmo-3-7B-Think
• Config: thinking=high, decoding=gcd, format=json_simple
• Task: Any structured output task (math, QA, etc.)

Command:

from ftx.runner import run_experiment

config = {
    "model": {"name": "olmo3-7b", "checkpoint": "allenai/Olmo-3-7B-Think"},
    "format": {"type": "json", "complexity": "simple"},
    "generation": {
        "output": {"decoding": "gcd"},
        "thinking": {"mode": "high"}
    }
}
run_experiment(config)

Expected: JSON output with format_valid: true
Actual: Free-form text with thinking_tokens == total_tokens, format_valid: false

Related Code

• Reasoning parser: libs/vllm/vllm/reasoning/olmo3_reasoning_parser.py
• Structured output manager: libs/vllm/vllm/v1/structured_output/__init__.py (lines 303-330)
• Config: libs/vllm/vllm/config/structured_outputs.py

Impact

Critical: This makes GCD + thinking mode completely non-functional for OLMo3 models. All experiments with this configuration produce invalid structured output (100% failure rate in our tests).

Affected configurations:

• OLMo3-7B-Think models
• Any configuration using thinking=high + GCD
• Potentially other OLMo variants

Diagnostic Data

All 3 test examples show identical failure pattern:

• thinking_tokens == total_tokens (100% of output classified as thinking)
• non_thinking_tokens: 0 (GCD never applied)
• format_valid: false (no valid JSON produced)
• Model generates </think> tags but they're not detected by the parser

Suggested Investigation

Check olmo3_reasoning_parser.py:

1. Verify is_reasoning_end() correctly detects </think> tokens
2. Check token sequence matching logic for </think> detection
3. Verify tokenization consistency between prompt and completion
4. Ensure prompt_length offset is correctly applied to skip prompt tokens
5. Add debug logging to trace when/if </think> detection fires

Environment

• vLLM version: 0.13.0rc2.dev291+gf5e2446e2 (custom fork)
• vLLM upstream: vllm-project/vllm
• Model: allenai/Olmo-3-7B-Think
• Thinking parser: olmo3
• GCD backend: xgrammar
• CUDA: 12.8
• GPU: NVIDIA L40S
• Platform: Linux

Additional Context

This issue was discovered during systematic testing of thinking modes with structured output constraints for the Format Tax research project. The bug completely breaks the intended workflow where:

1. Model reasons freely within <think>...</think> tags
2. After thinking, model outputs strict JSON/XML according to schema

Without working </think> detection, the structured output portion is never constrained, defeating the purpose of GCD.

[ivnle]

Apologies, this issue was filed in the wrong repository by mistake. We're investigating this on our end first. Closing to avoid noise.