Fix GuidedDecodingParams backend_name issue

[sethkimmel3]

For some reason the "@Property" decorator on backend_name is creating issues, where the ValueError Unknown guided decoding backend.. gets raised when trying to use a backend like xgrammar:disable-any-whitespace.

What's especially odd is that printing out guided_params.backend_name before the conditional checks makes it work, as does assigning it to a variable. I have a feeling there's some kind of observer effect or race condition going on. Regardless, this seems to fix it and its unclear why this is a property rather than a method anyway.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[russellb]

Thanks for the PR!

Can you clarify how you trigger the problem?

[sethkimmel3]

@russellb the problem gets triggered when using the backend xgrammar:disable-any-whitespace within GuidedCodingParams.

[sethkimmel3]

Whoops - sorry if that last push somehow triggered a review request from a ton of people! My bad

[russellb]

@sethkimmel3 I'm not able to reproduce the problem. I tried both online and offline methods and specified xgrammar:disable-any-whitespace as the backend. Can you provide something that reproduces it?

Here's the offline test, using a modified version of examples/offline_inference/structured_outputs.py:

# SPDX-License-Identifier: Apache-2.0

from vllm import LLM, SamplingParams
from vllm.sampling_params import GuidedDecodingParams

llm = LLM(model="Qwen/Qwen2.5-3B-Instruct", max_model_len=100)

# Guided decoding by Choice (list of possible options)
guided_decoding_params = GuidedDecodingParams(choice=["Positive", "Negative"], backend="xgrammar:disable-any-whitespace")
sampling_params = SamplingParams(guided_decoding=guided_decoding_params)
outputs = llm.generate(
    prompts="Classify this sentiment: vLLM is wonderful!",
    sampling_params=sampling_params,
)
print(outputs[0].outputs[0].text)

or online:

import json
import openai

system_prompt = """
Fill the following json schema for a character creator in D&D:
{
    "name": "string",
    "race": "string",
    "class": "string",
    "level": "int",
    "background": "string",
    "alignment": "string",
    "backstory": "string"
}
"""

model_id = "Qwen/Qwen2.5-1.5B-Instruct"

user_prompt = "Make me a bunch of characters from Jason Bourne movies. Output a list (array) of character json objects."

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": user_prompt}
]

json_schema_multiple = {
    "type": "array",
    "items": {
        "type": "object",
        "properties": {
            "name": {"type": "string"},
            "race": {"type": "string"}, 
            "class": {"type": "string"},
            "level": {"type": "integer"},
            "background": {"type": "string"},
            "alignment": {"type": "string"},
            "backstory": {"type": "string"}
        },
        "required": ["name", "race", "class", "level", "background", "alignment", 
                    "backstory"]
    }
}
client = openai.Client(api_key="your_openai_api_key", base_url="http://localhost:8000/v1")
result = client.chat.completions.create(
    model=model_id,
    messages=messages,
    max_tokens=3000,
    temperature=0.,
    stream=False,
    extra_body={"guided_json": json_schema_multiple, "guided_decoding_backend": "xgrammar:disable-any-whitespace"}
)
output = result.choices[0].message.content
#print(output)

json_output = json.loads(output)
print(json.dumps(json_output, indent=4))

[sethkimmel3]

Thanks @russellb for trying to repro. I too am actually unable to repro now, but bumped the version to the nightly build (and now the latest commit hash fb0acb6c72874e98617cabee4ff4851569374fc9).

I'm going to close this for now assuming that a change has been made that resolves this, but will reopen if it's encountered again.