[Bug]: ERNIE-4.5-21B-A3B-Thinking accuracy issue due to PR 23991 #25833
Description
Your current environment
H800 GPU
🐛 Describe the bug
start server
vllm serve baidu/ERNIE-4.5-21B-A3B-Thinking \
--served-model-name ERNIE-4.5-21B-A3B-Thinking \
--port 8508 \
--max-model-len 32768 \
--reasoning-parser deepseek_r1
test script
from openai import OpenAI
import argparse
# Set OpenAI's API key and API base to use vLLM's API server.
# 解析命令行参数
parser = argparse.ArgumentParser(description="Run OpenAI client with custom API base.")
parser.add_argument(
"--api-base",
type=str,
default="http://127.0.0.1:8508/v1",
help="OpenAI API base URL (e.g. http://127.0.0.1:8508/v1)"
)
args = parser.parse_args()
openai_api_key = "EMPTY"
openai_api_base = args.api_base
client = OpenAI(
api_key=openai_api_key,
base_url=openai_api_base,
)
tools = [
{
"type": "function",
"function": {
"strict": True,
"name": "get_current_temperature",
"description": "Get current temperature at a location.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": 'The location to get the temperature for, in the format "City, State, Country".',
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": 'The unit to return the temperature in. Defaults to "celsius".',
},
},
"required": ["location"],
},
},
},
{
"type": "function",
"function": {
"name": "get_temperature_date",
"description": "Get temperature at a location and date.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": 'The location to get the temperature for, in the format "City, State, Country".',
},
"date": {
"type": "string",
"description": 'The date to get the temperature for, in the format "Year-Month-Day".',
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": 'The unit to return the temperature in. Defaults to "celsius".',
},
},
"required": ["location", "date"],
},
},
},
]
query = "What's the temperature in Beijing today? What's the temperature in Guangzhou on October 1st?"
# query = "Where is the capital of France"
isStream = True
chat_stream = client.chat.completions.create(
model="ERNIE-4.5-21B-A3B-Thinking",
# model="/root/paddlejob/wangyafeng/models/ERNIE-4.5-21B-A3B-PT",
messages=[
# {"role": "system", "content": "你是一个乐于助人的助手。"},
{"role": "user", "content": f"{query}"},
],
# tools=tools,
stream=isStream, # enable streaming
temperature=0,
top_p=1,
max_tokens=1024*31,
# reasoning_effort="medium", # high|medium|low
extra_body={
# "skip_special_tokens": False,
"chat_template_kwargs":{"enable_thinking": True}
}
)
print(f"Input:{query}")
print("Output")
reasoning_content_list = []
content_list = []
tool_calls_list = []
if isStream:
for chunk in chat_stream:
# print(chunk)
reasoning_content = getattr(chunk.choices[0].delta, "reasoning_content", None)
tool_calls = getattr(chunk.choices[0].delta, "tool_calls", None)
content = chunk.choices[0].delta.content
if reasoning_content:
print(reasoning_content, end="", flush=True)
reasoning_content_list.append(reasoning_content)
if content:
print(content, end="", flush=True)
content_list.append(content)
if tool_calls:
print(tool_calls, end="", flush=True)
tool_calls_list.append(tool_calls)
print()
print("========reasoning_content========")
for i in range(len(reasoning_content_list)):
print(reasoning_content_list[i], end="", flush=True)
print("========content========")
for i in range(len(content_list)):
print(content_list[i], end="", flush=True)
print("========tool_calls========")
tool_calls_res = {}
for tool_calls in tool_calls_list:
# print(tool_calls, end="", flush=True)
# print(tool_calls)
for tool_call_chunk in tool_calls:
call_index = tool_call_chunk.index
if call_index not in tool_calls_res:
if tool_call_chunk.function.arguments == None:
tool_call_chunk.function.arguments = ''
tool_calls_res[call_index] = tool_call_chunk
else:
tool_calls_res[call_index].function.arguments += tool_call_chunk.function.arguments
print(tool_calls_res)
print("========finish========")
else:
print("reasoning_content:",chat_stream.choices[0].message.reasoning_content)
print("content:",chat_stream.choices[0].message.content)
print("tool_calls:",chat_stream.choices[0].message.tool_calls)
print("function_call:",chat_stream.choices[0].message.function_call)Input:What's the temperature in Beijing today? What's the temperature in Guangzhou on October 1st?
Output
We are going to use the `weatherstation` package to get the temperature data.
However, note that the user is asking for the current temperature in Beijing and the temperature in Guangzhou on October 1st.
We don't have historical data for Beijing, but we do have current weather data for many stations. But note: the question asks for Beijing today and Guangzhou on October 1st.
Since we don't have historical data for Beijing, we can only answer the first part (Beijing today) with current data.
For the second part (Guangzhou on October 1st), we need to know the date and station. We are going to set the date to today's date, but note that October 1st is a specific date.
However, note: the current date is when the user is asking. We must be cautious because the data might not be available for historical dates.
Steps:
1. We'll get the current weather for Beijing.
2. We'll get the weather data for Guangzhou for the specific date of October 1st, but we must set the date to October 1st of the current year? But note: the user might be asking for a past date that falls already in the past.
But note: the problem is that if we are in October, then October 1st is in the past. We cannot get historical weather data for free from the `weatherstation` package.
Let me check the `weatherstation` package:
- The `weatherstation` package from MeteoWorldCard! is a library that accesses the weather data from www.meteoworld.com.
However, the terms of service might not allow us to do that.
Actually, the package's main purpose is for current weather. For historical data, we might need to use a different source.
Since we cannot get historical data without a time period, we have to reconsider.
Alternatively, we can use the `pytz` and `pystee` for timezones? But the problem says today's temperature.
Let me re-read the question:
"What's the temperature in Beijing today? What's the temperature in Guangzhou on October 1 st?"
For Beijing today: we can get today's temperature in Beijing? But note: the `weatherstation` package does not provide a way to get past today's data? Actually, the package is for current observations.
How about we use an API that provides current and historical data? But the `weatherstation` package might not support historical.
Alternatively, we can use an alternative source. However, the problem says to use the `weatherstation` package.
Since we don't have a better idea, we'lll need to use an external source. But the instructions says to use the `weatherstation` package.
Let me check the documentation of `weatherstation`:
We don't have to install, but the package might be designed for current only.
Actually, the package's functions are:
- `current_extrainfo`
- `today_extra`
But note: the `weatherstation` package might not have historical data.
How about we use the `weather` package under the `pywmo_uib`? Not sure.
Alternatively, let's use the `meteo` package? But the problem says `weatherstation`.
Since we are not able to get historical data, we have to reconsider the approach.
How about we get the current weather in Beijing and then for Guangzhou on October 1st? We can't because that's in the past.
We must admit that we cannot answer the second part because of the data limitation.
But note: the problem says "What's the temperature in Guangzhou on October 1 st?" - that could be a fixed date? Or we can check the weather in Guangzhou on that day? But that day has already passed, so we don't have it.
Alternatively, we can use the `weatherstation` package to get the current weather and then try to get October 1st? But it won't work.
Since we are stuck, let me look for an alternative:
We can use the `weatherstation` package to get the weather in a particular station in Guangzhou for today, and then we adjust the date to October 1st? But that would be today, not October 1st.
How about we get the weather in Guangzhou for today and then for October 1st we can't get.
So we have two options:
1. We get the temperature for Beijing today (current) and then for Guangzhou on October 1st, but we can only do it if we have a way to get historical data? But the `weatherstation` package doesn't provide that.
After reading the documentation of the `weatherstation` package, I see that it requires a station identifier. We can get the station identifiers for Beijing and Guangzhou, but then we can't get past data.
Therefore, we must conclude that we cannot get the temperature for October 1st because it is in the past and we don't have historical data.
But wait, the problem says: "What's the temperature in Guangzhou on October 1 st?"
How about we get the weather report for October 1st? We can try to get the weather station's history? But the `weatherstation` package doesn't provide that.
Alternatively, we can use the `datetime` to generate a query for October 1st? But we don't have an API for historical data.
So we have to answer only the first part:
- Beijing today: we can get using the current weather station in Beijing.
- For Guangzhou on October 1st: we cannot get it because we don't have historical data.
But note: the user might be asking for October 1st of this year, which has already happened. So we cannot get it.
How about we use an alternative source? We are allowed to use any source. But the problem says to use the `weatherstation` package.
Since we cannot, we must return an error for the second part.
However, let me check if there is a way:
- The `weatherstation` package might have a function that provides the last few days? I doubt.
.......... **Continuously outputting**
✅ commit 90b139c before PR #23991
========reasoning_content========
The user is asking about the current temperature in Beijing and also wants to know the temperature in Guangzhou on October 1st. Let me start by breaking down the query into two parts.
First, for the current temperature in Beijing. Since I need to provide real-time data, I should check the latest weather reports. However, since my knowledge is up to October 2023, I can't access real-time information. But maybe the user is referring to a specific date, or perhaps they want to know how to find it themselves. Wait, the user mentioned "today" for Beijing, so that would be the current date. But without real-time access, I can't give an exact number. I should explain that I can't provide live data but suggest checking a reliable source like Weather.com or a local weather app.
Next, the temperature in Guangzhou on October 1st. Again, my knowledge cutoff is October 2023, so I can recall general weather patterns for that time. October in Guangzhou is typically autumn, so the temperature would be mild to cool, maybe around 20-25°C (68-77°F) during the day, with cooler evenings. But I should note that this is an estimate based on historical data and might not be exact. Encouraging the user to check a specific weather service for precise figures would be helpful.
I also need to consider why the user is asking. They might be planning a trip, comparing climates, or just curious. Providing both current and historical data could be useful. However, since I can't access real-time data, I must be clear about the limitations. Maybe the user wants to know if they need a jacket in Beijing today or if they should pack for cooler weather in Guangzhou in early October.
I should structure the answer to first address the inability to provide real-time Beijing temps, then give a general October forecast for Guangzhou. Including a tip to use a weather service for accuracy would add value. Also, checking if there's a specific date in mind beyond October 1st, but since the user specified that date, I'll proceed with that.
Wait, the user's question is in Chinese, but the assistant's response should be in English. The original query is in English, so the answer should be in English as well. Make sure the response is clear and helpful, even with the limitations.
========content========
<response>
I can't provide real-time weather data, but I can share general trends based on historical climate information:
1. **Beijing Today**:
As of my last update (October 2023), Beijing typically experiences mild to cool weather in autumn. Current real-time data would be best obtained via a reliable weather app or website.
2. **Guangzhou on October 1st**:
Guangzhou in early October usually has mild, pleasant weather, with daytime temperatures around 20–25°C (68–77°F) and cooler evenings. However, exact conditions depend on the specific year and year-end climate patterns. For precise details, check a weather forecast platform like [Weather.com](https://weather.com) or a local Chinese weather service.
For the most accurate and up-to-date information, I recommend using a weather app or visiting a trusted weather website.
</response>
========tool_calls========
{}
========finish========
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.