Current Behavior
开启日志记录:
nginx_config:
http:
access_log_format: "$remote_addr - $remote_user [$time_local] $http_host "$request_line" $status $body_bytes_sent $request_time "$http_referer" "$http_user_agent" $upstream_addr $upstream_status $apisix_upstream_response_time "$upstream_scheme://$upstream_host$upstream_uri" "$apisix_request_id" "$request_type" "$llm_time_to_first_token" "$llm_model" "$request_llm_model" "$llm_prompt_tokens" "$llm_completion_tokens""
使用ai-proxy-multi创建路由 开启fallback_strategy, 发生重试后$request_llm_model不是原始请求的model
创建路由的curl:
curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
-H "X-API-KEY: ${admin_key}" \
-d '{
"id": "ai-proxy-multi-route",
"uri": "/v1/chat/completions",
"methods": ["POST"],
"vars": [
[
"post_arg.model",
"==",
"model_test"
]
],
"plugins": {
"ai-proxy-multi": {
"instances": [
{
"provider": "openai-compatible",
"options": {
"model": "upstream-model-A"
},
"override": {
"endpoint": "http://172.0.0.1:8081/test-upstream/v1/chat/completions"
},
"priority": 1,
"name": "test1",
"auth": {},
"weight": 1
},
{
"provider": "openai-compatible",
"options": {
"model": "upstream-model-B"
},
"override": {
"endpoint": "http://172.0.0.1:8081/test-upstream/v1/chat/completions"
},
"priority": 0,
"name": "test2",
"auth": {},
"weight": 1
}
],
"balancer": {
"hash_on": "consumer",
"algorithm": "chash"
},
"timeout": 500,
"fallback_strategy": [
"http_429",
"http_5xx"
],
"logging": {
"summaries": true,
"payloads": true
},
"keepalive": false
},
}
}'
步骤:
发送请求:{
"model": "model_test",
"messages": [
{
"role": "user",
"content": "返回1"
}
]
}
上游正常返回 $request_llm_model 输出的是请求中的model参数 model_test $llm_model输出 upstream-model-A 符合预期

发生重试后 $request_llm_model 输出为 upstream-model-A $llm_model输出 upstream-model-B 不符合预期 $request_llm_model应还是输出 model_test :
Expected Behavior
$request_llm_model 应是保持原始的模型名称 重试不应该改变, 即使为了兼容不使用 "vars": [["post_arg.model","==","model_test"],请求参数不带model 也应该在赋值的时候判断下 是否为空 如果不为空的情况下不重新赋值
Error Logs
重试会调用base.before_proxy的方法重试
if request_model then
ctx.var.request_llm_model = request_model
end
赋值的时候需要添加判断是否已经赋过值了
Steps to Reproduce
步骤如描述中
Environment
- APISIX version (run
apisix version): 3.17.0
- Operating system (run
uname -a):
- OpenResty / Nginx version (run
openresty -V or nginx -V):
- etcd version, if relevant (run
curl http://127.0.0.1:9090/v1/server_info):
- APISIX Dashboard version, if relevant:
- Plugin runner version, for issues related to plugin runners:
- LuaRocks version, for installation issues (run
luarocks --version):
Current Behavior
开启日志记录:
nginx_config:
http:
access_log_format: "$remote_addr - $remote_user [$time_local] $http_host "$request_line" $status $body_bytes_sent $request_time "$http_referer" "$http_user_agent" $upstream_addr $upstream_status $apisix_upstream_response_time "$upstream_scheme://$upstream_host$upstream_uri" "$apisix_request_id" "$request_type" "$llm_time_to_first_token" "$llm_model" "$request_llm_model" "$llm_prompt_tokens" "$llm_completion_tokens""
使用ai-proxy-multi创建路由 开启fallback_strategy, 发生重试后$request_llm_model不是原始请求的model
创建路由的curl:
步骤:
发送请求:{
"model": "model_test",
"messages": [
{
"role": "user",
"content": "返回1"
}
]
}
上游正常返回 $request_llm_model 输出的是请求中的model参数 model_test $llm_model输出 upstream-model-A 符合预期


发生重试后 $request_llm_model 输出为 upstream-model-A $llm_model输出 upstream-model-B 不符合预期 $request_llm_model应还是输出 model_test :
Expected Behavior
$request_llm_model 应是保持原始的模型名称 重试不应该改变, 即使为了兼容不使用 "vars": [["post_arg.model","==","model_test"],请求参数不带model 也应该在赋值的时候判断下 是否为空 如果不为空的情况下不重新赋值
Error Logs
重试会调用base.before_proxy的方法重试
if request_model then
ctx.var.request_llm_model = request_model
end
赋值的时候需要添加判断是否已经赋过值了
Steps to Reproduce
步骤如描述中
Environment
apisix version): 3.17.0uname -a):openresty -Vornginx -V):curl http://127.0.0.1:9090/v1/server_info):luarocks --version):