Update vLLM-SR RouterArena submission

[Xunzhuo]

Summary

Update the vllm-sr RouterArena submission artifacts for the vLLM Semantic Router

This submission updates:

• router_inference/config/vllm-sr.json
• router_inference/predictions/vllm-sr.json
• router_inference/predictions/vllm-sr-robustness.json

Notes

• The router is not trained, fit, or tuned on RouterArena data.
• The routing policy is a general vLLM Semantic Router recipe using deterministic signals/projections; it does not encode RouterArena sample IDs, gold answers, or generated-result lookup tables.
• Full prediction generated_result fields are populated for all 8,400 regular entries with success=true.
• Robustness predictions include 420 entries; per README, no robustness generated_result fields are required.
• The vLLM Semantic Router service used for generation was served on AMD

[Xunzhuo]

/evaluate

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Updates the vLLM Semantic Router configuration to use a new RouterArena “recipe” with a refreshed model list and adds an explicit description, while removing endpoint and category-mapping fields.

Changes:

• Replaces the previous model set with updated provider/model identifiers and changes the default model.
• Removes router_endpoint, base_url, and category_model_mapping from the config.
• Adds a descriptive description field clarifying the recipe and data-embedding constraints.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

Router Evaluation Results

Router: vllm-sr
Dataset Split: full

RouterArena Metrics

Metric	Value
RouterArena Score	0.7420
Accuracy	75.07%
Total Cost	$1.340293
Avg Cost per Query	$0.000160
Avg Cost per 1K Queries	$0.1596
Number of Queries	8400
Abnormal Entries	0
Robustness Score	0.7690

Optimality Metrics

Metric	Value
Opt.Sel (Optimal Selection)	0.1809
Opt.Cost (Cost Efficiency)	0.2407
Opt.Acc (Accuracy vs Optimal)	0.8969

---

Evaluation completed by RouterArena automated workflow

[Xunzhuo]

/evaluate

[github-actions]

Router Evaluation Results

Router: vllm-sr
Dataset Split: full

RouterArena Metrics

Metric	Value
RouterArena Score	0.7538
Accuracy	75.97%
Total Cost	$0.921463
Avg Cost per Query	$0.000110
Avg Cost per 1K Queries	$0.1097
Number of Queries	8400
Abnormal Entries	0
Robustness Score	0.7310

Optimality Metrics

Metric	Value
Opt.Sel (Optimal Selection)	0.2012
Opt.Cost (Cost Efficiency)	0.2452
Opt.Acc (Accuracy vs Optimal)	0.8987

---

Evaluation completed by RouterArena automated workflow