The Model Router feature within Azure OpenAI is a deployable model trained to select the best large language model (LLM) to respond to a given prompt in real time. By assessing factors such as prompt complexity, cost, and performance, Model Router dynamically routes requests to the most suitable underlying model. This approach helps you optimize both cost and quality: smaller, more efficient models are used when possible, while larger, more capable models are chosen when needed.
Key features of the Model Router include:
- Intelligent model selection for optimal cost and quality
- Transparent versioning with fixed underlying model sets per release
- Support for vision-enabled chats (image input)
- No additional charges for routing—you're billed only for the usage of underlying models
For more details, see the official documentation.
This repository provides ready-to-use Python samples demonstrating how to interact with the Model Router feature in Azure OpenAI. The sample model-router-foundrysample.py is based on the getting started code from Azure AI Foundry. The sample model-router-gradio.py provides a Gradio front end to the sample, and exposes the chosen model at the top.
-
Clone this repository:
git clone https://github.com/guygregory/ModelRouter.git cd ModelRouter -
Install dependencies: Most samples require
azure-ai-openaiand related libraries. Install with:pip install -r requirements.txt
-
Configure your Azure credentials:
- Set the
AZURE_OPENAI_API_KEYandAZURE_OPENAI_API_ENDPOINTenvironment variables as described in the Azure OpenAI documentation.
- Set the
-
Run a sample:
- Review the individual Python scripts in this repo, and use whichever is the best fit for your need:
For example:
python model-router-gradio.py
-
Review and adapt:
- Modify the samples to fit your use case and deployment.