|
1 | | -# Setup your Environment (skip is using a Lab environment) |
| 1 | +# Setup your Environment |
2 | 2 |
|
3 | | -## 1. Local Developer env Setup |
4 | | -- Install required tools and libraries |
5 | | -- Clone the prebuilt app repository |
6 | | -- Set up your development environment |
| 3 | +## Running this sample |
7 | 4 |
|
8 | | -## 2. Provision Azure Infrastructure |
9 | | -- Setting up Azure Landing Zones for infrastructure |
| 5 | +We'll follow 4 steps to get this example running in your own environment: pre-requisites, creating an index, setting up the environment, and running the app. |
| 6 | + |
| 7 | +### Pre-requisites |
| 8 | + |
| 9 | +You'll need instances of the following Azure services. You can re-use service instances you have already or create new ones. |
| 10 | + |
| 11 | +1. [Azure OpenAI](https://ms.portal.azure.com/#create/Microsoft.CognitiveServicesOpenAI), with 2 model deployments, one of the **gpt-4o-realtime-preview** model, a regular **gpt-4o-mini** model. |
| 12 | +1. [Optional] Train an intent_detection model with a SLM using Azure AI Studio. Check [the training data](./intent_detection_model) |
| 13 | + |
| 14 | +### Setting up the environment |
| 15 | + |
| 16 | +The app needs to know which service endpoints to use for the Azure OpenAI and Azure AI Search. The following variables can be set as environment variables, or you can create a ".env" file in the "app/backend/" directory with this content. |
| 17 | + |
| 18 | +The voice agent can use a fine-tuned SLM deployment to classify intent to minimize latency. If you do not have this deployment available then you can use the Azure OpenAI **gpt-4o-mini** deployment, which is fast enough to classify intent with minimal impact on latency. To use **gpt-4o-mini** leave the `INTENT_SHIFT_API_*` env variables empty and supply `AZURE_OPENAI_4O_MINI_DEPLOYMENT`. |
| 19 | + |
| 20 | +```bash |
| 21 | +AZURE_OPENAI_ENDPOINT="https://.openai.azure.com/" |
| 22 | +AZURE_OPENAI_API_KEY= |
| 23 | +AZURE_OPENAI_API_VERSION=2024-10-01-preview |
| 24 | +AZURE_OPENAI_CHAT_DEPLOYMENT=gpt-4o-mini |
| 25 | +AZURE_OPENAI_EMB_DEPLOYMENT="text-embedding-ada-002" |
| 26 | +AZURE_OPENAI_EMB_ENDPOINT= [Optional] if different from your realtime endpoint |
| 27 | +AZURE_OPENAI_EMB_API_KEY= [Optional] if providing an embedding endpoint |
| 28 | +AZURE_OPENAI_4O_MINI_DEPLOYMENT=YOUR_AZURE_OPENAI_4O_MINI_DEPLOYMENT_NAME |
| 29 | +INTENT_SHIFT_API_KEY= |
| 30 | +INTENT_SHIFT_API_URL=https://YOUR_ML_DEPLOYMENT.westus2.inference.ml.azure.com/score |
| 31 | +INTENT_SHIFT_API_DEPLOYMENT=YOUR_ML_DEPLOYMENT_NAME |
| 32 | +AZURE_OPENAI_API_VERSION=2024-10-01-preview |
| 33 | +AZURE_OPENAI_REALTIME_DEPLOYMENT_NAME=gpt-4o-realtime-preview |
| 34 | +``` |
| 35 | +
|
| 36 | +### Running the app |
| 37 | +
|
| 38 | +#### GitHub Codespaces |
| 39 | +
|
| 40 | +You can run this repo virtually by using GitHub Codespaces, which will open a web-based VS Code in your browser: |
| 41 | +
|
| 42 | +[](https://github.com/codespaces/new?hide_repo_select=true&ref=main&skip_quickstart=true&machine=basicLinux32gb&repo=840462613&devcontainer_path=.devcontainer%2Fdevcontainer.json&geo=WestUs2) |
| 43 | +
|
| 44 | +Once the codespace opens (this may take several minutes), open a new terminal. |
| 45 | +
|
| 46 | +#### VS Code Dev Containers |
| 47 | +
|
| 48 | +You can run the project in your local VS Code Dev Container using the [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers): |
| 49 | +
|
| 50 | +1. Start Docker Desktop (install it if not already installed) |
| 51 | +2. Open the project: |
| 52 | +
|
| 53 | + [](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/microsoft/multi-modal-customer-service-agent) |
| 54 | +3. In the VS Code window that opens, once the project files show up (this may take several minutes), open a new terminal. |
| 55 | +
|
| 56 | +#### Local environment |
| 57 | +
|
| 58 | +1. Install the required tools: |
| 59 | + - [Node.js](https://nodejs.org/en) |
| 60 | + - [Python >=3.11](https://www.python.org/downloads/) |
| 61 | + - **Important**: Python and the pip package manager must be in the path in Windows for the setup scripts to work. |
| 62 | + - **Important**: Ensure you can run `python --version` from console. On Ubuntu, you might need to run `sudo apt install python-is-python3` to link `python` to `python3`. |
| 63 | + - [Powershell](https://learn.microsoft.com/powershell/scripting/install/installing-powershell) |
| 64 | +
|
| 65 | +1. Clone the repo (`git clone https://github.com/microsoft/multi-modal-customer-service-agent`) |
| 66 | +1. Ensure env variables are set per [Setting up the environment](#2-setting-up-the-environment) |
| 67 | +1. Run this command to start the app: |
| 68 | +
|
| 69 | + Windows: |
| 70 | +
|
| 71 | + ```pwsh |
| 72 | + cd voice_agent\app |
| 73 | + pwsh .\start.ps1 |
| 74 | + ``` |
| 75 | +
|
| 76 | + Linux/Mac: |
| 77 | +
|
| 78 | + ```bash |
| 79 | + cd voice_agent/app |
| 80 | + ./start.sh |
| 81 | + ``` |
| 82 | +
|
| 83 | +1. The app is available on http://localhost:8765 |
| 84 | +
|
| 85 | +#### Deploy to Azure using azd |
| 86 | +
|
| 87 | +1. Download and install [azd](https://aka.ms/azd/install) (Azure Developer CLI) |
| 88 | +
|
| 89 | +1. Change directory to: |
| 90 | +
|
| 91 | + ```bash |
| 92 | + cd voice_agent/app |
| 93 | + ``` |
| 94 | +
|
| 95 | +1. Execute this command: |
| 96 | +
|
| 97 | + ```bash |
| 98 | + azd up |
| 99 | + ``` |
| 100 | +
|
| 101 | +1. When prompted provided the following names: |
| 102 | + 1. Environment name (used as the name of the new resource group to deploy into) |
| 103 | + 1. Azure Subscription to use |
| 104 | + 1. Azure location to use |
10 | 105 |
|
11 | 106 | --- |
12 | | -#### Navigation: [Home](../../README.md) | [Previous Section](../01_architecture/README.md) | [Next Section](../03_build/README.md) |
| 107 | +#### Navigation: [Home](../../README.md) | [Previous Section](../01_architecture/README.md) | [Next Section](../03_observability/README.md) |
0 commit comments