Skip to content

Commit

Permalink
Addition of deployment templates and scripts (microsoft#704)
Browse files Browse the repository at this point in the history
### Motivation and Context
We want to make it easy to set up and use the Copilot Chat in a cloud setting.

### Description
- Added bicep deployment templates and scripts
- Added related documentation
- Added ARM template equivalents to the bicep files (only ARM is supported for "Deploy to Azure" buttons)
  • Loading branch information
glahaye authored and codebrain committed May 16, 2023
1 parent 891e935 commit 326c827
Show file tree
Hide file tree
Showing 10 changed files with 1,295 additions and 1 deletion.
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
<#
.SYNOPSIS
Creates a Semantic Kernel service deployment using an existing OpenAI or Azure OpenAI instance.
#>

param(
[Parameter(Mandatory)]
[string]
# Name for the deployment
$DeploymentName,

[Parameter(Mandatory)]
[SecureString]
# Azure OpenAI or OpenAI API key
$ApiKey,

[string]
# Azure OpenAI endpoint to use (ignored when AI service is not AzureOpenAI
$Endpoint = "",

[ValidateSet("AzureOpenAI", "OpenAI")]
[string]
# Underlying AI service (AzureOpenAI or OpenAI)
$AIService = "AzureOpenAI",

[string]
# Model to use for chat completions
$CompletionModel = "gpt-35-turbo",

[string]
# Model to use for text embeddings
$EmbeddingModel = "text-embedding-ada-002",

[Parameter(Mandatory)]
[string]
# Subscription to which to make the deployment
$Subscription,

[string]
# Resource group to which to make the deployment
$ResourceGroup = "",

[string]
# Region to which to make the deployment (ignored when deploying to an existing resource group)
$Region = "South Central US",

[string]
# Package to deploy to web service
$PackageUri = 'https://skaasdeploy.blob.core.windows.net/api/skaas.zip',

[string]
# SKU for the Azure App Service plan
$AppServiceSku = "B1",

[switch]
# Switches on verbose template deployment output
$DebugDeployment
)

$ErrorActionPreference = "Stop"

$templateFile = "$($PSScriptRoot)/sk-existing-ai.bicep"

if (!$ResourceGroup)
{
$ResourceGroup = $DeploymentName + "-rg"
}

Write-Host "Log into your Azure account"
az login | out-null

az account set -s $Subscription

Write-Host "Creating resource group $($ResourceGroup) if it doesn't exist..."
az group create --location $Region --name $ResourceGroup --tags Creator=$env:UserName
if ($LASTEXITCODE -ne 0) {
exit $LASTEXITCODE
}

Write-Host "Validating template file..."
az deployment group validate --name $DeploymentName --resource-group $ResourceGroup --template-file $templateFile --parameters name=$DeploymentName packageUri=$PackageUri aiService=$AIService completionModel=$CompletionModel embeddingModel=$EmbeddingModel endpoint=$Endpoint apiKey=$ApiKey appServiceSku=$AppServiceSku
if ($LASTEXITCODE -ne 0) {
exit $LASTEXITCODE
}

Write-Host "Deploying..."
if ($DebugDeployment) {
az deployment group create --name $DeploymentName --resource-group $ResourceGroup --template-file $templateFile --debug --parameters name=$DeploymentName packageUri=$PackageUri aiService=$AIService completionModel=$CompletionModel embeddingModel=$EmbeddingModel endpoint=$Endpoint apiKey=$ApiKey appServiceSku=$AppServiceSku
}
else {
az deployment group create --name $DeploymentName --resource-group $ResourceGroup --template-file $templateFile --parameters name=$DeploymentName packageUri=$PackageUri aiService=$AIService completionModel=$CompletionModel embeddingModel=$EmbeddingModel endpoint=$Endpoint apiKey=$ApiKey appServiceSku=$AppServiceSku
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
#!/bin/bash

# Creates a Semantic Kernel service deployment using an existing OpenAI or Azure OpenAI instance.

set -e

# Parameters
DeploymentName="$1"
Subscription="$2"
ApiKey="$3"
Endpoint="${4:-}"
AIService="${5:-AzureOpenAI}"
CompletionModel="${6:-gpt-35-turbo}"
EmbeddingModel="${7:-text-embedding-ada-002}"
ResourceGroup="${8:-}"
Region="${9:-South Central US}"
PackageUri="${10:-https://skaasdeploy.blob.core.windows.net/api/skaas.zip}"
AppServiceSku="${11:-B1}"
DebugDeployment="${12:-false}"

if [ -z "$ResourceGroup" ]; then
ResourceGroup="${DeploymentName}-rg"
fi

templateFile="$(dirname "$0")/sk-existing-ai.bicep"

az login --use-device-code

az account set -s "$Subscription"

echo "Creating resource group $ResourceGroup if it doesn't exist..."
az group create --location "$Region" --name "$ResourceGroup" --tags Creator="$USER"

echo "Validating template file..."
az deployment group validate --name "$DeploymentName" --resource-group "$ResourceGroup" --template-file "$templateFile" --parameters name="$DeploymentName" packageUri="$PackageUri" aiService="$AIService" completionModel="$CompletionModel" embeddingModel="$EmbeddingModel" endpoint="$Endpoint" apiKey="$ApiKey" appServiceSku="$AppServiceSku"

echo "Deploying..."
if [ "$DebugDeployment" = "true" ]; then
az deployment group create --name "$DeploymentName" --resource-group "$ResourceGroup" --template-file "$templateFile" --debug --parameters name="$DeploymentName" packageUri="$PackageUri" aiService="$AIService" completionModel="$CompletionModel" embeddingModel="$EmbeddingModel" endpoint="$Endpoint" apiKey="$ApiKey" appServiceSku="$AppServiceSku"
else
az deployment group create --name "$DeploymentName" --resource-group "$ResourceGroup" --template-file "$templateFile" --parameters name="$DeploymentName" packageUri="$PackageUri" aiService="$AIService" completionModel="$CompletionModel" embeddingModel="$EmbeddingModel" endpoint="$Endpoint" apiKey="$ApiKey" appServiceSku="$AppServiceSku"
fi
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
<#
.SYNOPSIS
Creates a Semantic Kernel service deployment.
#>

param(
[Parameter(Mandatory)]
[string]
# Name for the deployment
$DeploymentName,

[Parameter(Mandatory)]
[string]
# Subscription to which to make the deployment
$Subscription,

[string]
# Resource group to which to make the deployment
$ResourceGroup = "",

[string]
# Region to which to make the deployment (ignored when deploying to an existing resource group)
$Region = "South Central US",

[string]
# Package to deploy to web service
$PackageUri = 'https://skaasdeploy.blob.core.windows.net/api/skaas.zip',

[string]
# SKU for the Azure App Service plan
$AppServiceSku = "B1",

[switch]
# Switches on verbose template deployment output
$DebugDeployment
)

$ErrorActionPreference = "Stop"

$templateFile = "$($PSScriptRoot)/sk.bicep"

if (!$ResourceGroup)
{
$ResourceGroup = $DeploymentName + "-rg"
}

Write-Host "Log into your Azure account"
az login | out-null

az account set -s $Subscription

Write-Host "Creating resource group $($ResourceGroup) if it doesn't exist..."
az group create --location $Region --name $ResourceGroup --tags Creator=$env:UserName
if ($LASTEXITCODE -ne 0) {
exit $LASTEXITCODE
}

Write-Host "Validating template file..."
az deployment group validate --name $DeploymentName --resource-group $ResourceGroup --template-file $templateFile --parameters name=$DeploymentName packageUri=$PackageUri appServiceSku=$AppServiceSku
if ($LASTEXITCODE -ne 0) {
exit $LASTEXITCODE
}

Write-Host "Deploying..."
if ($DebugDeployment) {
az deployment group create --name $DeploymentName --resource-group $ResourceGroup --template-file $templateFile --debug --parameters name=$DeploymentName packageUri=$PackageUri appServiceSku=$AppServiceSku
}
else {
az deployment group create --name $DeploymentName --resource-group $ResourceGroup --template-file $templateFile --parameters name=$DeploymentName packageUri=$PackageUri appServiceSku=$AppServiceSku
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
#!/bin/bash

# Creates a Semantic Kernel service deployment.

set -e

# Parameters
DeploymentName="$1"
Subscription="$2"
ResourceGroup="${3:-}"
Region="${4:-South Central US}"
PackageUri="${5:-https://skaasdeploy.blob.core.windows.net/api/skaas.zip}"
AppServiceSku="${6:-B1}"
DebugDeployment="$7"

templateFile="$(dirname "$0")/sk.bicep"

if [ -z "$ResourceGroup" ]; then
ResourceGroup="${DeploymentName}-rg"
fi

az login --use-device-code

az account set -s "$Subscription"

echo "Creating resource group $ResourceGroup if it doesn't exist..."
az group create --location "$Region" --name "$ResourceGroup" --tags Creator="$USER"

echo "Validating template file..."
az deployment group validate --name "$DeploymentName" --resource-group "$ResourceGroup" --template-file "$templateFile" --parameters name="$DeploymentName" packageUri="$PackageUri" appServiceSku="$AppServiceSku"

echo "Deploying..."
if [ -n "$DebugDeployment" ]; then
az deployment group create --name "$DeploymentName" --resource-group "$ResourceGroup" --template-file "$templateFile" --debug --parameters name="$DeploymentName" packageUri="$PackageUri" appServiceSku="$AppServiceSku"
else
az deployment group create --name "$DeploymentName" --resource-group "$ResourceGroup" --template-file "$templateFile" --parameters name="$DeploymentName" packageUri="$PackageUri" appServiceSku="$AppServiceSku"
fi
128 changes: 128 additions & 0 deletions samples/apps/copilot-chat-app/webapi/DeploymentTemplates/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
# Deploying Semantic Kernel to Azure in a web app service

## Things to know

Azure currently limits the number of OpenAI resources per region per subscription to 3. Also, OpenAI is not available in every region.
(Refer to this [availability map](https://azure.microsoft.com/en-us/explore/global-infrastructure/products-by-region/?products=cognitive-services))
Bearing this in mind, you might want to use the same Azure OpenAI instance for multiple deployments of Semantic Kernel to Azure.

To do so, or to use an OpenAI instance from [openai.com](https://openai.com), use the version of the deployment template that uses existing OpenAI resources.

Either way, you also need to have the necessary permissions to create resources in the target subscription.

Also note that the F1 and D1 App Service SKU's (the Free and Shared ones) are not supported for this deployment.

## Deploying with a new Azure OpenAI instance

You can deploy an instance of Semantic Kernel in a web app service within a resource group that bears the name YOUR_DEPLOYMENT_NAME preceded by the "rg-" prefix using any of the following methods.

### PowerShell

Use the [DeploySK.ps1](DeploySK.ps1) file found in this folder:
```powershell
.\DeploySK.ps1 -DeploymentName YOUR_DEPLOYMENT_NAME -Subscription YOUR_SUBSCRIPTION_ID
```

For more options, see the deployment script.

### Bash

After ensuring DeploySK.sh file found in this folder is executable, enter the following command:

```bash
./DeploySK.sh YOUR_DEPLOYMENT_NAME YOUR_SUBSCRIPTION_ID
```

### Azure Portal

Alternatively, you can deploy by clicking on the following button:

[![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fglahaye%2Fsemantic-kernel%2Fdeploy%2Fsamples%2Fapps%2Fcopilot-chat-app%2Fwebapi%2FDeploymentTemplates%2Fsk.json)

## Deploying with an existing OpenAI / Azure OpenAI instance

Note that, with the following methods, the Azure OpenAI endpoint is ignored and can be omitted when you are using an OpenAI instance from [openai.com](https://openai.com).

### PowerShell

Use the [DeploySK-Existing-AI.ps1](DeploySK-Existing-AI.ps1) file found in this folder:
```powershell
.\DeploySK-Existing-AI.ps1 -DeploymentName YOUR_DEPLOYMENT_NAME -Subscription YOUR_SUBSCRIPTION_ID -Endpoint "YOUR_AZURE_OPENAI_ENDPOINT"
```

After entering the command above, you will be prompted to enter your OpenAI or Azure OpenAI API key. (You can also pass in the API key using the -ApiKey parameter followed by a SecureString)

### bash

After ensuring DeploySK-Existing-AI.sh file found in this folder is executable, enter the following command:

```bash
./DeploySK-Existing-AI.sh YOUR_DEPLOYMENT_NAME YOUR_SUBSCRIPTION_ID YOUR_API_KEY "YOUR_AZURE_OPENAI_ENDPOINT"
```

### Azure Portal

Alternatively, you can deploy by clicking on the following button:

[![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fglahaye%2Fsemantic-kernel%2Fdeploy%2Fsamples%2Fapps%2Fcopilot-chat-app%2Fwebapi%2FDeploymentTemplates%2Fsk-existing-ai.json)

## Verifying the deployment

To make sure your web app service is running, go to <!-- markdown-link-check-disable -->https://YOUR_INSTANCE_NAME.azurewebsites.net/probe<!-- markdown-link-check-enable-->

To get your instance's URL, click on the "Go to resource group" button you see at the end of your deployment. Then click on the resource whose name starts with "app-".

This will bring you to the Overview page on your web service. Your instance's URL is the value that appears next to the "Default domain" field.

## Changing your configuration, monitoring your deployment and troubleshooting

From the page just mentioned in the section above, you can change your configuration by clicking on the "Configuration" item in the "Settings" section of the left pane.

Scrolling down in that same pane to the "Monitoring" section gives you access to a multitude of ways to monitor your deployment.

In addition to this, the "Diagnose and "solve problems" item near the top of the pane can yield crucial insight into some problems your deployment may be experiencing.

If the service itself if functioning properly but you keep getting errors (perhaps reported as 400 HTTP errors) when making calls to the Semantic Kernel,
check that you have correctly entered the values for the following settings:
- Completion:AzureOpenAI
- Completion:DeploymentOrModelId
- Completion:Endpoint
- Completion:Label
- Embedding:AzureOpenAI
- Embedding:DeploymentOrModelId
- Embedding:Endpoint
- Embedding:Label

Both Completion:Endpoint and Embedding:Endpoint are ignored for OpenAI instances from [openai.com](https://openai.com) but MUST be properly populated when using Azure OpenAI instances.

## Using web frontends to access your deployment

Make sure to include your frontend's URL as an allowed origin in your deployment's CORS settings. Otherwise, web browsers will refuse to let JavaScript make calls to your deployment.

# Deploying your custom version of Semantic Kernel

You can build and upload a customized version of the Semantic Kernel service.

To do so, clone the code from this repo then modify it to your needs (for example, by adding your own skills). Once that is done, go into the ../semantic-kernel/samples/apps/copilot-chat-app/webapi
directory and enter the following command:
```powershell
dotnet publish CopilotChatApi.csproj --configuration Release --arch x64 --os win
```

This will create the following directory, which will contain all the files needed for a deployment:
../semantic-kernel/samples/apps/copilot-chat-app/webapi/bin/Release/net6.0/win-x64/publish

Zip the contents of that directory then put the resulting zip file on the web.

Put its URI in the "Package Uri" field in the web deployment page you access through the "Deploy to Azure" buttons above, or use its URI as the value for the PackageUri parameter of the Powershell scripts above.

Your deployment will then use your customized deployment package.


## Cleaning up

Once you are done with your resources, you can delete them from the Azure portal. You can also simply delete the resource group in which they are from the portal or through the
following [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/) command:
```powershell
az group delete --name YOUR_RESOURCE_GROUP
```

0 comments on commit 326c827

Please sign in to comment.